Page MenuHomeFreeBSD

vmd(4): Major driver refactoring
ClosedPublic

Authored by mav on Aug 31 2021, 10:36 PM.
Tags
None
Referenced Files
F82181605: D31762.id94594.diff
Fri, Apr 26, 6:11 AM
F82166342: D31762.id94459.diff
Fri, Apr 26, 3:15 AM
F82166318: D31762.id.diff
Fri, Apr 26, 3:15 AM
Unknown Object (File)
Thu, Apr 25, 1:13 AM
Unknown Object (File)
Sun, Apr 21, 9:38 PM
Unknown Object (File)
Sun, Apr 21, 9:27 PM
Unknown Object (File)
Sun, Apr 21, 9:27 PM
Unknown Object (File)
Sun, Apr 21, 9:27 PM
Subscribers
None

Details

Summary
  • Re-implement pcib interface to use standard pci bus driver on top of vmd(4) instead of custom one.
  • Re-implement memory/bus resource allocation to properly handle even complicated configurations.
  • Re-implement interrupt handling to evenly distribute children's MSI/MSI-X interrupts between available vmd(4) MSI-X vectors and setup them to be handled by standard OS mechanisms with minimal overhead, except sharing when unavoidable.
Test Plan

Successfully tested on laptop with Intel Gen11 CPU (VMD device ID 0x9a0b) and single NVMe SSD, dual-booting with Windows 10.
Successfully tested on Supermicro X11DPI-NT motherboard with Xeon(R) Gold 6242R CPUs (VMD device ID 0x201d), simultaneously handling single NVMe SSD on one PCIe port and PLX bridge with 3 NVMe and 1 AHCI SSDs on another, including SSDs hot-plug and different interrupt settings.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

mav requested review of this revision.Aug 31 2021, 10:36 PM
mav created this revision.

Limit the end of bus numbers range considering available BAR0 size.

Provide VMD dma tag to all children devices, since host should see it as originator for all requests. It fixes operation with IOMMU enabled at least for directly connected devices. Devices behind a bridge still don't work right for me for some reason, despite receiving the same tag and IOMMU not reporting violations.

This revision was not accepted when it landed; it landed in state Needs Review.Sep 3 2021, 1:16 AM
This revision was automatically updated to reflect the committed changes.