Page MenuHomeFreeBSD

bhyve: add BAR handler list for passthru devices
ClosedPublic

Authored by corvink on May 24 2024, 11:34 AM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Sep 25, 6:29 PM
Unknown Object (File)
Sun, Sep 21, 5:09 AM
Unknown Object (File)
Thu, Sep 18, 2:14 PM
Unknown Object (File)
Mon, Sep 15, 6:23 AM
Unknown Object (File)
Sep 14 2025, 11:18 AM
Unknown Object (File)
Sep 14 2025, 7:04 AM
Unknown Object (File)
Sep 13 2025, 8:29 PM
Unknown Object (File)
Sep 12 2025, 10:47 AM
Subscribers

Details

Summary

We have to emulate some specific register of a BAR for passthru devices. Our
current use case are Intels integrated graphic devices. They mirror some of
their PCI config space into the MMIO space. Unfortunately, the Windows driver
reads from MMIO instead of PCI config space. For that reason, we have to trap
and emulate those register. Instead of implementing a quirk for this special
device, we're implementing a generic approach by using a list of trapped
register. That's much cleaner and can be reused. E.g. Nvidia GPUs mirror their
PCI config space in MMIO too and we can reuse it to trap the MSI-X table in the
future.

Note that the handling of this new list requires a larger patch. For that
reason, we split it into multiple commits. This means that the list isn't used
yet. This commit adds the callback on BAR reads and writes. Some subsequent
commit will add the trap for BAR regions and an interface to easily add
protected regions.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

usr.sbin/bhyve/pci_passthru.c
1193
1196

Do you need to check the return value?

1240
1243

Do you need to check the return value?

corvink added inline comments.
usr.sbin/bhyve/pci_passthru.c
1196

What should we do on error return? passthru_write is a void function.

1243

What should we do on error return? passthru_read returns the read value.

This revision is now accepted and ready to land.Jun 17 2025, 12:13 PM
usr.sbin/bhyve/pci_passthru.c
85

Why did you use vm_* types here?

usr.sbin/bhyve/pci_passthru.c
85

The next commit (https://reviews.freebsd.org/D45340) passes those values to vm_map_pptdev_mmio which uses vm_paddr_t as input.

Hmmm, while looking at the next commit, it doesn't handle vm_* types consistently. It converts those types to uint64_t which then passes it to vm_map_pptdev_mmio as vm_paddr_t.

  • use uint64_t types instead of vm_* types
  • add baridx parameter to read/write handler
    • this parameter can be used in the future to e.g. add a generic PCI config mirror in BAR space which is quite common according to QEMU [1]

[1] https://elixir.bootlin.com/qemu/v10.0.2/source/hw/vfio/pci-quirks.c#L186

This revision now requires review to proceed.Jun 27 2025, 7:39 AM
jhb added inline comments.
usr.sbin/bhyve/pci_passthru.c
1193

I do wonder if you want to assert that a single MMIO request doesn't "span" multiple handlers (including the implicit default handler)?

This revision is now accepted and ready to land.Jun 30 2025, 6:28 PM