Page MenuHomeFreeBSD

pci: Implement pci_bar_enabled() for VFs
ClosedPublic

Authored by markj on Nov 4 2021, 9:47 PM.
Tags
None
Referenced Files
Unknown Object (File)
Mon, Nov 11, 8:15 PM
Unknown Object (File)
Sat, Nov 2, 9:08 PM
Unknown Object (File)
Fri, Nov 1, 7:15 AM
Unknown Object (File)
Oct 5 2024, 3:13 AM
Unknown Object (File)
Oct 5 2024, 1:05 AM
Unknown Object (File)
Oct 5 2024, 1:05 AM
Unknown Object (File)
Oct 5 2024, 1:05 AM
Unknown Object (File)
Oct 5 2024, 1:05 AM
Subscribers

Details

Summary

In a VF's configuration space, "memory space enable" is hard-wired to 0,
so the existing implementation always returns false. We need to read
the SR-IOV control register from the PF device to get the value of the
MSE bit.

Fix pci_bar_enabled() to read this register instead for VFs. I don't
see any way to access the PF's config space without a backpointer in the
pci device ivars, so I added one.

This fixes a regression where bhyve(8) fails to map the MSI-X table
after commit 7fa233534736 ("bhyve: Map the MSI-X table unconditionally
for passthrough") when a VF is passed through, since with that commit we
use PCIOCBARMMAP to map the table and that ioctl always fails for VFs
without this change. As a bonus, pciconf(8) now correctly reports the
enablement of BARs for VF devices.

While here, fix an unrelated bug in the PCIOCBARMMAP implementation that
I noticed: BUS_TRANSLATE_RESOURCE() returns an integer bool, not an
errno value.

Reported by: Raúl Muñoz <raul.munoz@custos.es>

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

markj requested review of this revision.Nov 4 2021, 9:47 PM

So my one question is what does pciconf -l show in a guest for passed through VFs? I feel like I had to do some hackery in bhyve to deal with this for VF's.

sys/dev/pci/pci_user.c
880 ↗(On Diff #98036)

This is terrible. No BUS_* API should be returning bool. That should be fixed instead, perhaps as a followup?

In D32839#741318, @jhb wrote:

So my one question is what does pciconf -l show in a guest for passed through VFs? I feel like I had to do some hackery in bhyve to deal with this for VF's.

In a guest with a patched kernel:

none0@pci0:0:4:0:       class=0x020000 rev=0x00 hdr=0x00 vendor=0x8086 device=0x15c5 subvendor=0x8086 subdevice=0x0000                                                                                                                                                                                                        
    vendor     = 'Intel Corporation'                                                                                                                                                                                                                                                                                          
    device     = 'X553 Virtual Function'                                                                                                                                                                                                                                                                                      
    class      = network                                                                                                                                                                                                                                                                                                      
    subclass   = ethernet                                                                                                                                                                                                                                                                                                     
    bar   [10] = type Memory, range 32, base 0xc0004000, size 16384, enabled                                                                                                                                                                                                                                                  
    bar   [1c] = type Memory, range 32, base 0xc0008000, size 16384, enabled                                                                                                                                                                                                                                                  
    cap 05[f0] = MSI supports 1 message, 64 bit                                                                                                                                                                                                                                                                               
    cap 11[70] = MSI-X supports 3 messages                                                                                                                                                                                                                                                                                    
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]                                                                                                                                                                                                                                                              
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR                                                                                                                                                                                                                                                                 
                 max read 128                                                                                                                                                                                                                                                                                                 
                 link x0(x1) speed 0.0(2.5) ASPM disabled(L0s/L1)

Note that pci_bar_enabled() is currently only used in /dev/pci ioctl handlers.

sys/dev/pci/pci_user.c
880 ↗(On Diff #98036)

Ok, I will post a separate review for this.

Drop the pci_user.c change.

This revision is now accepted and ready to land.Nov 8 2021, 2:11 PM