pci: Implement pci_bar_enabled() for VFs

[markj]

Summary

In a VF's configuration space, "memory space enable" is hard-wired to 0,
so the existing implementation always returns false. We need to read
the SR-IOV control register from the PF device to get the value of the
MSE bit.

Fix pci_bar_enabled() to read this register instead for VFs. I don't
see any way to access the PF's config space without a backpointer in the
pci device ivars, so I added one.

This fixes a regression where bhyve(8) fails to map the MSI-X table
after commit 7fa233534736 ("bhyve: Map the MSI-X table unconditionally
for passthrough") when a VF is passed through, since with that commit we
use PCIOCBARMMAP to map the table and that ioctl always fails for VFs
without this change. As a bonus, pciconf(8) now correctly reports the
enablement of BARs for VF devices.

While here, fix an unrelated bug in the PCIOCBARMMAP implementation that
I noticed: BUS_TRANSLATE_RESOURCE() returns an integer bool, not an
errno value.

Reported by: Raúl Muñoz <[e-mail]>

[jhb]

So my one question is what does pciconf -l show in a guest for passed through VFs? I feel like I had to do some hackery in bhyve to deal with this for VF's.

sys/dev/pci/pci_user.c

880 ↗

(On Diff #98036)

This is terrible. No BUS_* API should be returning bool. That should be fixed instead, perhaps as a followup?

[markj]

In D32839#741318, @jhb wrote:

So my one question is what does pciconf -l show in a guest for passed through VFs? I feel like I had to do some hackery in bhyve to deal with this for VF's.

In a guest with a patched kernel:

none0@pci0:0:4:0:       class=0x020000 rev=0x00 hdr=0x00 vendor=0x8086 device=0x15c5 subvendor=0x8086 subdevice=0x0000                                                                                                                                                                                                        
    vendor     = 'Intel Corporation'                                                                                                                                                                                                                                                                                          
    device     = 'X553 Virtual Function'                                                                                                                                                                                                                                                                                      
    class      = network                                                                                                                                                                                                                                                                                                      
    subclass   = ethernet                                                                                                                                                                                                                                                                                                     
    bar   [10] = type Memory, range 32, base 0xc0004000, size 16384, enabled                                                                                                                                                                                                                                                  
    bar   [1c] = type Memory, range 32, base 0xc0008000, size 16384, enabled                                                                                                                                                                                                                                                  
    cap 05[f0] = MSI supports 1 message, 64 bit                                                                                                                                                                                                                                                                               
    cap 11[70] = MSI-X supports 3 messages                                                                                                                                                                                                                                                                                    
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]                                                                                                                                                                                                                                                              
    cap 10[a0] = PCI-Express 2 endpoint max data 128(512) FLR                                                                                                                                                                                                                                                                 
                 max read 128                                                                                                                                                                                                                                                                                                 
                 link x0(x1) speed 0.0(2.5) ASPM disabled(L0s/L1)

Note that pci_bar_enabled() is currently only used in /dev/pci ioctl handlers.

sys/dev/pci/pci_user.c

880 ↗

(On Diff #98036)

Ok, I will post a separate review for this.

[markj]

Drop the pci_user.c change.