pci: Consistently use pci_vf_* for suballocated VF memory resources

[jhb]

Summary

Some of the bus resource methods were passing these up to the parent
which triggered rman mismatch assertions in INVARIANTS kernels.

Reported by: kp

[kp]

I still see a panic (on the host) when I kldload if_cxgbev in a bhyve guest that has a VF passed to it:

panic: pci_vf_map_mem_resource: rman 0xfffff800073e7e28 doesn't match for resource 0xfffff8000522dd00
cpuid = 1
time = 1717076698
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00544015f0
vpanic() at vpanic+0x13f/frame 0xfffffe0054401720
panic() at panic+0x43/frame 0xfffffe0054401780
pci_vf_map_mem_resource() at pci_vf_map_mem_resource+0x139/frame 0xfffffe0054401800
pcib_map_resource() at pcib_map_resource+0x1cd/frame 0xfffffe0054401880
pci_vf_map_mem_resource() at pci_vf_map_mem_resource+0xe9/frame 0xfffffe0054401900
pci_vf_activate_mem_resource() at pci_vf_activate_mem_resource+0xbb/frame 0xfffffe0054401960
pci_activate_resource() at pci_activate_resource+0x4c/frame 0xfffffe00544019a0
pci_vf_alloc_mem_resource() at pci_vf_alloc_mem_resource+0x134/frame 0xfffffe00544019f0
pci_alloc_resource() at pci_alloc_resource+0xd5/frame 0xfffffe0054401a50
bus_alloc_resource() at bus_alloc_resource+0x98/frame 0xfffffe0054401ab0
ppt_setup_msix() at ppt_setup_msix+0x2e2/frame 0xfffffe0054401b10
vmmdev_ioctl() at vmmdev_ioctl+0xa56/frame 0xfffffe0054401c00
devfs_ioctl() at devfs_ioctl+0xd1/frame 0xfffffe0054401c50
vn_ioctl() at vn_ioctl+0xbc/frame 0xfffffe0054401cc0
devfs_ioctl_f() at devfs_ioctl_f+0x1e/frame 0xfffffe0054401ce0
kern_ioctl() at kern_ioctl+0x286/frame 0xfffffe0054401d40
sys_ioctl() at sys_ioctl+0x12d/frame 0xfffffe0054401e00
amd64_syscall() at amd64_syscall+0x158/frame 0xfffffe0054401f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0054401f30
--- syscall (54, FreeBSD ELF64, ioctl), rip = 0xa67401ca8fa, rsp = 0xa6b951b4ab8, rbp = 0xa6b951b4b00 ---
KDB: enter: panic
[ thread pid 8266 tid 102795 ]
Stopped at      kdb_enter+0x33: movq    $0,0x1050d02(%rip)

[jhb]

Pass right device_t for map/unmap resource

[jhb]

I was able to test this by loading if_cxlv on a host after enabling iov (no need for bhyve to be involved). This version works fine for me now for that test and should also work for bhyve.

[jhb]

sys/dev/pci/pci_iov.c

1175

This can possibly be simplified to something like:

u_int i;

i = PCI_RID2BAR(rman_get_rid(r));
if (i <= PCIR_MAX_BAR_0) {
    pres = iov->iov_bar[i].res;
    ...

1218

My earlier bug was passing child here instead of iov->iov_pf so it recursively called back into here to handle the sub mapping of the parent resource.

[kp]

This version resolves the panic I saw.

[jhb]

Apply the fix from D45433