Right now we have to map the PBA in case it shares a page with the MSI-X
table. It turns out that some devices (newer Intel wifi devices in
particular) may map registers in the same page as the MSI-X table, in
violation of the PCI base spec. To handle both cases, and to avoid
having to use /dev/mem at all, use PCIOCBARMMAP to map the MSI-X BAR and
handle all accesses outside the table like we do today for the PBA.
After some discussion with jhb, I first tried implementing this by
having each trapped BAR access use the new PCIOCBARIO ioctl to actually
perform the access. This ends up being simple to implement, but the
downside is that the ioctl activates and deactivates the resource for
each call, which in this case means that each access will trigger a
global TLB shootdown. In my testing with an Intel wifi device, these
accesses may be quite frequent, and so I think this overhead is too high
since it also perturbs the rest of the system by generating IPIs. I
believe there isn't really much of a downside to unconditionally mapping
the MSI-X table, so I went with that approach instead.