So this is actually a bit worse than I had imagined. *sigh* The ppt(4) driver doesn't actually map its BARs (so does not ensure that MEMEN and/or PORTEN are set), and the ioctls used via /dev/pci to do I/O explicitly depend on that and would break if we did start allocating resources. I would rather have ppt(4) be more intentional and be responsible for mapping its BARs (right now memory BARs are only mapped by luck if the host firmware feels like it, the driver doesn't ensure they are allocated). One thing I considered doing earlier but did not do is establish IOMMU mappings for BARs not by passing in the HPA, but by passing in a BAR, offset, and length, and then ppt(4) could ensure the BAR was allocated. This might break the MSI-X shared BAR case though with the current /dev/pci approach. I really would much rather we had a /dev/ppt0 device and that all operations were done via ioctls on that. Some of them get a bit tricky because for assign you do need a way to get to the VM.

Anyway, my other problem is I'm currently dealing with a temperamental device that takes a while to FLR, and the end result is that the register state gets lost when the FLR takes too long. This happened to work before when bhyve was trying to set the command register from userspace, but it's really not ideal. I might be able to do this slightly hackier though if I enable MEMEN when an IOMMU mapping is created.

I have found a simpler approach for my MEMEN requirement I mentioned earlier that I will upload later today. Basically, I walk the BARs of the device during ppt_assign_device, and if it has any memory BARs I enable MEMEN in the command register at the same place we enable bus mastering. IO BARs are implicitly enabled by the /dev/pci ioctl to access the BAR.

I've tested this with a temperamental device whose FLR tends to take a bit long and it works fine across guest reboots.