Use IOAPIC PCI rid as the interrupt TLP source id for DMAR interrupt remapping.

[kib]

Summary

If IOAPIC has PCI binding, match the PCI device against MADT enumerated IOAPIC. Match is done first by registers window phys address, then by IOAPIC ID.

When calculating rid for remapping interrupts sourced from the IOAPIC, first try to get the PCI rid, otherwise use rid as specified in the DMAR scope entry (i.e. the only current way to get rid for IOAPIC).

Test Plan

Tested this on desktop-class machine where IOAPIC does not make PCI appearence, and on the Skylake server. There currently the DMAR scope entries for uncore IOAPICs still have length two and since no bus renumbering occured, the PCI rid and scope rid are equal.

I checked that legacy interrupts still work (MSI/MSI-X were disabled manually for xhci):

flix1a% vmstat -i | grep -v cpu     
interrupt                          total       rate
irq0: attimer0                      2434         21
irq3: uart1                          247          2
irq16: xhci0                         292          3
irq257: dmar0:qi                       1          0
irq259: dmar1:qi                       1          0
irq261: dmar2:qi                   79202        690
irq263: dmar3:qi                       1          0
irq265: dmar4:qi                       1          0
irq267: dmar5:qi                       1          0
irq269: dmar6:qi                       1          0
irq271: dmar7:qi                     141          1
irq280: em0:rxq0                   70915        618
irq281: em0:rxq1                     218          2
irq282: em0:aq                         2          0
Total                             290485       2530

Also apparently PCI BARs for IOAPICs are not mapped. The attach process triggers the resource allocation.

flix1a% dmesg | grep BAR0
ioapic0: pci0:133:5:4 pci BAR0@e0f00000 id 0 MADT id 0 paddr@fec00000
ioapic1: pci0:174:5:4 pci BAR0@ee700000 id 1 MADT id 1 paddr@fec01000
ioapic2: pci0:215:5:4 pci BAR0@fbf00000 id 2 MADT id 2 paddr@fec08000

[emaste]

sys/x86/iommu/intel_drv.c

826 ↗

(On Diff #32585)

If this is to eliminate the unused variable warning I think (void)error; is more conventional, but why not just eliminate the variable?

[kib]

sys/x86/iommu/intel_drv.c

826 ↗

(On Diff #32585)

I did not wanted the #ifdef clutter in the declaration section. But ok, redid it.

[kib]

Hide the variable declaration under #ifdef.