Implement EOI suppression mode.

[kib]

Summary

This patch implements EOI suppression mode, where LAPIC on EOI command for level-triggered interrupt does not broadcast the EOI message to all APICs in the system. Instead, interrupt handler must follow LAPIC EOI with IOAPIC EOI. For modern IOAPICs, the later is done by writing to EOIR register. Otherwise, Intel provided Linux with a trick of temporary switching the pin config to edge and then back to level.

Documentation for EOIR exists only as part of ICH or PCH. Summary info for IOAPIC versions seems to exist in Linux source only.

I believe that suppression mode is useful even for single-IO-APIC configurations, since EOI is broadcast to all cores as well as IO-APICs.

Test Plan

I booted this on sandy and haswell, both with suppression enabled and disabled. I checked that LAPIC.SVR was properly programmed. No interrupt delivery problems were noted, in particular, interrupt counts for non-MSI sources where reasonable.

[neel]

Looks good. A couple of minor comments inline.

sys/x86/x86/io_apic.c

597

I suspect that this comment only applies to Intel IO-APICs. I wasn't able to find any information about EOI suppression for AMD.

607

It might be useful to allow this to be set to 0 via a tunable to be able to test the case where local APIC supports EOI suppression but the IO-APIC does not.

[kib]

sys/x86/x86/io_apic.c

597

AMD seems to not provide EOI suppression mechanism in LAPIC. There is not capability bit in the version register, and no enable bit in the spurious interrupt vector register. Since I do not set lapic_eoi_suppression to true for such machine, io_haseoi does not matter at all.

But, according to the sb7xxx chipset documentation, AMD IO-APIC version is 0x21, and it has compatible EOIR implementation. I have no idea what bumped version of the IO-APIC signifies.

607

I tested this by always setting io_haseoi to 0. The way I test both io_haseoi 0 or 1 is to watch the interrupt statistic with vmstat -i. I claim that EOI is working properly when non-MSI level interrupts are delivered more than once.

Do you want this knob available at field, or just noted it so the behaviour with older ioapics can be tested ?

[kib]

This update fix setting of io_haseoi to zero, when IO-APIC version is less than 0x20. IO-APIC memory is not zeroed on allocation, so io_haseoi must be initialized always.

[neel]

sys/x86/x86/io_apic.c

607

Thanks. It was mainly for testing behavior of older ioapics.