ARM - fix IPI dispatching

[skra]

Summary

An IPI must be cleared before it's handled otherwise next IPI(s) handling can be missed. In other words, if a new request for an IPI is done in time when previous request is handled but IPI is no cleared yet, the clearing of previous IPI request clears the new one too and a handling is missed.

There are only three MP interrupt controllers in ARM now. Two of them are fixed by this change, the third one is correct, probably only just by accident. The fix is minimalistic as new interrupt framework is awaited.

It was debugged on rpi2 where missing IPI handling together with SCHED_ULE led to situation in which tdq_ipipending was not cleared and so IPI_PREEMPT was stopped to be sent. Various oddity was diplayed related to slow system response time like various timeouted events and slow console response.

Test Plan

Tested on rpi2 only.

[kib]

I think that the patch is right.

As a note, on x86, there are two types of IPIs. One is the directly dispatched by the IPI interrupt number, like rendezvous or tlb invalidation. These types send EOI after the handler is run, but also the sender must lock smp_ipi_mtx spinlock while waiting for ack, so second IPI cannot be send while first is not processed. Another types are dispatched from the single interrupt, the bitmap IPIs. There, caller is not required to hold a spinlock, and indeed, EOI is sent before the handler is run.

ARM seems to fall into the second kind, and your patch makes the handling similar to x86.