VMX: Handle external interrupts with interrupts disabled.

[markj]

Summary

Currently, an external interrupt-triggered vmexit causes host interrupts
to be enabled for a brief window before the ISR is called, during which
the vcpu thread could switch off-CPU and even migrate to a different
physical CPU.

Fix this by calling vmx_exit_process() with interrupts disabled and
enabling them only after the ISR returns.

[markj]

It was noted that the layer which invokes vmrun runs with preemption disabled, so it should not be possible for the vcpu thread to switch off-CPU. I'm not sure if there are any other problems that can arise from enabling host interrupts before calling the ISR.

[kib]

In D22764#498381, @markj wrote:

It was noted that the layer which invokes vmrun runs with preemption disabled, so it should not be possible for the vcpu thread to switch off-CPU. I'm not sure if there are any other problems that can arise from enabling host interrupts before calling the ISR.

It also reorders interrupts. If higher-priority interrupt comes in the window, it will be served first. I do not think that lower-priority interrupt can be handled before us because EOI is only sent by the handler.

I believe we would need to worry less about this code if we make the vm exit and handler call atomic.

[pmooney_pfmooney.com]

sys/amd64/vmm/intel/vmx.c

2986–2990

Rather than sprinkling the enable_intr() calls into vmx_exit_process(), would it perhaps be worth adding an extra handler function for the NMI and EXT_INTR cases? Leaving the enable_intr() call in vmx_run(), so the clear boundaries are maintained, that handler could be called prior to interrupt re-enabling. It would place a clear separation between "normal" exit processing (requiring no special interrupt-disabled state), and the special processing for NMI/extint.