vmm: add tunable to trap WBINVD

[corvink]

Summary

x86 is cache coherent. However, there are special cases where cache
coherency isn't ensured (e.g. when switching the caching mode). In these
cases, WBINVD can be used. WBINVD writes all cache lines back into main
memory and invalidates the whole cache.

Due to the invalidation of the whole cache, WBINVD is a very heavy
instruction and degrades the performance on all cores. So, we should
minimize the use of WBINVD as much as possible.

In a virtual environment, the WBINVD call is mostly useless. The guest
isn't able to break cache coherency because he can't switch the physical
cache mode. When using pci passthrough WBINVD might be useful.

Nevertheless, trapping and ignoring WBINVD is an unsafe operation. For
that reason, we implement it as tunable.

[jhb]

sys/amd64/vmm/amd/svm.c

509

Should we add a tunable for the SVM case? Also, would you consider making this a capability that can be specified on a per-VM basis the way HLT and PAUSE trapping is?

[corvink]

sys/amd64/vmm/amd/svm.c

509

Sry, I don't understand which tunable you want to add to SVM.

This patch adds two new tunables:

• hw.vmm.trap_wbinvd which enables the trap (used by vmx and svm)
• hw.vmm.vmx.cap.wbinvd_exit it just reports if vmx supports the trap. As far as I can see, svm always supports it and svm has no such cap tunables yet. I've only added it to be consistent with cap_pause_exit etc.

WBINVD degrades performance on all threads. So, either you want to avoid it for all VMs or you don't care. At the moment, it could make sense to specify the trap on a per-VM basis because (to my understanding) it's only unsafe under certain conditions when using passthru devices.

I'm thinking of a solution to emulate WBINVD instead of ignoring it in the future. When that's ready, the tunable could be removed and the trap could always be enabled.

KVM only executes WBINVD if the guest has access a passthru device with non-coherent cache, otherwise KVM ignores WBINVD too (https://elixir.bootlin.com/linux/latest/source/arch/x86/kvm/x86.c#L7375). ACRN does a page-walk on the whole EPT and calls clflushopt for each cacheline (https://github.com/projectacrn/acrn-hypervisor/blob/8b16be918533ba9395ec7fa4eea40412075c487b/hypervisor/arch/x86/guest/vmexit.c#L429).

[jhb]

sys/amd64/vmm/amd/svm.c

509

I had misread the tunable patch of the patch. Mmm, this is probably fine, yes.