Add x86 msr tweak KPI.

[kib]

Summary

It is consistently uses rendezvouz IPI instead of binding current thread to all CPUs.

Use the KPI to tweak MSRs in mitigation code.

[scottl]

This looks great. My only complaint is using the name "tweak", I think it's too casual and poorly descriptive. Maybe x86_program_msr_smp()?

[kib]

Rename to x86_msr_op().
Restore sched_bind() mode of broadcast to keep the ssb toggle algorithm unchanged. I do think that my switch to smp_rendezvous for ssb is actually fine because our use of mwait does not re-enable interrupts, but let do that as separate change.

[markj]

sys/x86/x86/cpu_machdep.c

153 ↗

(On Diff #64519)

Should we also assert that OP_LOCAL and OP_SCHED are not both specified?

172 ↗

(On Diff #64519)

I think it would be nicer to have an explicit name for this case, to make it easier to read the calling code. Maybe MSR_OP_RENDEZVOUS.

[kib]

Add MSR_OP_RENDEZVOUS. Assert that exactly one command mode flag is passed.

[scottl]

Minor request, if the MSR_OP_LOCAL/SCHED/RENDEVOUS opcodes are mutually exclusive from each other then don't make them be bitfield definitions, just have them be sequential numbers.

[kib]

Make execution submode numeric.

[kib]

x86_var.h was missed.