amd64: Fix TLB invalidation routines in !SMP kernels

[markj]

Summary

amd64 is special in that its implementation of zpcpu_offset_cpu() is not
the identity transformation, even in !SMP kernels. Because the pm_pcidp
array of amd64's struct pmap is allocated from a pcpu UMA zone, this
means that accessing pm_pcidp directly, as is done in !SMP
implementations of pmap_invalidate_*, does not work. Specifically, I
see occasional unexplicable crashes in userspace when PCIDs are enabled.

Apply a minimal patch to fix the problem. While it would also make
sense to provide separate implementations of zpcpu_* for !SMP kernels,
fixing it this way makes the SMP and !SMP implementations of
pmap_invalidate_* more similar.

[kib]

Should we remove SMP option from amd64 at all?

[alc]

sys/amd64/amd64/pmap.c

3589

Btw, recent Ryzens support this instruction even when they don't support PCIDs. I suspect that it's faster than what we do for invltlb_glob().

[markj]

In D41230#939077, @kib wrote:

Should we remove SMP option from amd64 at all?

I am in favour of this. Clearly it is not being tested - on recent Intel CPUs at least - and amd64 atomics and pcpu accessors are not optimized in !SMP kernels. The kernel itself shinks by 0.3% when options SMP is disabled. On amd64 most of the SMP-specific code is in pmap_invalidate_*(), so simply removing the alternate implementations would be a useful step.

In general, SMP is enabled in almost all of the kernel configurations available in main. kern.smp.disabled=1 can be set in the kernel environment at compile time, if needed (e.g., for bringup on new systems). Maybe we should consider dropping the option entirely.