MFC r368649 / 3fd989da by kib: amd64 pmap: fix PCID mode invalidations

[avg]

Summary

r368649 fixed a regression in r362031 that was MFC-ed to stable/12 as
a part of r362572. That commit reordered IPI send and local TLB flush in
TLB invalidations.

Without this fix we've been seeing problems with stale memory content
where changes done under a mutex were not immediately observed by
another thread after taking the same mutex. Those inconsistenices were
correlated to copy-on-write faults for pages contaning the data.

The change needed some adaptations as I elected to skip two significant
intermediate changes:

• r363195 / dc43978a, amd64: allow parallel shootdown IPIs
• r363311 / 3ec7e169, amd64 pmap: microoptimize local shootdowns for PCID PTI configurations

[emaste]

Do we want to make this an EN for 12.3 as well?

[avg]

In D33413#755863, @emaste wrote:

Do we want to make this an EN for 12.3 as well?

I think that some people may appreciate it, especially if the Go deadlocks / crashes also have the same root cause.