amd64: ifunc-ify pmap_invalidate_XXX() TLB invalidation functions.

[kib]

Summary

This is a continuation of D17181.

[kib]

This diff is generated from the branch where pmap_activate_sw() work was done. I will update the review after the context switch chunks will be committed.

[kib]

Distilled patch after pmap_activate_sw() changes were committed.

[kib]

Peter, could you, please, give some run time to this patch ? It is only relevant for amd64, but in two modes, with pcid enabled and with vm.pmap.pcid_enabled=0. In both cases, some threading loads and some mmap tests are enough.

[pho]

In D17184#367884, @kib wrote:

Peter, could you, please, give some run time to this patch ? It is only relevant for amd64, but in two modes, with pcid enabled and with vm.pmap.pcid_enabled=0. In both cases, some threading loads and some mmap tests are enough.

Did that + a buildworld / installworld. No problems seen.

[alc]

sys/amd64/amd64/pmap.c

1718

I would suggest refactoring these sentences:

"Because pm_pcid is recalculated on a context switch, we must disable switching. Otherwise, we might use a stale value below."

1722

Arguably, the blank line isn't needed here because the comment above is relevant to this entire block.

1745–1752

This has been deindented by one level. It might be worth reflowing the paragraph.

1801–1802

This is an aside. I wonder how often these days we wake up a halted processor to perform a shootdown on the kernel address space.

2020–2021

Is there any reason for this particular ordering of these statements? Swapping their order, would mean that the variable mask did not have to be live across a function call, and thus the compiler would not have to use a callee-saves register.

[kib]

sys/amd64/amd64/pmap.c

1745–1752

My emacs does not want to change the formatting.

1801–1802

Yes. I read some descriptions (but not the code) that Linux does not IPI CPUs which are in deep sleep, say C2 or C3. Instead, a flag is set and CPU does TLB invalidation on wakeup.

2020–2021

It might be not, but I want this change to be tested separately, if done. Also, at least for symmetry, other top level invalidation functions can get the same structure by the cost of one more if().

[kib]

Reword comment, strip at eol.

[alc]

sys/amd64/amd64/pmap.c

1801–1802

Given recent changes, we could start using the kernel pmap cpu mask to track which processors actually require shootdown. I know that some of the sleep states actually flush the TLB based on experimental observation, but I don't know what is documented in this regard.

[kib]

sys/amd64/amd64/pmap.c

1801–1802

You mean vmspace0.pmap, or kernel pmap ?

Anyway, documentation states that C3 writes back and invalidates CPU caches, but I do not remember any mention of the TLB flush.

[alc]

kernel pmap. Specifically, I would propose clearing the processor's bit in the kernel pmap when it halts. That said, we would still need to have a separate flag indicating when the processor needs to take action when it wakes. (Depending on the documentation of the deeper sleep states maybe that action is a NOP.)