arm64 pmap: optimize TLB management by pmap_update_entry()

[alc]

Summary

To date, pmap_update_entry() has unconditionally passed false as final_only to pmap_s1_invalidate_range(). Passing false means that we invalidate the intermediate "page walk cache" entries in the TLB as well as the leaf that is being replaced. However, invalidating intermediate entries is only necessary when doing a superpage promotion that replaces a pointer to a page table page by a large page mapping.

[markj]

sys/arm64/arm64/pmap.c

5115–5116

This comment is a bit stale now. It's not really wrong but it's essentially just restating what was added to the function header comment.

[alc]

sys/arm64/arm64/pmap.c

5090

Without the __always_inline here the compiler already inlines all of the calls except the one from pmap_promote_l2. For that, the compiler generates the one and only call to a non-specialized static function that doesn't exploit constant propagation and dead code elimination. This surprised me, since functions with a single call site are normally inlined. (I guess that heuristic is being applied early based on the source level, rather than late based on IR.) The __always_inline here enables specialization, reducing the overall code size by ~90 bytes.

[alc]

sys/arm64/arm64/pmap.c

5115–5116

Would a more useful comment be along the lines of "We aggressively inline this function so that constant propagation and dead code elimination will effectively specialize the following code."?

[markj]

sys/arm64/arm64/pmap.c

5090

Indeed, I can get clang to output:

sys/arm64/arm64/pmap.c:5353:2: remark: 'pmap_update_entry' not inlined into 'pmap_promote_l2' because too costly to inline (cost=65, threshold=45) [-Rpass-missed=inline]

but I don't know how it determines those numbers. Looking at InlineCost.cpp and InlineAdvisor.cpp, I see some heuristics regarding the last call of a static function, but it seems they are failing somehow.

5115–5116

I think that would be good.

[kib]

Does arm64 use recursive mapping? If yes, could the function be used on recursive mapping?

[alc]

In D58917#1352088, @kib wrote:

Does arm64 use recursive mapping? If yes, could the function be used on recursive mapping?

No, it does not.

[alc]

Revise a comment.