Add support for pmap_enter(..., psind == 1) to armv6's pmap

[alc]

Summary

See the title.

In addition, as an optimization to the psind == 0 case, I've eliminated the use of pte2_is_managed() from pmap_enter(). Unlike the x86 pmap implementations, armv6 does not have a managed bit defined within the PTE. So, pte2_is_managed() is actually a call to PHYS_TO_VM_PAGE(), which is O(n) in the number of vm_phys_segs[] on armv6. All but one call to PHYS_TO_VM_PAGE() in pmap_enter() can be avoided.

Test Plan

Ask mmel@ to test the patch.

[kib]

arm/arm/pmap-v6.c

3246 ↗

(On Diff #46155)

Should this call use noreclaim instead of TRUE ?

If not, what is the reason for noreclaim var ?

vm/vm_fault.c

289 ↗

(On Diff #46155)

I thihk, after this commit, we should provide a define in machine/pmap.h to indicate that pmap_enter() supports psind > 0.

[alc]

arm/arm/pmap-v6.c

3246 ↗

(On Diff #46155)

You are correct, and I just verified that i386 correctly passes noreclaim here.

vm/vm_fault.c

289 ↗

(On Diff #46155)

At this point, we could almost rely on VM_NRESERVLEVEL by itself. I say almost because we enable reservations on arm < v6 and sparc64. I would argue for disabling reservations on arm < v6. sparc64 is the less obvious case. The TSB layer appears to properly handle different page sizes, so implementing pmap_enter(psind == 1) might not be that difficult. Mostly, it would require some thought about how to handle the TSB layer returning a 4 MB mapping to pmap_protect() and pmap_remove(). And, I suspect that the bigger issue is getting the patches tested.

[alc]

Correct the argument passed to get_pv_entry().

[mmel]

Tested on Jetson TK1 (cortex-A15) without single problem.
From preliminary collected statistic it looks like pmap_enter(..., psind == 1) is used ~60 times per buildworld.
Unfortunately, the same statistic also told me that is pmap_enter_object() doesn't not create single 1MB mappings, even for shared libraries.
I'm pretty sure that this worked before - but this regression is not related to this patch.

arm/arm/pmap-v6.c

3903 ↗

(On Diff #46291)

trailing whitespace.

[alc]

In D16555#353196, @mmel wrote:

Tested on Jetson TK1 (cortex-A15) without single problem.
From preliminary collected statistic it looks like pmap_enter(..., psind == 1) is used ~60 times per buildworld.
Unfortunately, the same statistic also told me that is pmap_enter_object() doesn't not create single 1MB mappings, even for shared libraries.
I'm pretty sure that this worked before - but this regression is not related to this patch.

Which linker is being used?

Can you please email me the output of "procstat -av"?

[mmel]

In D16555#353199, @alc wrote:

Which linker is being used?

Can you please email me the output of "procstat -av"?

Output mailed directly.
And about linker - it's hard to say :)

My src tree is at r337375, userland is at 1200075, kernel 1200076 so probably ld.lld.
I will try this again with all components updated to current.
But procstat -av output looks good for me - at least
443 0x20100000 0x20296000 r-x 319 335 70 34 CN-- vn /lib/libc.so.7
is properly aligned with right attributes.

[alc]

In D16555#353215, @mmel wrote:

In D16555#353199, @alc wrote:

Which linker is being used?

Can you please email me the output of "procstat -av"?

Output mailed directly.
And about linker - it's hard to say :)

My src tree is at r337375, userland is at 1200075, kernel 1200076 so probably ld.lld.
I will try this again with all components updated to current.
But procstat -av output looks good for me - at least
443 0x20100000 0x20296000 r-x 319 335 70 34 CN-- vn /lib/libc.so.7
is properly aligned with right attributes.

Reboot the machine, login, and run the command dd if=/lib/libc.so.7 of=/dev/null count=2048. Thereafter, you should see a superpage mapping for the first 1 MB of every libc mapping. I didn't observe ntpd in the procstat output that you sent me. (Maybe I just missed it?) Normally, I would expect the mlockall() by ntpd to page in all of libc.so so that all future processes would have a superpage mapping for its first 1 MB. Moreover, with this change, if a process that was started before ntpd or the dd command faults on an unmapped page within the first 1 MB of libc.so, the mapping will be upgraded to a superpage mapping via pmap_enter(..., psind=1).

That said, the real blocker for superpage mappings by pmap_enter_object() is that the linker is not using a superpage-aligned default image base.

60372     0x8000  0x46a3000 r-x 8884 8901  16   8 CN-- vn /usr/bin/clang-cpp
60372  0x46aa000  0x46b3000 rw-    9    0   1   0 C--- vn /usr/bin/clang-cpp
60372  0x46b3000  0x4709000 rw-   67   67   1   0 ---- df 
60372 0x246aa000 0x24800000 rw-  308  308   1   0 ---- df 
60372 0x24800000 0x24a21000 rw-  108  108   1   0 ---- df 
60372 0x24a21000 0x24a46000 r--   37   37   1   0 CN-- vn /usr/src/sys/contrib/ngatm/netnatm/msg/uni_ie.c
60372 0x24a46000 0x24a5a000 rw-   20   20   1   0 ---- df 
60372 0x24a5a000 0x24a62000 r--    8    8   4   0 CN-- vn /usr/src/sys/sys/cdefs.h
60372 0x24a62000 0x24ae2000 rw-  128  128   1   0 ---- df 
60372 0x24ae2000 0x24aec000 r--   10   10   1   0 CN-- vn /usr/src/sys/contrib/ngatm/netnatm/msg/unistruct.h
60372 0x24aec000 0x24b31000 rw-   68   68   1   0 ---- df 
60372 0x24b31000 0x24b36000 r--    5    5   3   0 CN-- vn /usr/obj/usr/src/arm.armv7/tmp/usr/include/stdio.h
60372 0x24b36000 0x24be8000 rw-  177  177   1   0 ---- df 
60372 0x24be8000 0x24bf2000 r--   10   10   1   0 CN-- vn /usr/src/sys/contrib/ngatm/netnatm/msg/uni_ietab.h
60372 0x24bf2000 0x258aa000 rw- 3173 3173   1   0 ---- df 
60372 0xbf3fe000 0xbf3ff000 ---    0    0   0   0 ---- -- 
60372 0xbf3ff000 0xbfbdf000 ---    0    0   0   0 ---- -- 
60372 0xbfbdf000 0xbfbff000 rw-   23   23   1   0 ---D df 
60372 0xbfbff000 0xbfc00000 r-x    1    1  57   0 ---- ph

This looks to me like the old binutils ld, and not the new llvm ld, but I'm not 100% certain.