On systems configured with 16K pages, this change creates 1G page mapping within the direct map where possible. Previously, the largest page size that was used to implement the direct map was 32M. On systems configured with 4K pages, this change creates 32M page mappings within the direct map where 1G is too large.
Details
Details
On a Microsoft DevKit configured with 4K pages, we see the following after this change:
vm.pmap.l2c.demotions: 2 vm.pmap.l1.demotions: 2
and
Direct map: 0xffffa00000600000-0xffffa00000700000 rw--sg WB 0 0 0 16 0 0xffffa000008c0000-0xffffa000008f0000 rw--sg WB 0 0 0 3 0 0xffffa00000c00000-0xffffa00002700000 rw--sg WB 0 0 13 16 0 0xffffa0000e400000-0xffffa0001b26f000 rw--sg WB 0 5 23 6 15 0xffffa0001c68b000-0xffffa0001f5d0000 rw--sg WB 0 0 22 52 5 0xffffa0001f5f7000-0xffffa0002eb00000 rw--sg WB 0 7 10 16 9 0xffffa00048600000-0xffffa000edf08000 rw--sg WB 1 49 28 16 8 0xffffa000edf08000-0xffffa000edf0c000 rw--sg UC 0 0 0 0 4 0xffffa000edf0c000-0xffffa0014fe3a000 rw--sg WB 1 16 15 18 14 0xffffa0014fe3a000-0xffffa0014fe5c000 rw--sg UC 0 0 0 0 34 0xffffa0014fe5c000-0xffffa0014fe5e000 rw--sg WB 0 0 0 0 2 0xffffa0014fe5e000-0xffffa0014fe6e000 rw--sg UC 0 0 0 0 16 0xffffa0014fe6e000-0xffffa0014fe74000 rw--sg WB 0 0 0 0 6 0xffffa0014fe74000-0xffffa0014fe75000 rw--sg UC 0 0 0 0 1 0xffffa0014fe75000-0xffffa0014fe77000 rw--sg WB 0 0 0 0 2 0xffffa0014fe77000-0xffffa0014fe7b000 rw--sg UC 0 0 0 0 4 0xffffa0014fe7b000-0xffffa0014fe83000 rw--sg WB 0 0 0 0 8 0xffffa0014fe83000-0xffffa0014fe84000 rw--sg UC 0 0 0 0 1 0xffffa0014fe84000-0xffffa0014fe86000 rw--sg WB 0 0 0 0 2 0xffffa0014fe86000-0xffffa0014fe87000 rw--sg UC 0 0 0 0 1 0xffffa0014fe87000-0xffffa0014fe89000 rw--sg WB 0 0 0 0 2 0xffffa0014fe89000-0xffffa0014febf000 rw--sg UC 0 0 0 0 54 0xffffa0014febf000-0xffffa0014fec1000 rw--sg WB 0 0 0 0 2 0xffffa0014fec1000-0xffffa0014fec2000 rw--sg UC 0 0 0 0 1 0xffffa0014fec2000-0xffffa0014fec4000 rw--sg WB 0 0 0 0 2 0xffffa0014fec4000-0xffffa0014fec8000 rw--sg UC 0 0 0 0 4 0xffffa0014fec8000-0xffffa0014fed0000 rw--sg WB 0 0 0 0 8 0xffffa0014fed0000-0xffffa0014fed1000 rw--sg UC 0 0 0 0 1 0xffffa0014fed1000-0xffffa0014fed3000 rw--sg WB 0 0 0 0 2 0xffffa0014fed3000-0xffffa0014fed4000 rw--sg UC 0 0 0 0 1 0xffffa0014fed4000-0xffffa0014fed6000 rw--sg WB 0 0 0 0 2 0xffffa0014fed6000-0xffffa0014fed8000 rw--sg UC 0 0 0 0 2 0xffffa0014fed8000-0xffffa0014feda000 rw--sg WB 0 0 0 0 2 0xffffa0014feda000-0xffffa0014fedb000 rw--sg UC 0 0 0 0 1 0xffffa0014fedb000-0xffffa0014fedd000 rw--sg WB 0 0 0 0 2 0xffffa0014fedd000-0xffffa0014fedf000 rw--sg UC 0 0 0 0 2 0xffffa0014fedf000-0xffffa0014fee1000 rw--sg WB 0 0 0 0 2 0xffffa0014fee1000-0xffffa0014fee2000 rw--sg UC 0 0 0 0 1 0xffffa0014fee2000-0xffffa0014fee4000 rw--sg WB 0 0 0 0 2 0xffffa0014fee4000-0xffffa0014fee5000 rw--sg UC 0 0 0 0 1 0xffffa0014fee5000-0xffffa0014feed000 rw--sg WB 0 0 0 0 8 0xffffa0014feed000-0xffffa0014ff0f000 rw--sg UC 0 0 0 0 34 0xffffa0014ff0f000-0xffffa0014ff11000 rw--sg WB 0 0 0 0 2 0xffffa0014ff11000-0xffffa0014ff21000 rw--sg UC 0 0 0 0 16 0xffffa0014ff21000-0xffffa0014ff27000 rw--sg WB 0 0 0 0 6 0xffffa0014ff27000-0xffffa0014ff28000 rw--sg UC 0 0 0 0 1 0xffffa0014ff28000-0xffffa0014ff2a000 rw--sg WB 0 0 0 0 2 0xffffa0014ff2a000-0xffffa0014ff2f000 rw--sg UC 0 0 0 0 5 0xffffa0014ff2f000-0xffffa0014ff31000 rw--sg WB 0 0 0 0 2 0xffffa0014ff31000-0xffffa0014ff32000 rw--sg UC 0 0 0 0 1 0xffffa0014ff32000-0xffffa0014ff34000 rw--sg WB 0 0 0 0 2 0xffffa0014ff34000-0xffffa0014ff5a000 rw--sg UC 0 0 0 0 38 0xffffa0014ff5a000-0xffffa0014ff5c000 rw--sg WB 0 0 0 0 2 0xffffa0014ff5c000-0xffffa0014ff6c000 rw--sg UC 0 0 0 0 16 0xffffa0014ff6c000-0xffffa0014ff72000 rw--sg WB 0 0 0 0 6 0xffffa0014ff72000-0xffffa0014ff73000 rw--sg UC 0 0 0 0 1 0xffffa0014ff73000-0xffffa0014ff75000 rw--sg WB 0 0 0 0 2 0xffffa0014ff75000-0xffffa0014ff79000 rw--sg UC 0 0 0 0 4 0xffffa0014ff79000-0xffffa0014ff7b000 rw--sg WB 0 0 0 0 2 0xffffa0014ff7b000-0xffffa0014ff7f000 rw--sg UC 0 0 0 0 4 0xffffa0014ff7f000-0xffffa0014ff81000 rw--sg WB 0 0 0 0 2 0xffffa0014ff81000-0xffffa0014ff82000 rw--sg UC 0 0 0 0 1 0xffffa0014ff82000-0xffffa0014ff84000 rw--sg WB 0 0 0 0 2 0xffffa0014ff84000-0xffffa0014ff97000 rw--sg UC 0 0 0 0 19 0xffffa0014ff97000-0xffffa0014ff99000 rw--sg WB 0 0 0 0 2 0xffffa0014ff99000-0xffffa0014ff9d000 rw--sg UC 0 0 0 0 4 0xffffa0014ff9d000-0xffffa0014ff9f000 rw--sg WB 0 0 0 0 2 0xffffa0014ff9f000-0xffffa0014ffa0000 rw--sg UC 0 0 0 0 1 0xffffa0014ffa0000-0xffffa0014ffa2000 rw--sg WB 0 0 0 0 2 0xffffa0014ffa2000-0xffffa0014ffb5000 rw--sg UC 0 0 0 0 19 0xffffa0014ffb5000-0xffffa00380000000 rw--sg WB 8 24 0 4 11 0xffffa00780000000-0xffffa00c00000000 rw--sg WB 18 0 0 0 0
Diff Detail
Diff Detail
- Lint
Lint Skipped - Unit
Tests Skipped
Event Timeline
Comment Actions
I tried this on my ampere altra test box. Our (netflix) tree is roughly 1 month old (based at 63b0165cdcbb178df63ac57dcd39c29cf77f346e).
I cherry-picked a803837cec6e17e04849d59afac7b6431c70cb93 c1ebd76c3f283b10afe6b64f29fe68c4d1be3f8c b5a1f0406b9d6bba28e57377dcfc8b83bce987ad and then applied the patch. Its possible that I missed something critical elsewhere.. I just looked at the history of sys/arm64/arm64/pmap.c and cherry-picked what seemed to have come in after our latest upstream sync.
In the new kernel, I see:
# sysctl vm.pmap vm.pmap.l3c.promotions: 4284 vm.pmap.l3c.p_failures: 156 vm.pmap.l3c.mappings: 348 vm.pmap.l3c.demotions: 795 vm.pmap.l2.promotions: 276 vm.pmap.l2.p_failures: 0 vm.pmap.l2.mappings: 0 vm.pmap.l2.demotions: 120 vm.pmap.l2c.demotions: 0 vm.pmap.l1.demotions: 0 vm.pmap.superpages_enabled: 1 vm.pmap.vmid.epoch: 0 vm.pmap.vmid.next: 2 vm.pmap.vmid.bits: 16 vm.pmap.asid.epoch: 0 vm.pmap.asid.next: 12760 vm.pmap.asid.bits: 16 # sysctl vm.pmap.kernel_maps | tail -15 0xffff001d14fb8000-0xffff001d17b0c000 rw--sg WB 0 0 0 21 85 0xffff007fffe24000-0xffff007fffffc000 rw--sg WT 0 0 0 0 118 0xffff007fffffc000-0xffff008000000000 rw--sg DEV 0 0 0 0 1 Direct map: 0xffffa00088300000-0xffffa00088400000 rw--sg WB 0 0 0 0 64 0xffffa00090000000-0xffffa000ebf28000 rw--sg WB 0 0 45 15 74 0xffffa000ec308000-0xffffa000ec30c000 rw--sg WB 0 0 0 0 1 0xffffa000ec310000-0xffffa000ec530000 rw--sg WB 0 0 0 0 136 0xffffa000ec540000-0xffffa000ee550000 rw--sg WB 0 0 0 15 132 0xffffa000eeab8000-0xffffa000ffc8c000 rw--sg WB 0 0 7 24 117 0xffffa000ffc90000-0xffffa00100000000 rw--sg WB 0 0 0 1 92 0xffffa80000000000-0xffffa80080000000 rw--sg WB 0 2 0 0 0 0xffffa80100000000-0xffffa82000000000 rw--sg WB 0 124 0 0 0
Comment Actions
Thank you. This looks okay. The last two entries show 2 and 124 1 GB page mappings being used instead of 32 MB page mappings.
I was just hoping for another machine besides the DevKit that would have to perform the modified L1 and new L2C demotion code.
P.S. You would also benefit from my recent change to locore.S.
Comment Actions
Here are the vm.pmap counters and the direct map portion of vm.pmap.kernel_maps on a 4 GB Raspberry Pi 4:
vm.pmap.l3c.promotions: 9279 vm.pmap.l3c.p_failures: 450 vm.pmap.l3c.mappings: 498 vm.pmap.l3c.demotions: 216 vm.pmap.l2.promotions: 95 vm.pmap.l2.p_failures: 360 vm.pmap.l2.mappings: 0 vm.pmap.l2.demotions: 69 vm.pmap.l2c.demotions: 2 vm.pmap.l1.demotions: 0 vm.pmap.superpages_enabled: 1 vm.pmap.vmid.epoch: 0 vm.pmap.vmid.next: 2 vm.pmap.vmid.bits: 8 vm.pmap.asid.epoch: 0 vm.pmap.asid.next: 784 vm.pmap.asid.bits: 16 Direct map: 0xffffa00000002000-0xffffa000009bc000 rw--sg WB 0 0 3 58 26 0xffffa000009bc000-0xffffa000009c3000 rw--sg UC 0 0 0 0 7 0xffffa000009c3000-0xffffa00000d3d000 rw--sg WB 0 0 1 22 26 0xffffa00000d3d000-0xffffa00000d3e000 rw--sg UC 0 0 0 0 1 0xffffa00000d3e000-0xffffa00000d4e000 rw--sg WB 0 0 0 0 16 0xffffa00000d4e000-0xffffa00000d63000 rw--sg UC 0 0 0 0 21 0xffffa00000d63000-0xffffa00007386000 rw--sg WB 0 2 18 33 19 0xffffa00007386000-0xffffa00007392000 rw--sg UC 0 0 0 0 12 0xffffa00007392000-0xffffa00033adc000 rw--sg WB 0 21 19 19 26 0xffffa00033adc000-0xffffa00033add000 rw--sg UC 0 0 0 0 1 0xffffa00033add000-0xffffa00033adf000 rw--sg WB 0 0 0 0 2 0xffffa00033adf000-0xffffa00033ae0000 rw--sg UC 0 0 0 0 1 0xffffa00033ae0000-0xffffa00033ae4000 rw--sg WB 0 0 0 0 4 0xffffa00033ae4000-0xffffa00033ae5000 rw--sg UC 0 0 0 0 1 0xffffa00033ae5000-0xffffa00033ae9000 rw--sg WB 0 0 0 0 4 0xffffa00033ae9000-0xffffa00033aea000 rw--sg UC 0 0 0 0 1 0xffffa00033aea000-0xffffa00033aec000 rw--sg WB 0 0 0 0 2 0xffffa00033aec000-0xffffa00033aed000 rw--sg UC 0 0 0 0 1 0xffffa00033aed000-0xffffa00033af3000 rw--sg WB 0 0 0 0 6 0xffffa00033af3000-0xffffa00033af4000 rw--sg UC 0 0 0 0 1 0xffffa00033af4000-0xffffa00033b85000 rw--sg WB 0 0 0 8 17 0xffffa00033b85000-0xffffa00033ba6000 rw--sg UC 0 0 0 0 33 0xffffa00033ba6000-0xffffa00033ba8000 rw--sg WB 0 0 0 0 2 0xffffa00033ba8000-0xffffa00033bb8000 rw--sg UC 0 0 0 0 16 0xffffa00033bb8000-0xffffa00033e43000 rw--sg WB 0 0 1 4 75 0xffffa00033e43000-0xffffa00033e44000 rw--sg UC 0 0 0 0 1 0xffffa00033e44000-0xffffa00033e4e000 rw--sg WB 0 0 0 0 10 0xffffa00033e4e000-0xffffa00033e4f000 rw--sg UC 0 0 0 0 1 0xffffa00033e4f000-0xffffa00033e59000 rw--sg WB 0 0 0 0 10 0xffffa00033e59000-0xffffa00033e5a000 rw--sg UC 0 0 0 0 1 0xffffa00033e5a000-0xffffa00033e6b000 rw--sg WB 0 0 0 0 17 0xffffa00033e6b000-0xffffa00033e6c000 rw--sg UC 0 0 0 0 1 0xffffa00033e6c000-0xffffa00033e72000 rw--sg WB 0 0 0 0 6 0xffffa00033e72000-0xffffa00033e73000 rw--sg UC 0 0 0 0 1 0xffffa00033e73000-0xffffa0003b400000 rw--sg WB 0 3 10 24 13 0xffffa00040000000-0xffffa000fc000000 rw--sg WB 2 30 0 0 0
| sys/arm64/arm64/pmap.c | ||
|---|---|---|
| 8349 | I've spent a little time thinking about how to refactor the various demotion functions to reduce duplication, so I'm planning to do something along these lines. | |
| 8397 | Yes. This applies to a few other functions too, so either Eliot or I will deal with this separately. | |
| 8410 | I'm planning to change this later too. The L2 entry is invalid, so we really don't need to use fcmpset. | |
| sys/arm64/arm64/pmap.c | ||
|---|---|---|
| 4612 | I wonder why this isn't an assertion? | |