Since LA57 was moved to the main SDM document with revision 072, it seems that we should have a support for it, and silicons are coming.
This patch makes pmap support both LA48 and LA57 hardware. The selection of page table level is done at startup, kernel always receives control from loader with 4-level paging. It is not clear how UEFI spec would adapt LA57, for instance it could hand out control in LA57 mode sometimes.
To switch from LA48 to LA57 requires turning off long mode, requesting LA57 in CR4, then re-entering long mode. This is somewhat delicate and done in pmap_bootstrap_la57(). AP startup in LA57 mode is much easier, we only need to toggle a bit in CR4 and load right value in CR3.
I decided to not change kernel map for now. Single PML5 entry is created that points to the existing kernel_pml4 (KML4Phys) page, and a pml5 entry to create our recursive mapping for vtopte()/vtopde(). This decision is motivated by the fact that we cannot overcommit for KVA, so large space there is unusable until machines start providing wider physical memory addressing. Another reason is that I do not want to break our fragile autotuning, so the KVA expansion is not included into this first step.
On the other hand, (very) large address space is definitely immediately useful for some userspace applications.
For userspace, numbering of pte entries (or page table pages) is always done for 5-level structures even if we operate in 4-level mode. The pmap_is_la57() function is added to report the mode of the specified pmap, this is done not to allow simultaneous 4-/5-levels (which is not allowed by hw), but to accomodate for EPT which has separate level control and in principle might not allow 5-leve EPT despite x86 paging supports it. Anyway, it does not seems critical to have 5-level EPT support now.
elfcontrol and proccontrol allow to request or disable LA57 for specific binary, for ABI compat.
Bhyve, efirt, suspend/resume, and large map are adapted to LA57 but not tested.
PID START END PRT RES PRES REF SHD FLAG TP PATH 17 0x400000 0x426000 r-x 38 39 1 0 CN-- vn /bin/sh 17 0x626000 0x629000 rw- 3 3 1 0 C--- df 17 0x800626000 0x800648000 r-x 34 36 2 0 CN-- vn /libexec/ld-elf.so.1 17 0x800648000 0x80066b000 rw- 28 28 1 0 C--- df 17 0x80066b000 0x80066c000 r-- 1 1 3 0 ---- dv 17 0x80066c000 0x800706000 rw- 50 50 1 0 C--- df 17 0x800848000 0x80084a000 rw- 2 2 1 0 CN-- df 17 0x80084a000 0x80087e000 r-x 52 55 2 0 CN-- vn /lib/libedit.so.7 17 0x80087e000 0x800a7e000 --- 0 0 0 0 CN-- -- 17 0x800a7e000 0x800a80000 rw- 2 0 1 0 CN-- vn /lib/libedit.so.7 17 0x800a80000 0x800a84000 rw- 1 1 1 0 CN-- df 17 0x800a84000 0x800c4f000 r-x 355 384 4 0 CN-- vn /lib/libc.so.7 17 0x800c4f000 0x800e4e000 --- 0 0 0 0 CN-- -- 17 0x800e4e000 0x800e5d000 rw- 15 0 1 0 CN-- vn /lib/libc.so.7 17 0x800e5d000 0x801087000 rw- 17 17 1 0 CN-- df 17 0x801087000 0x8010e0000 r-x 89 94 2 0 CN-- vn /lib/libncursesw.so.8 17 0x8010e0000 0x8012df000 --- 0 0 0 0 CN-- -- 17 0x8012df000 0x8012e5000 rw- 6 0 1 0 CN-- vn /lib/libncursesw.so.8 17 0x8012e5000 0x8018e5000 rw- 12 12 1 0 CN-- df 17 0xffffffdffff000 0xfffffffffdf000 --- 0 0 0 0 ---- -- 17 0xfffffffffdf000 0xfffffffffff000 rw- 6 6 1 0 C--D df 17 0xfffffffffff000 0x100000000000000 r-x 1 1 4 0 ---- ph
Tested by: pho (LA48 hw)