This is required for KMSAN support, which applies MemorySanitizer[*] to
the kernel.
This looks similar to support for the KASAN shadow map, but has some
differences:
- KMSAN requires two shadow maps: one is to track the initialization
state of memory in the kernel map (referred to as the shadow), and the
origin map, which stores compressed pointers encoding the source of
uninitialized memory. In particular, KMSAN only raises warnings when
uninitialized memory is used as a source operand in operations which
affect a program's behaviour, such as conditional branches. Memory
copies do not raise warnings. So the origin map is only to help
debugging in cases where uninitialized memory is copied many times
before it is used. For some reason LLVM does not make it optional for
the kernel.
- Both shadow maps are 1:1 with the kernel map. In contrast, KASAN's
shadow map is 1:8, so KMSAN has much larger memory overhead.
- I chose to not create shadows of the vm_page array or for memory above
KERNBASE for now. Otherwise, the shadow maps consume a significant
fraction of memory (with KMSAN instrumentation the kernel text is
bloated significantly), and I don't believe that shadowing these
regions will help expose significant bugs. This may be changed later.
This change does the following:
- Reserve PML4 slots for the shadow maps. The origin map overlaps with
the KASAN shadow, this is ok as they won't be configured together.
- Creates a dummy mapping of the vm_page array, mapping the
corresponding region in both shadows to a single page. This is to
avoid extra branching in the KMSAN runtime, which has to determine
for each memory access whether the address falls within the kernel
map.
- Modify pmap_growkernel() to grow the shadows as well, using a function
in the KMSAN runtime which calls pmap_kmsan_enter() for each 4KB page
in the new region.
- Disable unmapped I/O when KMSAN is configured, as it can lead to false
positives if unmapped buffer pages are initialized.
- Modify some kernel memory size estimates to reflect the presence of
the shadows.