vm_phys_early_alloc(domain, alloc_size) restricts its search of
phys_avail to the single largest SRAT range of the domain (or, for
domain -1, the largest range of all). When phys_avail has little or
nothing inside that range the search fails and the kernel panics early
in boot with "cannot find a large enough size", although other ranges
of the same domain may have plenty of free memory.
How we found it: keelOS's hypervisor keeps most of the RAM for its
guest pool and gives its FreeBSD root partition a few GB of it. On a
two-node test bed (12 GB, 6 GB per node, 2 GB given to the root, all of
it below 4 GB) every boot panicked in pmap_page_array_startup() ->
vm_phys_early_alloc(): node 0's largest SRAT range is 4G-8G and
phys_avail has nothing there. Stock FreeBSD reaches the same state on
a NUMA machine when hw.physmem limits the usable memory to the low part
of node 0 (the test plan reproduces it in QEMU that way), and in general
whenever the firmware or a hypervisor presents a node whose largest
range is mostly unusable.
Drop the SRAT range pre-selection and pick the largest phys_avail
segment that belongs to the requested domain. vm_phys_early_startup()
has already split phys_avail at the domain boundaries, so every segment
lies in one domain; it is looked up by its last byte because
vm_phys_domain() treats a range's end as inclusive and a segment may
start exactly where the previous domain ends.