Fix boot on systems where NUMA domain 0 is unpopulated.

[markj]

Summary

Add vm_phys_early_add_seg(), complementing vm_phys_early_alloc(), to
ensure that segments registered during hammer_time() are placed in the
right domain. Otherwise, since SRAT is not parsed at that point, we
just add them to domain 0, which may be incorrect and results in a
domain with only several MB worth of memory.

Fix uma_startup1() to try allocating memory for zones from any domain.
If domain 0 is unpopulated, the allocation will simply fail, resulting
in a page fault slightly later during boot.

Test Plan

I popped the DIMMs out of socket 0 on a 2-socket system and tried
booting HEAD.

The submitter of PR 246638 reports that this fixes booting HEAD on
a threadripper.

[cem]

sys/vm/uma_core.c

2831–2833 ↗

(On Diff #72239)

I guess if we had locality information at this point it would be nice to try to the closer domains first. Clearly this is better than status quo, though.

[markj]

sys/vm/uma_core.c

2831–2833 ↗

(On Diff #72239)

The allocation here is for global structures, so there's no obvious way to choose the "right" domain, and the structures are rarely used anyway (mainly by uma_zcreate()). We only pass a domain here because startup_alloc() implements the keg allocator interface.

[cem]

sys/vm/uma_core.c

2831–2833 ↗

(On Diff #72239)

Ah, I failed to observe they were global.

[markj]

Modify _vm_phys_domain() to return -1 if the system has a memory
affinity table and the paddr is not referenced by any entry. This
happens in pmap_page_array_startup() since the page array is dense.

Handle wildcard domains in vm_phys_early_alloc(), just return memory
from any domain. Modify UMA to assert that items always come from a
valid domain.