uma: Let uma_zone_reserve() do preallocation as well

[markj]

Summary

Otherwise uma_zone_reserve() calls are always paired with a
uma_prealloc() call, but the latter doesn't really do the right thing on
NUMA systems: uk_reserve is a per-domain quantity, but uma_prealloc()
treats the number of items as a global quantity and divides over the set
of NUMA domains.

[rlibby]

I like simplifiying and clarifying the API.

I'm still a little unclear on what the new semantics are though. Can you remind me why we (sometimes?) need the reservation to be per domain, and not distributed among them? Is it just because we only have the accounting per domain? And what is the difference between first touch and round robin with respect to the reserve?

Also, kmapentzone has been added since you originally posted this.

sys/kern/subr_vmem.c

726–728

I don't think I noticed this before. Why do we do the prealloc before installing our allocf? I guess this doesn't trip ZONE_ASSERT_COLD() as coded... kmapentzone is also now doing this.

sys/vm/uma_core.c

1521

"into the partial slab list" is now partially stale.

1632–1635

In my opinion, it wouldn't be terrible to move this into keg_prealloc_slab (simlar to how uma_prealloc used to be). The reason is, that would avoid adding a prealloc param (and the wrapper) for keg_alloc_slab, and prealloc is rare whereas normal alloc is not.

1662

On fail (slab == NULL) we don't own the keg lock.

4791–4792

So, for round robin the reserve/prealloc is global and not per domain? Would appreciate a comment here.

[rlibby]

In D27169#616021, @rlibby wrote:

I'm still a little unclear on what the new semantics are though. Can you remind me why we (sometimes?) need the reservation to be per domain, and not distributed among them? Is it just because we only have the accounting per domain? And what is the difference between first touch and round robin with respect to the reserve?

So maybe this should have been obvious. Presumably with first touch, we may need to satisfy the entire reserve from any particular domain whereas with round robin we are free to cast about?