Ensure that imports into per-domain arenas are aligned.

[markj]

sys/vm/vm_kern.c

744–745

I can't remember the purpose of this global arena. Why can't the per-domain rwx arenas import directly from kernel_arena?

[kib]

sys/vm/vm_kern.c

744–745

I believe this is a consequence of r338318. I.e. it was used before that rev.

[alc]

sys/vm/vm_kern.c

703

Replace kernel_rwx_alloc by kva_domain_import.

744–745

The additional "use" before r338318 was essentially that of a flag. If the arena passed to kmem_free() wasn't the global kernel arena, then the per-domain rwx arena was selected. No operation was actually performed on the global rwx arena in kmem_free(). In other words, even before r338318 the per-domain rwx arenas could have imported directly from the global kernel arena.

I see no reason not to eliminate the global rwx arena.

[markj]

Remove the unused global rwx arena.

[markj]

Fix the function name in a KASSERT.

[alc]

sys/vm/vm_kern.c

695–696

How about adding some detail here. That the size of the returned KVA will be a multiple of KVA_QUANTUM, and the returned KVA will start at an address that is a multiple of KVA_QUANTUM. Also, there is nothing in the comments here that explains why we do this.

[markj]

• Add some comments.
• Make the per-domain rwx arenas import directly from kernel_arena instead of the per-domain "standard" kernel arenas.

[alc]

Here is some food for thought. We might be better of if we used the per-domain arenas differently. Suppose that the import function allocated the physical memory and mapped it. Further, suppose that we had a release function, and that function actually unmapped and freed physical memory. The benefits being: (1) we wouldn't have to wait for full population to trigger promotion, and (2) uma_reclaim() wouldn't trigger demotions (and hopefully later repromotions).

[markj]

In D17249#367826, @alc wrote:

Here is some food for thought. We might be better of if we used the per-domain arenas differently. Suppose that the import function allocated the physical memory and mapped it. Further, suppose that we had a release function, and that function actually unmapped and freed physical memory. The benefits being: (1) we wouldn't have to wait for full population to trigger promotion, and (2) uma_reclaim() wouldn't trigger demotions (and hopefully later repromotions).

That's a cool idea. My main concern would be fragmentation, but kmem_* already use M_BESTFIT and we can at least easily measure the amount of physical memory wasted using vmem_size().

[alc]

In D17249#367852, @markj wrote:

In D17249#367826, @alc wrote:

Here is some food for thought. We might be better of if we used the per-domain arenas differently. Suppose that the import function allocated the physical memory and mapped it. Further, suppose that we had a release function, and that function actually unmapped and freed physical memory. The benefits being: (1) we wouldn't have to wait for full population to trigger promotion, and (2) uma_reclaim() wouldn't trigger demotions (and hopefully later repromotions).

That's a cool idea. My main concern would be fragmentation, but kmem_* already use M_BESTFIT and we can at least easily measure the amount of physical memory wasted using vmem_size().

Switching the vmem free lists to FIFO instead of LIFO should further reduce fragmentation at the cost of making the arena structure larger.