Eliminate kmem_arena in preparation for NUMA

[jeff]

Summary

This eliminates kmem_arena and kmem_object. The intent is to make the API simpler so in a later patch I can use a vmem per-domain to keep kva aligned with reservations.

I replaced the hard kmem virtual address limit with a soft limit in UMA. This allows the system to much more gracefully continue working once kmem is starved. When the soft limit is reached UMA will attempt to reduce KVA footprint by flushing caches once per-second as well as firing the lowmem handler.

[markj]

sys/vm/uma_core.c

852 ↗

(On Diff #35567)

There are three instances of PAGE_SIZE * keg->uk_ppera that could be replaced with a local var as in keg_alloc_slab().

999 ↗

(On Diff #35567)

Won't this count direct-mapped pages?

3170 ↗

(On Diff #35567)

You could possibly use a separate mutex to synchronize wakeups instead of overloading uma_drain_lock. Then, uma_reclaim_wakeup() could use that mutex and we wouldn't have to wake up once per second to handle lost wakeups.

3261 ↗

(On Diff #35567)

Extra newline.

[kib]

sys/kern/subr_vmem.c

848 ↗

(On Diff #35567)

return (ENOMEM);

sys/vm/uma_core.c

3181 ↗

(On Diff #35567)

I thought about this solution.

I think I would prefer just maybe_yield() there instead of pause(). Or, only call pause when we are over the limit.

3252 ↗

(On Diff #35567)

return ();

3258 ↗

(On Diff #35567)

Blank line is required before.

[jeff]

sys/vm/uma_core.c

999 ↗

(On Diff #35567)

I consider it something of a bug that they weren't considered in kmem limits before.

3170 ↗

(On Diff #35567)

Notice there is no locking in uma_reclaim_wakeup(). Every allocation from uma to a backing store will trigger this function. I would prefer cheap but harmlessly racy synchronization.

3181 ↗

(On Diff #35567)

Maybe yield only gives up the CPU. It'll still spin acquiring every lock in uma if you don't time limit it.