Better boot_pages autotuning.

[glebius]

Summary

Followup on r302393 by cperciva. Always calculate amount of pages that
are required to allocate UMA zones, not only if size of a zone is larger
than page size.

• Introduce another stage of UMA startup, which is entered after vm_page_startup() finishes. After this stage we don't yet enable buckets, but we can ask VM for pages. Rename stages to meaningful names while here. New list of stages: BOOT_COLD, BOOT_STRAPPED, BOOT_PAGEALLOC, BOOT_BUCKETS, BOOT_RUNNING. Enabling page alloc earlier, allows us to dramatically reduce number of boot page allocation. What is more important it becomes consistent across different machines. Now only UMA internal zones actually need to use startup_alloc().

• Introduce uma_startup_count() function, to avoid code duplication. It calculates sizes of zone zone and keg zone, and calculates how many pages UMA need to bootstrap. Keep count not only of zone structures, but also of kegs, slabs and hashes. Remove the hardcoded vm.boot_pages completely.

• Hide uma_startup_foo() declarations from public file.

• Provide several DIAGNOSTIC printfs on boot_pages use.

• Bugfix: when calculating zone of zones size use (mp_maxid + 1) instead of mp_ncpus. Use resulting number not only in the size argument to zone_ctor() but also as args.size.

[imp]

This looks fairly sane to me. I see no craziness, except maybe the last += 4. And even that seems sane.

[kib]

sys/vm/vm_page.c

538

Which pmap ?

Are you sure that non-x86 pmaps are limited to one allocation during pmap_init() ?

[glebius]

sys/vm/vm_page.c

538

amd64. Not sure, that's why posted the review :) I will review other arches and followup.

[glebius]

A bigger approach to boot_pages management. The most important
change is that we actually can ask VM for pages much earlier
than we do now right after vm_page_startup(), thus do so.

o Introduce another stage of UMA startup, which is entered

after vm_page_startup() finishes. After this stage we don't
yet enable buckets, but we can ask VM for pages. Rename
stages to meaningful names while here. New list of stages:
BOOT_COLD, BOOT_STRAPPED, BOOT_PAGEALLOC, BOOT_BUCKETS,
BOOT_RUNNING.

o Enabling page alloc earlier, allows us to dramatically reduce

number of boot page allocation. What is more important it becomes
consistent across different machines. Now only UMA internal zones
actually need to use startup_alloc().

o Hide uma_startup_foo() declarations from public file.
o Introduce uma_startup_count() function, to avoid code duplication.

It calculates sizes of zone zone and keg zone, and calculates
how many pages UMA need to bootstrap.

o Provide several DIAGNOSTIC printfs on boot_pages use.
o Bugfix: when calculating zone of zones size use (mp_maxid + 1)

instead of mp_ncpus. Use resulting number not only in the size
argument to zone_ctor() but also as args.size.