amd64: rework PCPU allocation

[kib]

Summary

Move pcpu KVA out of .bss into dynamically allocated VA at pmap_bootstrap(). This avoids demoting superpage mapping .data/.bss. Also it makes possible to use pmap_qenter() for installation of domain-local pcpu page on NUMA configs.

Remove unecessary VM_ALLOC_ZERO from allocation of the domain-local page for pcpu.

Do not constrain allocations for doublefault, boot, and mce stacks. All these stacks are used only once (doublefault, boot) or very rare (mce).

Refactor pcpu and IST initialization by moving it to helper functions.

[Basically this is my previous patch, rebased on top of Jeff' commit, with unnecessary diffs reduced. I do not like extern struct pcpus declarations spread over sources, and have a plan to fix that after this commit lands]

Test Plan

Peter, could you, please, sanity-check this change ?

[markj]

sys/amd64/amd64/mp_machdep.c

479 ↗

(On Diff #60986)

There is a whitespace bug here.

sys/amd64/amd64/pmap.c

1706 ↗

(On Diff #60986)

pg_nx?

[markj]

sys/amd64/amd64/machdep.c

1583 ↗

(On Diff #60986)

Style: extra space after the cast. This is duplicated below.

sys/amd64/amd64/pmap.c

1712 ↗

(On Diff #60986)

Why don't we need to call amd64_bsp_pcpu_init2() here?

I would wrap this in a amd64_bsp_pcpu_reinit() function, defined in machdep.c.

1714 ↗

(On Diff #60986)

Extra newline.

[kib]

sys/amd64/amd64/pmap.c

1712 ↗

(On Diff #60986)

This is pmap_bootstrap(), called from getmemsize(). There is no point in calling amd64_bp_pcpu_init2() this early. More, I suspect that some values to put into pcpu are not yet ready.

That was the reason for doing init1/init2.

init2 is called much later in hammer_time().

[kib]

Style.

[markj]

I see a hang during boot with this patch applied on a dual socket Xeon. It occurs fairly late, after disk and USB controller drivers attach, but before I can break into the debugger. It also happens with SMP disabled.

[kib]

Preserve apic_id on temp bsp pcpu->final bsp pcpu transition.
Remove unneeded segment registration.

[kib]

In D21320#463948, @markj wrote:

I see a hang during boot with this patch applied on a dual socket Xeon. It occurs fairly late, after disk and USB controller drivers attach, but before I can break into the debugger. It also happens with SMP disabled.

Yes, I do not have USB in regular testing kernel. Apparently contig scan hung because I did unneeded (now) segment registration for pcpu range.

[markj]

I would still find it neater to somehow group the bsp pcpu reinit with related code in machdep.c.

[pho]

I ran a brief 2 hour test with D21320.61033.diff. No problems seen.