Create the radix SMR structures earlier, during SI_SUB_SMR, right after
SI_SUB_VM. Use a global list of SMR structures to perform deferred
initialization of per-CPU structures. This allows us to choose a layout
for per-CPU data at SI_SUB_CPU.

Seems ok to me aside from the comment.

This fixes the problem for me.

In D24755#548937, @rlibby wrote:

This makes me a little uneasy. Allocation before attach is okay, but free is not. Do we have some kind of guarantee that no free will occur?

In D24839#548684, @avg wrote:

Mark, thank you very much for this!
I think that this change is good.
I still wonder if we can do some trick to avoid busying valid pages.
Maybe there is some way to be more sloppy when checking the validity.
E.g., we could possibly rely on a fact that with ZFS the range lock makes sure the validity cannot change because both page-in and page-out would need to lock the range.

• Simplify layout calculation.
• Add a block comment above pcpu_layout().

• Fix cpuhead initialization.
• Weaken the amd64 assertion about sizeof(struct pcpu).

Fix a syntax error reported by pho@.

@avg what do you think of this patch?

In D24621#548401, @hselasky wrote:

I need some more time to review this. Any updates?

In D24790#547203, @emaste wrote:

We should also check the other modules -- for example, ftp and irc are in common use; I'm not sure off hand what skinny and smedia are.

Don't forget to destroy the zone.

yes please

In D23848#546300, @bdrewery wrote:

I think deadlock at https://lists.freebsd.org/pipermail/freebsd-current/2020-May/076100.html may be related to this but I don't understand a lot of it.

I see the recursion: socantrcvmore_locked() calls sorwakeup_locked(), which may call sowakeup(), which may call soisconnected() if an accept filter decided to accept the connection based on data resident in the receive buffer. In this case, the so_state update is rolled back...

• Restore DPCPU indirection in kernel modules for now. The value of DPCPU_START is kld-specific, so DPCPU_BASE_OFFSET() gives the wrong value for dpcpu fields allocated from the kld region (the modspace field).
• Fix a bug in uma_pcpu_init2(): when marking the bootstrap region as allocated, we have to allocate the region at the same granularity as it gets freed, i.e., a page at a time.

Looking at DPDK, it wants to determine the number of huge pages available in each domain before performing any allocation. I think the only way to get that information today is to read vm.phys_free, which is meant to be human-readable. We could add a set of sysctls vm.domain.<domidx>.largepage.<psind>.free to provide that information.

I thought the socket and sockbuf lock already had a well-defined lock order, the socket lock is acquired first. There are many examples in e.g., the poll and kevent code for sockets.

Remove executable bits.

I verified that head panics when running the test suite with kern.geom.mirror.launch_mirror_before_timeout=0, and that it passes with this patch applied.

In D24803#545731, @kp wrote:

It'd be nice (but it's not required for this to proceed) to have a test to exercise this, i.e. create a table with counters, send some traffic through, check that the counters incremented.

• Add a vm_phys_seg[] entry for the bootstrap pcpu region.
• When pcpu_layout() backs the rest of the bootstrap region with 2MB pages, it must subtract 2MB from the allocation size for domain 0, since it is already backed by the initial pcpu bootstrap allocation.
• Rebase on D24755 to shrink the uma_core.c diff slightly.

Remove handling of pcpu kegs from zone_kva_available(). Now that such
kegs never use startup_alloc(), we don't need code to switch them from
startup_alloc() to the real pcpu slab allocator.