• Link cap_syslog into rtsol(8)
• Use the right name for the syslog service

Use a per-keg domainset_ref to maintain per-keg round-robin, as requested
by Jeff.

This is no longer needed given the changes in D17418.

Remove malloc_domainset() and change malloc_domain() to fall back to
other domains if an allocation from the requested domain is not possible.

Add vm_domainset_iter_policy_ref_init(), needed for D17420.

• Update man page as well

• Rename to DOMAINSET_RR() and DOMAINSET_PREF()

• Address Alan's comments.

Thanks

• Drop unnecessary M_NOWAIT check.

In D17483#374212, @brett_gutste.in wrote:

Regarding the fallback solution, I believe it is okay to fall back on vmem_xalloc() regardless of whether vmem_alloc() is called with M_WAITOK or M_NOWAIT (i.e. the if statement on line 1299 is not necessary). If vmem_alloc() is called with M_NOWAIT, then the fallback vmem_xalloc() call will also have M_NOWAIT and could also allocate the range starting at 0.

I am curious about your motives. :)

In D17557#374794, @trasz wrote:

As it is now, I believe they could work in FreeBSD. In Linux, glibc already provides the built-in modules as shared libraries, so it would clash there. I've grepped the ports tree for '\<nss_.*so', and I don't see an example of anyone doing that.

Which is to say: I believe it would be rather silly thing to try install a third-party module with a name of a built-in source.

If someone actually created dynamic NSS modules with these names today, would they work? Or is there some reason that would fail? If it works, then this change would break such a configuration, though I'd be surprised if anyone actually does that. At the very least, we should document these names as being reserved by the implementation.

I was able to use this on a Dell latitude 7480 to get a working trackpad. Thank you!

• Don't cache 0 in vmem_free().
• Avoid blocking allocations in qc_import().

In D17483#373816, @brett_gutste.in wrote:

As I understand it, there are now two qualitatively different vmem_xalloc() calls that could be made when a user calls vmem_alloc(). The first is the vmem_xalloc() call used in qc_import() to fill the quantum caches when needed, and the change to this call's parameters means that a quantum cache will never contain the range from 0 to QUANTUM_SIZE - 1. The second is called directly by vmem_alloc() if the allocation size exceeds those supported by the quantum caches, and this call could still allocate the range from 0 to QUANTUM_SIZE - 1.

With quantum caching enabled, this change means that in practice vmem_alloc() will almost never allocate the range from 0 to QUANTUM_SIZE - 1. (If the allocation request hits the second vmem_xalloc() call described above, it is necessarily asking for a range whose size is greater than the quantum size. Unless this is the very first allocation request, it's unlikely that the range from 0 to QUANTUM_SIZE - 1 would be used to satisfy it.) I'm not sure whether this behavior is a big deal, but vmem consumers may not expect it.

I think we could side-step the problem entirely by having vmem_alloc() perform an M_NOWAIT allocation from the quantum cache before falling back to vmem_xalloc(). We would still attempt to import resources into the cache upon a cache miss, but we would only block in vmem_xalloc(). Does that seem reasonable?

I think that because resources would still be imported into the cache by the first vmem_xalloc() call described above and used to satisfy the allocation request on a cache miss, the behavior of not allocating the range from 0 to QUANTUM_SIZE - 1 would still apply. Please let me know whether this makes sense.

In D17419#373799, @gallatin wrote:

Stepping back for a second, are we going to get to a point where we can make malloc() and uma_zalloc() use the current thread's domainset policy, rather than one derived from the kernel object?

In D17419#373675, @jeff wrote:

I would prefer to make this change by changing the semantics of malloc_domain() to mean prefer.

If people really want a single domain we can create another policy array that means 'only this domain'. I doubt it will see much if any use however.

Just to be clear: based on this comment and the one in D17418 you're suggesting keeping malloc_domainset() but having malloc_domain() be a wrapper that just selects DSET_PREF(domain)? I'm not opposed to that, but note that I did not convert malloc_domain(M_NOWAIT) callers (all of which are in busdma). I think it's probably fine to have them fall back to other domains though. That said, with your suggested change malloc_domain() would behave differently from kmem_malloc_domain(), uma_zalloc_domain() and vm_page_alloc_domain(), which is not ideal.

Yes, or we could just change malloc_domain() to take a set argument instead of a domain int. I think it was probably a mistake to make APIs that strictly require a specific domain. All of these things could take a policy instead. With your policy iterator changes this starts to look much nicer anyway.

In D17419#371977, @mmacy wrote:

As I pointed out on IRC, this problem is not specific to hwpmc. Nonetheless it does fix the issue here.

• Add keg_fetch_free_slab().

In D17420#373676, @jeff wrote:

This is surprisingly simple. I am a little uneasy with the prospect of not doing round-robin on the keg's iterator. Can we not pass in an iterator somehow?

Looks ok to me otherwise.

In D17490#373345, @kib wrote:

I do not remember that we ever supported non-current kernel in the core/live kernel tools.

In D17483#373152, @brett_gutste.in wrote:

I believe this change means that the range from 0 to QUANTUM_SIZE - 1 cannot be allocated by vmem_alloc() when quantum caching is enabled (except as part of an allocation request that is too large to be satisfied by a quantum cache).

In D17485#373089, @oshogbo wrote:

Why don't use casper for that (e.g. cap_sysctlbyname) ?

BTW, I plan to write some tests for this code to exercise corner cases, though we don't have much precedent for in-kernel tests.

Don't fall back to vmem_xalloc(). Instead, factor out handling of
resource shortages into vmem_try_fetch() and use that if the nextfit
search fails. This way we ensure that the cursor is updated on all
M_NEXTFIT allocations, so the policy is applied strictly.

• Remove some stale comments referencing the use of a vm_map to manage memguard KVA.
• Reimplement the mapused sysctl so that we can see how much KVA is consumed at a given point in time.

In D17227#373052, @markj wrote:

Don't fall back to vmem_xalloc(). Instead, factor out handling of
resource shortages into vmem_try_fetch() and use that if the nextfit
search fails. This way we ensure that the cursor is updated on all
M_NEXTFIT allocations, so the policy is applied strictly.

Don't fall back to vmem_xalloc(). Instead, factor out handling of
resource shortages into vmem_try_fetch() and use that if the nextfit
search fails. This way we ensure that the cursor is updated on all
M_NEXTFIT allocations, so the policy is applied strictly.

• Rename and document macros.
• Annotate the global domainsets with __read_mostly.

• Fix vm_ndomains=1 case on arm64.
• Change the tunable to vm.numa.disabled.

In D17416#372648, @gallatin wrote:

You mention kstacks. I have not gone through the rest of your reviews, but I was wondering if you were working on affinitizing kstacks. Did you plan to break the cache out by domain, or drop it for affinitized kstack allocations? The default is so undersized these days, we're mostly not getting the benefit of the cache anyway (and the cache might be pessimal, since it is protected by a global lock).

Handle kib's notes.

In D17418#371888, @gallatin wrote:

Do we really need a new malloc variant? Couldn't this be done with a new malloc flag that was specific to malloc_domain(). Something like M_ALTOK or something? (or maybe M_EXACT if you want the default to be the other way around, and have malloc_domain() fall back by default)

As an aside, this same pattern could be used to implement uma_zalloc_domainset() or vm_page_alloc_domainset(). For example: https://reviews.freebsd.org/D17425 , which I posted just as an example.

Another trivial fix

Fix obvious bugs

• Use the global domainset defined in D17416.
• Don't start at domain 0 for each stack.

Remove a useless comment.