UMA: Use LIFO for non-SMR bucket caches.

[markj]

Summary

When SMR was introduced, zone_put_bucket() was changed to always place
full buckets at the end of the queue. However, it is generally
preferable to use recently used buckets since items are more likely to
be resident in data caches. For buckets that have no constraint on
reuse, use a last-in-first-out ordering. Gleb reported that this
measurably reduces time spent in UMA when allocating from vm_page cache
zones.

[markj]

Any objections to this change? I'd like to commit soon if not.

[rlibby]

sys/vm/uma_core.c

741–744

Maybe I'm confused, but isn't this test backward? Shouldn't it be, if smr then tail else head? Maybe also better written as ub_seq == SMR_SEQ_INVALID?

[markj]

sys/vm/uma_core.c

741–744

Sorry, I missed uploading the fix. You're right, this is supposed to be ub_seq == SMR_SEQ_INVALID.

[markj]

Fix the ub_seq test.