witness: add tunables debug.witness.lock_order_{data_count,hash_size}

[rlibby]

Summary

Add tunable debug.witness.lock_order_data_count to allow adjusting the
number of witness lock order data entries (stacks) without recompiling
the kernel. This may help to display stacks when a lock order reversal
is reported but the number of entries is exhausted before recording the
first lock order, by allowing the user to reboot with an adjusted
tunable and try again.

Tunable debug.witness.lock_order_hash_size is also provided to allow the
hash table load factor to be managed, though that is not required.

Also tweak witness_lock_order_add to avoid computing a hash when it
won't be needed because the lock order data entries are exhausted.

Sponsored by: Dell Inc.

Test Plan

Boot new kernel with these set in loader:

set debug.witness.lock_order_data_count 20000
set debug.witness.lock_order_hash_size 8191

examine

sysctl debug.witness

[markj]

sys/kern/subr_witness.c

3448

This change isn't required for correctness, right?

[rlibby]

sys/kern/subr_witness.c

3448

Correct. This would have a very minor performance benefit of avoiding calculating the hash when we're already out of lock order data entries. I can drop it, or move it to another commit, if you prefer.

[markj]

sys/kern/subr_witness.c

3448

It's fine, I just wasn't sure if I was missing something. I missed the last sentence in the description.

[rlibby]

Addressing problems uncovered in testing. I had assumed that the
mechanism used by debug.witness.witness_count actually worked, but it
didn't. I added a new patch D57613 to address the problem with
witness_count by itself, and then applied that fix here too. I also
fixed an alignment problem.