witness: actually set read-only tunables in time for witness_startup

[rlibby]

Summary

SYSCTL_XXX with CTLFLAG_RDTUN and without CTLFLAG_NOFETCH should not be
used for values that are needed before SI_SUB_KLD. Otherwise they are
tuned after they are needed. Set CTLFLAG_RDTUN | CTLFLAG_NOFETCH for
the debug.witness.witness_count and debug.witness.skipspin sysctls and
add separate tunables for them, which run at SI_SUB_TUNABLES time, i.e.,
in time for witness_startup.

Sponsored by: Dell Inc.

[rlibby]

So, why mess with the types (int -> bool and int -> u_long)? Strictly speaking it is not necessary and I can revert it if you prefer. I note though that there is no TUNABLE_UINT() and a comment for TUNABLE_INT() suggests not to use it. I don't know why this is, but just converting to bool and u_long seems fine anyway.

[markj]

In D57613#1320890, @rlibby wrote:

So, why mess with the types (int -> bool and int -> u_long)? Strictly speaking it is not necessary and I can revert it if you prefer. I note though that there is no TUNABLE_UINT() and a comment for TUNABLE_INT() suggests not to use it. I don't know why this is, but just converting to bool and u_long seems fine anyway.

Converting to bool is fine IMO. sysctl_handle_bool() provides some backward compatibility for code which assumes the old type. The int -> u_long conversion is not backwards compatible but I can't imagine that it's a problem in this case.