Add sysctl to disable the TCP hostcache

[j-nitrology.com]

Summary

This adds a sysctl which allows you to disable the TCP hostcache. This is handy during testing of network related changes where cached entries may pollute your results, or during known congestion events where you don't want to unfairly penalize hosts.

Prior to r232346 this would have meant you would break any connection with a sub 1500 MTU, as the hostcache was authoritative. All entries as they stand today should simply be used to pre populate values for efficiency.

Test Plan

This has been running in production at Limelight

[j-nitrology.com]

This comment was removed by j-nitrology.com.

[j-nitrology.com]

The default needs to maintain the current behavior

[bz]

sys/netinet/tcp_hostcache.c

283 ↗

(On Diff #15883)

This needs to go after variable declarations

341 ↗

(On Diff #15883)

This needs to go after the variable declarations; same applies for all further down; I'll stop repeating myself ;)

470 ↗

(On Diff #15883)

Maybe we should also fix the comment to say zero instead of NULL

[rwatson]

This seems like a good idea, but fixing the style bugs (e.g., local variable definitions should be at the start of the function before any executable code) is necessary.

Making it per-VNET caused me slight pondering, but seems like the right overall choice.

I'm glad that the description of the patch indicates that code calling into the cache has been inspected for poor interactions with the cache being disabled. I assume experimentation has taken place on both IPv4 and IPv6?

[j-nitrology.com]

I've moved all functionality to post declaration and updated the comment about the return on tcp_hc_getmtu().
@rwatson yes I just figure it's a near free feature should you be using jails to do various testing. It has been tested with both v4 and v6 traffic.

[j-nitrology.com]

When someone has a moment, might this look good after the updates?

[hiren]

Seems like this is good to go in. @bz @rwatson ?

I'll go ahead with a commit in a couple days if I don't hear anything back. Thank you for looking into it.

[sbruno]

All review comments have been addressed by submitter.