Compile the default TCP stack as a module

[jtl]

Summary

Compile the default stack as a module.

The biggest changes are:

• Move tcp_do_segment() into its own file.
• Move sysctl declarations out of the files that are compiled per-stack.
• Move tcp_setpersist() into the timer code.
• Create a new file to contain the code to register the default TCP stack.

We can use this to compile both new and old versions of the default TCP stack. We can also use this to upgrade the TCP stack without reboots: compile new versions of the TCP stack, load it as a module, switch the default to the new version, and restart any applications with listen sockets.

This change still includes the default TCP stack in the monolithic kernel. You could trivially extend this so the default TCP stack was itself a module.

Test Plan

I did the following:

• Compiled the kernel, installed it, and rebooted.
• Made a trivial change to the options format produced by tcp_addoptions().
• Compiled the default stack as a module.
• Loaded the module.
• Observed the packet format for TCP packets.
• Switched the default stack to point to the version from the module.
• Observed the packet format for a new TCP connection. It contained the change.
• Switched the default stack to point to the version in the monolithic kernel.
• Observed the packet format for a new TCP connection. It returned to the normal behavior.
• Unloaded the kernel module.

[kbowling]

Should tcp_default.c move under tcp_stacks/default.c?

[jtl]

In D11105#230134, @kevin.bowling_kev009.com wrote:

Should tcp_default.c move under tcp_stacks/default.c?

That almost seems like a philosophical or existential question... :-)

No, seriously, what is the meaning of the tcp_stacks directory? In current usage (both in the public upstream repo and the private repo used at $work), the tcp_stacks directory contains code not included in the default TCP stack. Once we enable compiling the default TCP stack as a module, does even the default stack's code go there? If so, why not move tcp_output and tcp_do_segment there, too?

Also, at the moment, I am not proposing that we change the system where the default stack is included in the monolithic kernel binary. So, this same file will be used both in the kernel build as well as the module build.

Note that I'm not opposed to moving this file into the tcp_stacks directory. In fact, I considered that while making the patch. However, I decided that I had insufficient information to make the decision, as the Project has not clearly defined the meaning/purpose of the tcp_stacks directory. If we decide that is the right place for this file to live, I'm fine with that. (And, if we decide that before I commit this patch, I'm fine with making the change as part of this patch. :-) )

[kbowling]

Ok, sounds like it's not the time to have that discussion since we don't yet really know how the other stacks will compose.

[kbowling]

We are a fan of this at LLNW. My position on the potential concerns: it wont affect most users that do not modify TCP, the users that are modifying TCP are supporters and able to navigate the MFC process (which is low tempo anyway). Out of tree consumers will have to adapt, but the lack of feedback or participation make it difficult to predict what they may or may not want; it will boundary on FreeBSD 12 so they have plenty of time to adapt.

[rrs]

One of the nice things that having a module which is the default stack does is that
changes of any type can be made in the module. Then you can test those changes
before applying the changes to the main stack. That makes it rather nice that
one can try before we buy changes to the included stack. We have used that
fairly effectively at NF on deploying new versions of things (not the default stack but rack)

[jason_eggnet.com]

I agree that having the default stack as a module would be useful for A/B testing the default stack and pushing out fixes without rebooting, and is sufficiently valuable for inclusion into FreeBSD.

[rscheff]

What was the reason this Diff got effectively abandonden, jtl@?

For the reasons stated above (A/B testing, but potentially also providing quick bugfixes which don't necesitate a system reboot, this would be nice to have...