Create a way to measure the impact of changing timer_slop

[rrs]

Summary

Timer_slop, in TCP, has been 200ms for a long time. This value dates back
a long time when delayed ack timers were longer and links were slower. A
200ms timer slop allows 1 MSS to be sent over a 60kbps link. Its possible that
lowering this value to something more in line with todays delayed ack values (40ms)
might improve TCP. This bit of code makes it so rack can, via a socket option,
adjust the timer slop.

Test Plan

The initial test will be to make sure via blackbox logs that the default 200ms
timer slop shows up, but also if we change the socket option for timer slop
we get a different value. Once the initial work is done then next phase of
testing will be to arrange to run some NF experiments to see if it impacts
TCP in any way (changing it). It could be that it harms things (which I doubt) but
if it helps then we need to contemplate changing the global sysctl value.

[rrs]

I realized when we set the socket option, we also need to set the rxt timer
with the new slop if an srtt has been collected.

[rrs]

These two pkt drill scripts can be used to validate that the feature is working

No changes (slop is 200ms)

Copyright (c) 2018 Randall Stewart
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
//
--tolerance_usecs=10000
--ip_version=ipv4

+0.00 sysctl -w net.inet.tcp.syncookies_only=0
+0.00 sysctl -w net.inet.tcp.syncookies=1
+0.00 sysctl -w net.inet.tcp.rfc1323=1
+0.00 sysctl -w net.inet.tcp.sack.enable=1
+0.00 sysctl -w net.inet.tcp.ecn.enable=2
Create a TCP endpoint in the ESTABLISHED state.
+0.00 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3
+0.00 fcntl(3, F_GETFL) = 0x02 (flags O_RDWR)
+0.00 bind(3, ..., ...) = 0
+0.00 listen(3, 1) = 0
+0.000 < S 0:0(0) win 1500 <mss 1460>
+0.000 > S. 0:0(0) ack 1 win 65535 <...>
+0.100 < . 1:1(0) ack 1 win 40000
+0.000 accept(3, ..., ...) = 4
+0.00 setsockopt(4, IPPROTO_TCP, TCP_LOG, [4], 4) = 0
+0.100 write(4, ..., 2920) = 2920
+0.00 > . 1:1461 (1460) ack 1 win 65535
+0.00 > P. 1461:2921 (1460) ack 1 win 65535
+0.100 < . 1:1(0) ack 2921 win 40001
+0.100 write(4, ..., 1460) = 1460
+0.00 > P. 2921:4381 (1460) ack 1 win 65535
+0.512 > P. 2921:4381 (1460) ack 1 win 65535
+0.100 < . 1:1(0) ack 4381 win 40001
Tear it down.
+0.200 < F. 1:1(0) ack 4381 win 40002
+0.00 > . 4381:4381 (0) ack 2 win 65535
+.010 close(4) = 0
+0.00 > F. 4381:4381 (0) ack 2 win 65535
+0.200 <F. 2:2(0) ack 4382 win 40002

Change slop to 50ms

rrs@lgml-rrs ~ % cat slop02.pkt

Copyright (c) 2018 Randall Stewart
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
//
--tolerance_usecs=10000
--ip_version=ipv4

+0.00 sysctl -w net.inet.tcp.syncookies_only=0
+0.00 sysctl -w net.inet.tcp.syncookies=1
+0.00 sysctl -w net.inet.tcp.rfc1323=1
+0.00 sysctl -w net.inet.tcp.sack.enable=1
+0.00 sysctl -w net.inet.tcp.ecn.enable=2
Create a TCP endpoint in the ESTABLISHED state.
+0.00 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3
+0.00 fcntl(3, F_GETFL) = 0x02 (flags O_RDWR)
+0.00 bind(3, ..., ...) = 0
+0.00 listen(3, 1) = 0
+0.000 < S 0:0(0) win 1500 <mss 1460>
+0.000 > S. 0:0(0) ack 1 win 65535 <...>
+0.100 < . 1:1(0) ack 1 win 40000
+0.000 accept(3, ..., ...) = 4
+0.00 setsockopt(4, IPPROTO_TCP, TCP_LOG, [4], 4) = 0
+0.00 setsockopt(4, IPPROTO_TCP, TCP_RACK_TIMER_SLOP, [50000], 4) = 0
+0.100 write(4, ..., 2920) = 2920
+0.00 > . 1:1461 (1460) ack 1 win 65535
+0.00 > P. 1461:2921 (1460) ack 1 win 65535
+0.100 < . 1:1(0) ack 2921 win 40001
+0.100 write(4, ..., 1460) = 1460
+0.00 > P. 2921:4381 (1460) ack 1 win 65535
+0.362 > P. 2921:4381 (1460) ack 1 win 65535
+0.100 < . 1:1(0) ack 4381 win 40001
Tear it down.
+0.200 < F. 1:1(0) ack 4381 win 40002
+0.00 > . 4381:4381 (0) ack 2 win 65535
+.010 close(4) = 0
+0.00 > F. 4381:4381 (0) ack 2 win 65535
+0.200 <F. 2:2(0) ack 4382 win 40002

[rrs]

It makes it easier for debugging if we remove the dup rtt log in TCP_LOG_RTT and instead
put the actual slop being used by the stack. This way we can validate in the BB log
that the slop was set properly via the socket option.