Create a way to measure the impact of changing timer_slop
ClosedPublic
Actions

Authored by rrs on May 13 2021, 5:02 PM.

Details

Reviewers

tuexen
rrs

Group Reviewers

transport

Commits

rG362f95f52897: tcp: Add a socket option to rack so we can test various changes to the slop…
rG4f3addd94be5: tcp: Add a socket option to rack so we can test various changes to the slop…

Summary

Timer_slop, in TCP, has been 200ms for a long time. This value dates back
a long time when delayed ack timers were longer and links were slower. A
200ms timer slop allows 1 MSS to be sent over a 60kbps link. Its possible that
lowering this value to something more in line with todays delayed ack values (40ms)
might improve TCP. This bit of code makes it so rack can, via a socket option,
adjust the timer slop.

Test Plan

The initial test will be to make sure via blackbox logs that the default 200ms
timer slop shows up, but also if we change the socket option for timer slop
we get a different value. Once the initial work is done then next phase of
testing will be to arrange to run some NF experiments to see if it impacts
TCP in any way (changing it). It could be that it harms things (which I doubt) but
if it helps then we need to contemplate changing the global sysctl value.

Diff Detail

Lint

Lint Skipped

Unit

Tests Skipped

Event Timeline

rrs created this revision.May 13 2021, 5:02 PM

Herald added 1 blocking reviewer(s): transport. · View Herald TranscriptMay 13 2021, 5:02 PM

Herald added a subscriber: melifaro. · View Herald Transcript

rrs requested review of this revision.May 13 2021, 5:02 PM

tuexen accepted this revision.May 13 2021, 6:41 PM

This revision is now accepted and ready to land.May 13 2021, 6:41 PM

I realized when we set the socket option, we also need to set the rxt timer
with the new slop if an srtt has been collected.

This revision now requires review to proceed.May 18 2021, 4:50 PM

tuexen accepted this revision.May 19 2021, 7:26 AM

This revision is now accepted and ready to land.May 19 2021, 7:26 AM

These two pkt drill scripts can be used to validate that the feature is working

No changes (slop is 200ms)

Copyright (c) 2018 Randall Stewart
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
//
--tolerance_usecs=10000
--ip_version=ipv4

+0.00 sysctl -w net.inet.tcp.syncookies_only=0
+0.00 sysctl -w net.inet.tcp.syncookies=1
+0.00 sysctl -w net.inet.tcp.rfc1323=1
+0.00 sysctl -w net.inet.tcp.sack.enable=1
+0.00 sysctl -w net.inet.tcp.ecn.enable=2
Create a TCP endpoint in the ESTABLISHED state.
+0.00 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3
+0.00 fcntl(3, F_GETFL) = 0x02 (flags O_RDWR)
+0.00 bind(3, ..., ...) = 0
+0.00 listen(3, 1) = 0
+0.000 < S 0:0(0) win 1500 <mss 1460>
+0.000 > S. 0:0(0) ack 1 win 65535 <...>
+0.100 < . 1:1(0) ack 1 win 40000
+0.000 accept(3, ..., ...) = 4
+0.00 setsockopt(4, IPPROTO_TCP, TCP_LOG, [4], 4) = 0
+0.100 write(4, ..., 2920) = 2920
+0.00 > . 1:1461 (1460) ack 1 win 65535
+0.00 > P. 1461:2921 (1460) ack 1 win 65535
+0.100 < . 1:1(0) ack 2921 win 40001
+0.100 write(4, ..., 1460) = 1460
+0.00 > P. 2921:4381 (1460) ack 1 win 65535
+0.512 > P. 2921:4381 (1460) ack 1 win 65535
+0.100 < . 1:1(0) ack 4381 win 40001
Tear it down.
+0.200 < F. 1:1(0) ack 4381 win 40002
+0.00 > . 4381:4381 (0) ack 2 win 65535
+.010 close(4) = 0
+0.00 > F. 4381:4381 (0) ack 2 win 65535
+0.200 <F. 2:2(0) ack 4382 win 40002

Change slop to 50ms

rrs@lgml-rrs ~ % cat slop02.pkt

Copyright (c) 2018 Randall Stewart
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
//
--tolerance_usecs=10000
--ip_version=ipv4

+0.00 sysctl -w net.inet.tcp.syncookies_only=0
+0.00 sysctl -w net.inet.tcp.syncookies=1
+0.00 sysctl -w net.inet.tcp.rfc1323=1
+0.00 sysctl -w net.inet.tcp.sack.enable=1
+0.00 sysctl -w net.inet.tcp.ecn.enable=2
Create a TCP endpoint in the ESTABLISHED state.
+0.00 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3
+0.00 fcntl(3, F_GETFL) = 0x02 (flags O_RDWR)
+0.00 bind(3, ..., ...) = 0
+0.00 listen(3, 1) = 0
+0.000 < S 0:0(0) win 1500 <mss 1460>
+0.000 > S. 0:0(0) ack 1 win 65535 <...>
+0.100 < . 1:1(0) ack 1 win 40000
+0.000 accept(3, ..., ...) = 4
+0.00 setsockopt(4, IPPROTO_TCP, TCP_LOG, [4], 4) = 0
+0.00 setsockopt(4, IPPROTO_TCP, TCP_RACK_TIMER_SLOP, [50000], 4) = 0
+0.100 write(4, ..., 2920) = 2920
+0.00 > . 1:1461 (1460) ack 1 win 65535
+0.00 > P. 1461:2921 (1460) ack 1 win 65535
+0.100 < . 1:1(0) ack 2921 win 40001
+0.100 write(4, ..., 1460) = 1460
+0.00 > P. 2921:4381 (1460) ack 1 win 65535
+0.362 > P. 2921:4381 (1460) ack 1 win 65535
+0.100 < . 1:1(0) ack 4381 win 40001
Tear it down.
+0.200 < F. 1:1(0) ack 4381 win 40002
+0.00 > . 4381:4381 (0) ack 2 win 65535
+.010 close(4) = 0
+0.00 > F. 4381:4381 (0) ack 2 win 65535
+0.200 <F. 2:2(0) ack 4382 win 40002

It makes it easier for debugging if we remove the dup rtt log in TCP_LOG_RTT and instead
put the actual slop being used by the stack. This way we can validate in the BB log
that the slop was set properly via the socket option.

This revision now requires review to proceed.May 19 2021, 2:00 PM

rrs accepted this revision.May 24 2021, 1:29 PM

This revision is now accepted and ready to land.May 24 2021, 1:29 PM

tuexen accepted this revision.May 24 2021, 1:30 PM

Closed by commit rG4f3addd94be5: tcp: Add a socket option to rack so we can test various changes to the slop… (authored by rrs). · Explain WhyMay 26 2021, 10:46 AM

This revision was automatically updated to reflect the committed changes.

rrs added a commit: rG4f3addd94be5: tcp: Add a socket option to rack so we can test various changes to the slop….

tuexen added a commit: rG362f95f52897: tcp: Add a socket option to rack so we can test various changes to the slop….Jun 9 2021, 12:18 AM