tcp: send SACK rescue retransmission also mid-stream

[rscheff]

Summary

Previously, SACK rescue retransmissions would only trigger
on a partial ACK with no SACK block, when the loss recovery
was entered while no more data was ready to send beyond that.

However, when additional data is in the send buffer, it's
common for the sender to transmit data beyond that point.
Nevertheless, these packets may still hit an overloaded queue and
get dropped.

Construct a rescue retransmission to only include the initial
loss recovery point, such that subsequent further loss can
be reacted to with another congestion reaction.

Test Plan

// Testing SACK RTO and mid-stream rescue retransmission

--tolerance_usecs=50000

0.0 `sysctl net.inet.tcp.cc.algorithm=newreno`
+0.1 `sysctl net.inet.tcp.initcwnd_segments=10`
+0.1 `sysctl net.inet.tcp.ecn.enable=1`
+0.1 `sysctl net.inet.tcp.do_prr=1`
+0.1 `sysctl net.inet.tcp.sack.revised=1`
+0.1 `sysctl net.inet.tcp.hostcache.purgenow=1`

// Create a listening TCP socket.
+0.50 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3
+0.01 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
+0.01 setsockopt(3, SOL_SOCKET, SO_SNDBUF, [1048576], 4) = 0
+0.01 setsockopt(3, SOL_SOCKET, SO_DEBUG, [1], 4) = 0
+0.01 bind(3, ..., ...) = 0
+0.01 listen(3, 1) = 0


// Establish a TCP connection.
+0.04 < S  0:0(0) win 65535 <mss 1000, sackOK, wscale 10, eol, nop, nop >
+0.00 > S. 0:0(0) ack 1 win 65535 <mss 1460,nop,wscale 6,sackOK,eol,eol>
+0.00 <  . 1:1(0) ack 1 win 65535
+0.00 accept(3, ..., ...) = 4
+0.01 setsockopt(4, SOL_SOCKET, SO_DEBUG, [1], 4) = 0


2.00 write(4, ..., 250000) = 250000
+0 > .    1:1001(1000) ack 1
+0 > . 1001:2001(1000) ack 1
+0 > . 2001:3001(1000) ack 1
+0 > . 3001:4001(1000) ack 1
+0 > . 4001:5001(1000) ack 1
+0 > . 5001:6001(1000) ack 1
+0 > . 6001:7001(1000) ack 1
+0 > . 7001:8001(1000) ack 1
+0 > . 8001:9001(1000) ack 1
+0 > . 9001:10001(1000) ack 1

+0.001 < .    1:1(0) ack 2001 win 65535
+0 > . 10001:11001(1000) ack 1
+0 > . 11001:12001(1000) ack 1
+0 > . 12001:13001(1000) ack 1
+0 > . 13001:14001(1000) ack 1

+0.001 < .    1:1(0) ack 4001 win 65535
+0 > . 14001:15001(1000) ack 1
+0 > . 15001:16001(1000) ack 1
+0 > . 16001:17001(1000) ack 1
+0 > . 17001:18001(1000) ack 1

+0.001 < .    1:1(0) ack 6001 win 65535
+0 > . 18001:19001(1000) ack 1
+0 > . 19001:20001(1000) ack 1
+0 > . 20001:21001(1000) ack 1
+0 > . 21001:22001(1000) ack 1

+0.001 < .    1:1(0) ack 8001 win 65535
+0 > . 22001:23001(1000) ack 1
+0 > . 23001:24001(1000) ack 1
+0 > . 24001:25001(1000) ack 1
+0 > . 25001:26001(1000) ack 1

+0.001 < .    1:1(0) ack 10001 win 65535
+0 > . 26001:27001(1000) ack 1
+0 > . 27001:28001(1000) ack 1
+0 > . 28001:29001(1000) ack 1
+0 > . 29001:30001(1000) ack 1

+0.001 < .    1:1(0) ack 12001 win 65535
+0 > . 30001:31001(1000) ack 1
+0 > . 31001:32001(1000) ack 1
+0 > . 32001:33001(1000) ack 1
+0 > . 33001:34001(1000) ack 1

+0.001 < .    1:1(0) ack 14001 win 65535
+0 > . 34001:35001(1000) ack 1
+0 > . 35001:36001(1000) ack 1
+0 > . 36001:37001(1000) ack 1
+0 > . 37001:38001(1000) ack 1

+0.001 < .    1:1(0) ack 16001 win 65535
+0 > . 38001:39001(1000) ack 1
+0 > . 39001:40001(1000) ack 1
+0 > . 40001:41001(1000) ack 1
+0 > . 41001:42001(1000) ack 1

+0.001 < .    1:1(0) ack 18001 win 65535
+0 > . 42001:43001(1000) ack 1
+0 > . 43001:44001(1000) ack 1
+0 > . 44001:45001(1000) ack 1
+0 > . 45001:46001(1000) ack 1

+0.001 < .    1:1(0) ack 20001 win 65535
+0 > . 46001:47001(1000) ack 1
+0 > . 47001:48001(1000) ack 1
+0 > . 48001:49001(1000) ack 1
+0 > . 49001:50001(1000) ack 1

+0.001 < .    1:1(0) ack 22001 win 65535
+0 > . 50001:51001(1000) ack 1
+0 > . 51001:52001(1000) ack 1
+0 > . 52001:53001(1000) ack 1
+0 > . 53001:54001(1000) ack 1

+0.001 < .    1:1(0) ack 24001 win 65535
+0 > . 54001:55001(1000) ack 1
+0 > . 55001:56001(1000) ack 1
+0 > . 56001:57001(1000) ack 1
+0 > . 57001:58001(1000) ack 1

+0.001 < .    1:1(0) ack 26001 win 65535
+0 > . 58001:59001(1000) ack 1
+0 > . 59001:60001(1000) ack 1
+0 > . 60001:61001(1000) ack 1
+0 > . 61001:62001(1000) ack 1

+0.001 < .    1:1(0) ack 28001 win 65535
+0 > . 62001:63001(1000) ack 1
+0 > . 63001:64001(1000) ack 1
+0 > . 64001:65001(1000) ack 1
+0 > . 65001:66001(1000) ack 1

+0.001 < .    1:1(0) ack 30001 win 65535
+0 > . 66001:67001(1000) ack 1
+0 > . 67001:68001(1000) ack 1
+0 > . 68001:69001(1000) ack 1
+0 > . 69001:70001(1000) ack 1

+0.001 < .   1:1(0) ack 30001 win 65535 <sack 31001:32001, nop, nop>
+0 > . 70001:71001(1000) ack 1

+0.001 < .   1:1(0) ack 30001 win 65535 <sack 33001:34001 31001:32001, nop, nop>
+0 > . 71001:72001(1000) ack 1

+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 33001:35001 31001:32001, nop, nop>
+0 > . 30001:31001(1000) ack 1

+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 33001:36001 31001:32001, nop, nop>
+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 33001:37001 31001:32001, nop, nop>
+0 > . 32001:33001(1000) ack 1

+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 33001:38001 31001:32001, nop, nop>
+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 33001:39001 31001:32001, nop, nop>
+0 > . 72001:73001(1000) ack 1

+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 33001:40001 31001:32001, nop, nop>
+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 33001:41001 31001:32001, nop, nop>
+0 > . 73001:74001(1000) ack 1

+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 33001:42001 31001:32001, nop, nop>
+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 33001:43001 31001:32001, nop, nop>
+0 > . 74001:75001(1000) ack 1

+0.0015 < .  1:1(0) ack 30001 win 65535 <sack 45001:46001 33001:43001 31001:32001, nop, nop>
+0 > . 43001:44001(1000) ack 1

+0.0015 < .  1:1(0) ack 30001 win 65535 <sack 45001:47001 33001:43001 31001:32001, nop, nop>
+0 > . 44001:45001(1000) ack 1

+0.0020 < .  1:1(0) ack 30001 win 65535 <sack 50001:51001 45001:47001 33001:43001, nop, nop>
+0 > . 47001:48001(1000) ack 1
+0 > . 48001:49001(1000) ack 1
+0 > . 49001:50001(1000) ack 1

+0.0020 < .  1:1(0) ack 30001 win 65535 <sack 54001:55001 50001:51001 45001:47001, nop, nop>
+0 > . 51001:52001(1000) ack 1
+0 > . 52001:53001(1000) ack 1
+0 > . 53001:54001(1000) ack 1

+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 54001:56001 50001:51001 45001:47001, nop, nop>
+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 54001:57001 50001:51001 45001:47001, nop, nop>
+0.0005 < .  1:1(0) ack 30001 win 65535 <sack 54001:58001 50001:51001 45001:47001, nop, nop>
+0.0010 < .  1:1(0) ack 30001 win 65535 <sack 59001:60001 54001:58001 50001:51001, nop, nop>
+0 > . 58001:59001(1000) ack 1

+0.0050 < .  1:1(0) ack 32001 win 65535 <sack 59001:60001 54001:58001 50001:51001, nop, nop>
+0 > . 75001:75001(0) ack 1

+0.0010 < .  1:1(0) ack 43001 win 65535 <sack 59001:60001 54001:58001 50001:51001, nop, nop>
+0 > . 75001:75001(0) ack 1

+0.0045 < .  1:1(0) ack 44001 win 65535 <sack 59001:60001 54001:58001 50001:51001, nop, nop>
+0 > . 75001:75001(0) ack 1

+0.0005 < .  1:1(0) ack 47001 win 65535 <sack 59001:60001 54001:58001 50001:51001, nop, nop>
+0 > . 75001:75001(0) ack 1

+0.0020 < .  1:1(0) ack 48001 win 65535 <sack 59001:60001 54001:58001 50001:51001, nop, nop>
+0 > . 75001:75001(0) ack 1

+0.0005 < .  1:1(0) ack 49001 win 65535 <sack 59001:60001 54001:58001 50001:51001, nop, nop>
+0 > . 75001:75001(0) ack 1

+0.0005 < .  1:1(0) ack 51001 win 65535 <sack 59001:60001 54001:58001, nop, nop>
+0 > . 75001:75001(0) ack 1

+0.0005 < .  1:1(0) ack 52001 win 65535 <sack 59001:60001 54001:58001, nop, nop>
+0 > . 75001:75001(0) ack 1

+0.0005 < .  1:1(0) ack 53001 win 65535 <sack 59001:60001 54001:58001, nop, nop>
+0 > . 75001:75001(0) ack 1

+0.0005 < .  1:1(0) ack 58001 win 65535 <sack 59001:60001, nop, nop>
+0 > . 75001:75001(0) ack 1

+0.0020 < .  1:1(0) ack 60001 win 65535
// previous behavior
//+0 > . 75001:75001(0) ack 1
// new behavior - rescue retransmission including snd_recover, and further data (based on cwnd)
+0 > . 71001:72001(1000) ack 1
+0 > . 75001:76001(1000) ack 1

+0.240 > . 60001:61001(1000) ack 1
+0.040 < .   1:1(0) ack 61001 win 65535 <sack 69001:70001, nop,nop>

+0 > . 61001:62001(1000) ack 1
+0 > . 62001:63001(1000) ack 1

+0.001 < .   1:1(0) ack 63001 win 65535 <sack 69001:70001, nop,nop>
+0 > . 63001:64001(1000) ack 1
+0 > . 64001:65001(1000) ack 1
+0 > . 65001:66001(1000) ack 1

+0.001 < .   1:1(0) ack 65001 win 65535 <sack 69001:70001, nop,nop>
+0 > . 66001:67001(1000) ack 1
+0 > . 67001:68001(1000) ack 1
+0.001 < .   1:1(0) ack 67001 win 65535 <sack 69001:70001, nop,nop>
+0 > . 68001:69001(1000) ack 1
+0 > . 70001:71001(1000) ack 1
+0.001 < .   1:1(0) ack 70001 win 65535


+1.00 close(4) = 0