tcp: improve KASSERT in limited retransmit

[tuexen]

Summary

When doing a limited retransmit, allow up to 2 * MSS - 1 if the Nagle algorithm has been disabled.
This fixes another triggering of this KASSERT as reported in PR 282605.

Test Plan

Run the following reproducer:

//
// Copyright (c) 2025 Michael Tuexen
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions
// are met:
// 1. Redistributions of source code must retain the above copyright
//    notice, this list of conditions and the following disclaimer.
// 2. Redistributions in binary form must reproduce the above copyright
//    notice, this list of conditions and the following disclaimer in the
//    documentation and/or other materials provided with the distribution.
//
// THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
// ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
// OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
// HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
// LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
// OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
// SUCH DAMAGE.
//

--ip_version=ipv6

 0.000 `sysctl -w net.inet.tcp.hostcache.purgenow=1`
+0.000 `sysctl -w net.inet.tcp.syncookies_only=0`
+0.000 `sysctl -w net.inet.tcp.syncookies=1`
+0.000 `sysctl -w net.inet.tcp.sack.enable=1`
+0.000 `sysctl -w net.inet.tcp.functions_default=freebsd`
+0.000 `sysctl -w kern.timecounter.alloweddeviation=0`
// Create a listening TCP socket
+0.000 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3
+0.000 setsockopt(3, IPPROTO_IPV6, IPV6_USE_MIN_MTU, [1], 4) = 0
+0.000 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
+0.000 setsockopt(3, IPPROTO_TCP, TCP_LOG, [TCP_LOG_STATE_CONTINUAL], 4) = 0
+0.000 bind(3, ..., ...) = 0
+0.000 listen(3, 1) = 0
// Establish the connection
+0.000 < S     0:0(0)                win 65535 <mss 1220,sackOK,eol,eol>
+0.000 > S.    0:0(0)       ack    1 win 65535 <mss 1220,sackOK,eol,eol>
+0.050 <  .    1:1(0)       ack    1 win 65535
+0.000 accept(3, ..., ...) = 4
+0.000 close(3) = 0
+0.000 setsockopt(4, IPPROTO_TCP, TCP_NODELAY, [1], 4) = 0
+0.000 send(4, ..., 2020, 0) = 2020
// Trigger a timer based retransmission of the two segments in flight.
+0.000 >  .    1:1221(1220) ack    1 win 65535
+0.000 > P. 1221:2021(800)  ack    1 win 65535
+1.000 >  .    1:1221(1220) ack    1 win 65535
+0.050 <  .    1:1(0)       ack 1221 win 65535
+0.000 > P. 1221:2021(800)  ack    1 win 65535
+0.050 <  .    1:1(0)       ack 2021 win 65535
// Now cwnd=3240 and ssthresh=2440.
+0.300 send(4, ..., 1018, 0) = 1018
+0.000 > P. 2021:3039(1018) ack    1 win 65535 // now we have 1018 bytes in flight.
+0.300 send(4, ..., 2703, 0) = 2703
+0.000 >  . 3039:4259(1220) ack    1 win 65535 // now we have 2238 bytes in flight.
// 3230 - 2238 = 992 is less than an MSS, so no more user data is sent out.
// The first segment was dropped in the network, the second arrived at the peer.
+0.050 <  .    1:1(0)       ack 2021 win 65535 <nop,nop,sack 3039:4259>
// Now cwnd=3240, but only 1018 are in flight.
+0.000 >  . 4259:5479(1220) ack    1 win 65535
+0.000 > P. 5479:5742(263)  ack    1 win 65535
+0.000 <  .    1:1(0)       ack 5742 win 65535
+0.000 close(4) = 0
+0.000 > F. 5742:5742(0)    ack    1 win 65535
+0.050 < F.    1:1(0)       ack 5743 win 65535
+0.000 >  . 5743:5743(0)    ack    2 win 65535

[glebius]

Thanks a lot for working on this! Sorry, I don't have enough time & skill to make a proper review, so don't wait for my review. Trusting Richard's, Chen's, Peter's and Lawrence's, and of course your Michael's expertise. Thanks a lot again!