tcp: refactor cwnd during SACK transmissions and enable TSO

[rscheff]

Summary

Since the dawn of the SACK feature, TSO was never enabled
during SACK loss recovery episodes. With the use of
LRO / GRO leading to ACK compression (partial ACKs and
SACK blocks stepping by more than a singular segment),
and also features like PRR which allow increasing the
cwnd in the final phase of loss recovery (SSRB), this
frequently results in multiple passes of the
tcp_default_output() codepath.

It turns out that previously, the SACK transmission path would
only really work as the restriction to 1 MSS per call to tcp_output()
work, but allowing more than 1 MSS could result in invalid transmission
selection.

Refactoring of cwnd and moving the adjustment for SACKed data into
tcp_output() - cwnd tracking the maximum extend from snd_una - allows
both SACK loss recovery as well as SACK transmissions after RTO during
slow start.

Test Plan

On a TSO enabled NIC, with the receiver also using TSO and
LRO, verify that TSO chunks of more than 1 SMSS show up in a
packet capture. Also, when taking an RTO during a SACK loss
recovery episode, the flight size increases as per SlowStart and
Congestion Avoidance while excluding the re-transmission of
previously SACKed data.
Note that additional losses in that phase result in very small
congestion windows (since cwnd is reduced multiple times),
and entering a loss recovery then can result in lengthy
recovery until snd_max is finally reached.

[rscheff]

• change cwnd during SACK LR to flightsize
• put cwnd inflation during classic retransmissions into tcp_output()
• properly calculate available data during SACK transmissions
• allow cwnd adjustments by more than MSS (LRO/TSO side effect)
• reset scoreboard when entering LR after RTO
• various minor style changes

[rscheff]

With this change, cwnd will reflect the expected flightsize also during SACK transmissions.
This allows the proper operation of skipping previously SACKed data after an RTO, and also the use of TSO while retransmitting.
The SACK related cwin calculation previously seems to only have worked as only a single MSS per tcp_output() call could be sent.
Keep sack_rxmit set when an attempt was made to send from the scoreboard - this prevents a longer codepath before returning. If the snd_cwnd would allow for more than the current hole to be sent, we can cycle back by specifically setting sendalot only in those cases.

While there, also do some various style changes and readability enhancements.

[tuexen]

Is it possible to split this up? At least separate the style changes?

[rscheff]

In D43470#992578, @tuexen wrote:

Is it possible to split this up? At least separate the style changes?

This includes changes in
D43520
D43447
and D43355.

I'll extract the pure style changes with no functional impact as a separate one, but want to keep this homogeneous for testing.

[rscheff]

• (int32_t)min(a,b) is still not the same as imin(a,b)

[rscheff]

• rebase main

[rscheff]

• delta to D43355_sack_after_rto