In D36721#833707, @rscheff wrote:

I have a vage recollection, of something along these lines having been checked in "recently" (within last 2 years?)

Ah, here - https://reviews.freebsd.org/D25016

was that incomplete, or not implemented properly?

It shouldn't matter that much if we reduce cwnd right at the receipt of a new segment (in tcp_input), or just prior of sending out the next segment (in tcp_output)?

Add the man page entry that Michael requested :)

Sure I can do that :)

This updates and gets some of the bits I missed previously.

This will need a few more changes yet.. so its not yet complete.

Take out the commented out old code.

Address Hans ULL comment for clock freq

Addresses comments by jhb & np

Thinking about your "bintime" comment, I think this is probably a good idea, but
I suspect it needs to be applied even broader than in here. You have very similar
code in the Mlx driver since this was my original model. I have a review out on
that side as well making the more highly precision timer fix there.

Fix a comment spelling error as well as reduce out a / and %.

Fix the spelling pointed out by kib. And take Hans suggestion to cut out a / and *.

one more extra line

Address Drew's nits and also get rid of the debugging dump out printing fun.

Take out my debug stuff that I forgot to remove .. opps.. this will now
compile properly and no longer has the calibration query or the float up
of the actual hw_timestamp (had to expand the mbuf to have that and
thats not a good idea in general except for debugging).

Address the rest of Hans comment less one

Fix a nit from review of Peter Lei

After a bit of work and digging I have found on top of the timestamp
issue where LRO is reordering packets. Some in response to the
compressed ack and mbuf queuing. Others in just the normal
old flush paths. In any event this is not desirable for TCP, we need
to see the packets arrive in order. The most horrible one (for TCP)
is where the peer sends a RST followed quickly by a new SYN. This
will get reordered into SYN first then RST.

So if you are not doing soisconnected() here where does it happen at now?

These look ok but I also think it worth having Michael have a look in
case I am missing something ... :)

Just found an interesting bug when SACKs are mixing in with compressed acks. We can
push the acks in the wrong order by remembering the last compressed ack we were filling.
We should NULL the cmp field when it turns out we have to add a packet that is
not-compressable.

Drew suggested we use the mbufqueue flag as an indication that
we have stacks that are interested in the M_TSTMP_LRO so we only
do it if thats true.

Ok I have finished testing this and the pkt drills scripts show it works (one change on line 59 though had
to add the push bit.. which is correct).

You need to update files also so that you can compile this
as an option as well i.e.

Please fix what you broke in the LRO sort code above.

update the comment Michael pointed out.

Thanks for finding the debugging printf I missed in my cleanup. Lets also
take out the "no cc" printf too while we are at it. Only INVARIANT type
printfs as well as kld load failures should be in rack.