tcp: Correctly compute the retransmit length for all 64-bit platforms.

[• hselasky]

Summary

When the TCP sequence number subtracted is greater than 2**32 minus
the limit, or 2**31 minus the limit, the use of unsigned long as an
intermediate variable, may result in an incorrect retransmit length
computation on all 64-bit platforms in the TCP stack.

While at it create a helper macro to facilitate the computation of
the difference between TCP sequence numbers.

MFC after:      3 days
Sponsored by:   NVIDIA Networking

[• hselasky]

test7.c1 KBDownload

How to reproduce:

cc -DSIMULATE_32BIT_LONG test7.c && ./a.out > a.txt

cc test7.c && ./a.out > b.txt

diff -u a.txt b.txt

[rscheff]

There may be other places where tcp seq numbers are calculated using simple substraction...

[• hselasky]

In D35388#802317, @rscheff wrote:

There may be other places where tcp seq numbers are calculated using simple substraction...

I'll have a look. Could you check the sequence number in the dump's you've got, if they match this criteria?

[• hselasky]

In D35388#802317, @rscheff wrote:

There may be other places where tcp seq numbers are calculated using simple substraction...

I did a grep for unsigned long and u_long and this was the only place I could see was affected.