Page MenuHomeFreeBSD

Implement ECN++ (draft-generalized-ecn)
Needs ReviewPublic

Authored by rscheff on Jan 17 2020, 11:28 AM.



RFC3168 only allows regular data segments to be sent as
ECN-capable transport (ECT). Control (RST, FIN), pure ACK
and retransmissions are explicitly not to be sent with ECT.

However, it has been shown over the years, that this
approach stifles the usefulness of ECN, in particular when
deploying DCTCP in controlled environments.

Current work in the IETF (draft-generalized-ecn, also known
as ECN++) aims to allow all control and retransmission
packets to be marked ECT. In the case of retransmissions,
the only standardized recourse when one of those is dropped,
is to wait for the RTO timer to expire - including the
penalty imposed by having to suffer an RTO.

Furthermore, allowing control and pure ACKs to convey
additional congestion signals on the return path enables
features like AckCC (RFC5690).

Also, adding code to make non-session related control packets
(eg. RST to non-listening ports) appear idential with those in
response to recently used sessions, to prevent sidechannel
information leakage.

Test Plan

Attaching packetdrill scripts for full coverage

Diff Detail

rS FreeBSD src repository - subversion
Lint OK
No Unit Test Coverage
Build Status
Buildable 44574
Build 41462: arc lint + arc unit

Event Timeline

A suite of packetdrill test cases, to validate all combinations between ECN (disabled, enabled, passive), ECN++ (disabled, ECN+, ECN++), Client with and without ECN capability, Active and passive session setup.

  • Merge branch 'master' into ecn++ to keep diff current

Check what Linux / MacOS are doing in this space, may be beneficial to keep all this to a single sysctl instead.

Discussed with the people doing the TCP Prague / DCTCP implementation on Linux; the code there doesn't use an expicit, separate sysctl yet - all experimentation is done with hardcoded enabling of ECN and IP header markings, when their CC variant is active.

Also, RFC5562 has some additional handshake requirements (apparently added after the initial simulation, and without vetting against real-world issues observed in the internet, e.g. overriding the legacy and historic IP TOS byte with bit patterns, which effectively keep marking every packet as CE...

Due to these shortcomings of RFC5562 (known incompabilitiy to a rare, but observable misconfiguration of devices on the internet, and no other expirience in deployed stacks, will remove the "ECN+" (RFC5562) signaling part.

The sysctl will only enable full signaling according to draft-ietf-tcpm-generalized-ecn, with ecn.enable still being the "main switch".

After this diff, will add a patch to automatically enable ecn and ecnplusplus with the dctcp cc loadable module, to get these settings enabled in a VNET automatically, reducing chances that DCTCP is run without ECN, or with poor session setup performance under load (without ECN++).

  • rebase
  • removing incomplete ECN+ code
  • adding sysctl dependencies
  • deal with non-socket related control packets
rscheff retitled this revision from Implement ECN+ (RFC5562) and ECN++ feature (draft-generalized-ecn) to Implement ECN++ (draft-generalized-ecn) .Feb 8 2021, 3:05 PM
rscheff edited the summary of this revision. (Show Details)
rscheff retitled this revision from Implement ECN++ (draft-generalized-ecn) to Implement ECN++ (draft-generalized-ecn).
  • updated timestamp in man file
  • rebase
  • streamline fastpath check
  • update man date
  • have to explicitly check for ECN SYN
  • add TF2_ECN_PLUSPLUS flag to reduce cacheline churn in fastpath when checking V_ecn_generalized
  • rebase
  • bump tcp.4 datestamp added inline comments.

Typo and move full stop to a more natural place.

Current work in the IETF (draft-generalized-ecn, also known as ECN++) does not return a document.
The IETF TSV WG is where I would expect any such work to be done, but does not appear to show
any documents that might be given the moniker "ECN++". Could you provide
a better reference or link for the draft specification that is being implemented?

Ah, a TCP-specific draft, not generic to all ECN usage. So I was looking in the wrong place :)
Thanks for the pointer.

LGTM now.

By which I mean, the language in the manual page change does. Can't say more.

Typo in man page.



OK for the manpages part of the change.

  • rebase
  • simplify the fastpath check if ECT should be set.
  • streamline fastpath check
  • update man date
  • have to explicitly check for ECN SYN
  • add TF2_ECN_PLUSPLUS flag to reduce cacheline churn in fastpath when checking V_ecn_generalized
  • bump tcp.4 datestamp
  • fix man page typos
  • fix typo in man page
  • update to use the common tcp_ecn.c sourcefile
  • fix types

Can you explain what you mean by that? I'm arguably not the indended audience, but if I don't understand it, there's a chance some of the intended audience and or some interested readers won't either.


When a host receives a TCP packet to a port which is not listening, or no connection exists, or the header information is in some other way not acceptable, the host may respond with a RST (reset) packet. Some of these RST packets are sent from "regular" TCP processing (e.g. outside the sequence window) and others from non-open ports. Making these distinctable by carrying different markings - depending which code path was sending them - would give clues as to what ports/services may be reachable, and give rise to more targetted attacks.
As part if this change, when introducing this new capability (modified TCP header markings for sessions), the irregular codepath is addressed too, such that a 3rd party can not deduct which of the two different paths was sending out the RST, reducing an information side-channel.


Gotcha, thanks. Would "This value also uses ECN for RST replies to probes of non-open ports." mean the same? It looks clearer to me.