In D20477#477820, @cem wrote:

I'm still confused why some of the structures are defined in endian-specific ways — it would be good if the author could answer that since guesses are just guesses.

Agreed and apologies, I misread/miunderstood the code. I also touched base with one of the original CDG authors to sanity check and they agree this change is a good idea.

Suggest moving stailq flush loops out of cdg_cb_destroy() into an inline function, changing smoothing_factor sysctl to a SYSCTL_PROC with custom handler similar to the exp_backoff_scale sysctl, and calling flush function from both cdg_cb_destroy() and sysctl handler when smoothing_factor is set to zero

Actually, we probably need to flush the qdiffmin_q and qdiffmax_q lists in the sample_q_size == 0 branch so that a change back to non-zero smoothing_factor doesn't start operating with stale data.

Thank you for fixing this.

Thanks all for the feedback... I'm a bit rusty on working with ports. Will commit sometime this week with my src commit bit hat on and "review/approved by:" if no further feedback materialises.

Checked with one of the authors. Preference is to reference the Bitbucket URL in pkg-descr if it can only list a single URL.

Full context diff, shuffle variable order and ditch DISTVERSION in favour of a custom variable to hold the commit hash used in the tarball directory name.

@tuexen Any thoughts on the TFO case?

In D16282#347654, @kbowling wrote:

I would prefer to leave the check in cc_cdg until we run to ground why the callback was getting hit twice (in the case someone has cc_cdg as default).

In D16282#347300, @kbowling wrote:

@lstewart we debated who should NULL before submitting this, I think it can be moved out of the destructors to the places I put the KASSERTs but I'll have to double check everything. One issue I foresaw NULLing in the framework is that we have to trust that the cc_cb_destroy destructor did indeed free() and can't catch mistakes with a KASSERT. But writing a new CC isn't exactly common and will probably be cribbed from an existing one so maybe I was over thinking that angle.

In D16282#347262, @jason_eggnet.com wrote:

If the cb_destroy virtual function shouldn't null the pointer, I assert that it shouldn't free it either.

There should be another cc_assert wrapper function that checks for not null, frees then nulls, after calling the virtual function.

When ABE was added (rS331214) to New Reno and leak fixed (rS333699) , some assumptions about the state of it as the default and always ready cc seem to have changed, notably it now has a constructor and destructor (newreno_cb_destroy) for per connection state.

Final commit candidate, rebased against FreeBSD head r333598, and with M_ZERO removed from malloc call.

newreno_plugleak_v3.diff with getsockopt(2)/setsockopt(2) updated.

@wollman Many thanks for the historical and standards related context - greatly appreciated. I realised that I probably need to update getsockopt(2)/setsockopt(2) as well...

In D15358#323834, @thj wrote:

In D15358#323759, @lstewart wrote:

Still to be tested, but I think something like this would address the leak and change memory allocation to being conditional on need: newreno_plugleak_v1.diff

This reads okay.

I ran through a loop with netcat and didn't doesn't leak(of course it wouldn't!) and tested setting the abe beta values via set sockopt with a modified netcat (https://people.freebsd.org/~thj/diffs/ncabe.diff). This also doesn't leak.

Still to be tested, but I think something like this would address the leak and change memory allocation to being conditional on need: newreno_plugleak_v1.diff

@jason_eggnet.com I thought about this some more and while there is no doubt that overflow/underflow due to the function's inputs is possible and needs to be remedied, the cause of overflow in your test case is not in fact the time since congestion being too large, but the bogus value of K, which for a wmax of 1460 bytes (i.e. slightly more than 1 MSS) should be 205 per my quick check:

Oh and @jason_eggnet.com , regarding your test code, I structured things the way they are so that you can simply #include <netinet/cc/cc_cubic.h> into your userspace test.c to get access to the various window calculation inlines rather than copy/pasting them.

This fix doesn't make sense to me. If it has been a long time since the last congestion epoch resulting in an overflow in the calculated congestion window, collapsing the window from <something_large> to 1 segment is a terrible idea. There's also no sound reason that cwnd shouldn't be allowed to shrink below 1 segment, and if that causes problems elsewhere, those places should be fixed.

yoda be gone

@cem Thanks!

Update diff post r322614 commit.

Conrad: would you be willing to sanity check this sbuf change for me as well?

In D8536#249937, @cem wrote:

Looks fine. I think it would be good to note in the commit message that this is only for top-level sections and does not nest. It's obvious when you think about it, but, I like clarity.

Rebase patch against current head and address review feedback.

The off-by-one error was incorrectly attributed to the condition that checks if vsnprintf() was successful at rendering all of the specified content into the sbuf.

Thanks Tom, looking good. Will wait a few days to see if any further feedback materialises.

This looks good barring a couple of nits as noted. Please update tcp(4) and cc_newreno(4) appropriately.

I'm pretty sure this wouldn't compile as proposed. Please remember to build test patches against FreeBSD's svn head branch, as all work always gets comitted to head first before potentially being backported later.

Following up on some discussion held on the fringe of the recent developer summit at BSDCan, what this work is missing is some accompanying documentation to demonstrate the dynamic behaviour of the proposed implementation over some range of relevant network parameters, along with a critique of expected versus measured/observed behaviour. I'm not asking for a full blown academic paper - a stream of consciousness Google-doc with some data, a few X vs time plots and some comments would suffice. Happy to provide guidance if required - you know where to reach me.

Apologies for the delay in getting to this.

@kib: You prefer this?

With this patch applied on top of svn head r308477, I built a custom memstick image, wiped the harddrives and reran my build guide steps including creation of the RAIDZ zpool with "-O checksum=skein" option and can confirm the system boots, so although I did not test without this patch applied, I think it's safe to say the patch fixed things given that there have been no other relevant changes in sys/boot between the revision I tested with previously (r307747) and r308477.

To clarify, I did not test prior to this commit, so am not sure if it ever worked. I merely looked up svn log history on sys/boot and this seemed like a plausible culprit assuming there has been a regression.

Is it possible this broke UEFI booting from RAIDZ pools with checksum=skein? If I omit "-O checksum=skein" from my zpool create step everything works great, but with skein checksums, boot1 sees my pool (prints my pool name after "found the following pools" message) but hangs where it should have run loader.efi and then requires a hard reset.

In D6442#136542, @pfg wrote:

Hello;

The improvement is probably not huge but given that we can provide better randomness ... why not? :)

In D6105#130864, @hiren wrote:

In D6105#130862, @lstewart wrote:

But why do we need such finely grained control? I don't get it. Either it works or it doesn't. We shouldn't be doing 6675 piecemeal. We should be doing 6675 in full and enabled by default. Providing any level of minutiae beyond enabled/disabled is not only unnecessary but a bad idea IMO.

iirc, @rrs tried the patch and it didn't work well for his workload. I
am just trying to avoid such situations.

But why do we need such finely grained control? I don't get it. Either it works or it doesn't. We shouldn't be doing 6675 piecemeal. We should be doing 6675 in full and enabled by default. Providing any level of minutiae beyond enabled/disabled is not only unnecessary but a bad idea IMO.

In D5872#130806, @sepherosa_gmail.com wrote:

In D5872#130805, @lstewart wrote:

In D5872#130179, @sepherosa_gmail.com wrote:

We probably can leave the cwnd resetting to later rexmt timeout or possible later fast retransmit (I think fast retransmit could kick in under some cases, if ENOBUFS happened); instead of resetting the cwnd immediately upon ENOBUFS.

Please leave the manipulation of cwnd as is so as to avoid conflating two different changes. The manipulation of cwnd on local drop has nothing to do with the subject of this particular change.

Yep, I am not going to delete the cwnd reset in this patch.

In D5872#130179, @sepherosa_gmail.com wrote:

We probably can leave the cwnd resetting to later rexmt timeout or possible later fast retransmit (I think fast retransmit could kick in under some cases, if ENOBUFS happened); instead of resetting the cwnd immediately upon ENOBUFS.

Why isn't there simply a do_rfc6675 knob that supersedes this and the previously committed work?

In D5872#128556, @hiren wrote:

In D5872#128555, @lstewart wrote:

I thought that had been fixed ages ago... oops.

Fixed? i.e. doing something other than setting cwnd to 1 seg?

I thought that had been fixed ages ago... oops. It should be calling cc_cong_signal() with a new congestion type. Just leave that line as is for the moment though as Mike says.

... but add a macro to check that the rexmit/persist timer is armed if appropriate! Should be added higher up though so that it is checked before all return statements in the vicinity.

I agree with Mike's proposal (although FYI, I do belive tcp_output() will send an ACK on RTO). TCP ACKs are intentionally unreliable by design and setting the retransmit timer there is nonsense - either there is a bug elsewhere which needs to be fixed, or it is trying to paper over local ACK loss in a dubious manner. The ENOBUFS case should also become a thing of the past when the back pressure work goes in any way. For the immediate change, perhaps replacing with a macro that expands to a KASSERT to double check the appropriate conditions for the retransmit or persist timers being set would be a good idea. The macro should be used elsewhere in tcp_output() and tcp_intput() as well but that can be done in follow up commit(s).

Apologies for the delay in getting to this, still heads down wrapping up my PhD thesis. The comments from Kib and Mark all appear to have been addressed and the changes look good. I don't really have an opinion on the "report first or most recent" error issue.

You're a bit "warmer" with the revised changes but still a fair ways off the mark. Apologies to anyone watching but I'm too time poor at the moment to engage in the proper but protracted back-and-forth public Phabricator discussion to resolve all the problems with this work. Perhaps another brief sync on IRC is in order and you can always summarise the chat logs here as context for others.

In D5124#109833, @hiren wrote:

@lstewart I agree and I think its time to improve the initcwnd handling code. But that'd be a separate commit.

What is your take on the problem at hand? Are you okay with the diffs? I'd like to get this in and possibly MFC for 10.3.

Oops, that should of course be 4 segments, not 3 (though recall that we really need an initcwnd_bytes variable as well in order to fully capture the spirit of the RFC 3390 and later RFCs - something perhaps you can add as part of this work).

I would suggest that the code to handle RFC3390 should be merged with the new code i.e. the net.inet.tcp.rfc3390 sysctl should become a SYSCTL_PROC and simply set V_tcp_initcwnd_segments=3 behind the scenes, and return the evaluated result of "V_tcp_initcwnd_segments==3" as the sysctl value.