I haven't been able to find a repro scenario for the leak outside of $WORK's repo, but by inspection this fix is obviously correct so I'm reopening this.

I can confirm don't have any out-of-tree consumers of linuxkpi, so looks fine to me.

Thanks for looking into this for me.

Fix lock leak. Sorry, this review was based off of an
older version of my patch that didn't have that fixed.
I've confirmed that this is now in sync with what was
tested internally at $WORK

Change the value of M_NORECLAIM to avoid the style issue

Fix overflap between new flag and count field

Sorry about the delay in coming back to this; I've had to deal with a number of critical issues at work. I ported the patch that we tested internally to main and posted it to this review:

sorry I got distracted by a different critical issue at work. Will be able to return to this next week

This patch doesn't work. After the goto retry, nothing stops the cancelling thread from seeing state still in WORK_ST_CANCEL, and then the timer thread can come in and change it to TASK, causing the same bug.

I don't think that spinning is the right answer. That could hold the CPU that the callout needs to make progress, resulting in a livelock.

Removing the ability to create unnamed vlans is an unnecessary POLA violation. It can make sense for the user to establish a convention that vlanX is for vlan tag X, but we absolutely shouldn't bake that assumption into the kernel.

Try putting entries in the witness_order_list_entry in kern/subr_witness.c specifying the ifnet_sx -> in_multi_sx ordering as correct. That should get better information. Likely the issue is a cycle of length larger than 2 (e.g. in_multi_sx before mystery lock, mystery lock before ifnet_sx, ifnet_sx before in_multi_sx).

In D26879#599402, @markj wrote:

That shouldn't be true. Did you test that recently? I'm not sure if it'll work for standalone module builds since KDTRACE_HOOKS might not be defined.

In D25085#560681, @lutz_donnerhacke.de wrote:

In my eyes, throwing away packets which successfully reached the destination for potential QoS/routing issues, is unexpected.

What's the status of this? We're hitting this problem very frequently at Isilon and this patch does fix the issue.

In D24902#550400, @rrs wrote:

Has discussed on the transport call, I will be updating these to have a uint64_t and a nano-second basis for the values (including t_starttime I think since this
is what you compare these against) :)

In D7821#510126, @rscheff_gmx.at wrote:

Is this still alive? As ECN has become more interesting recently, something like this is probably needed for RFC3168 ECN flows.

Also, forgive my ignorance, but can the LRO code add some side-band information to the upstream - e.g. number of packets coalesced, or number of CE marks observed? This would be very relevant in context of Accurate ECN. Where for RFC3168 it is sufficience that any segment marked as CE to appear in the final header, for AccECN the actual number of the CE marks, and also the number of coalesced packets would be relevant, to arrive at an appropriate alpha (reduction) value.

By just moving any number of CE marks to a single, the AccECN CC reaction would be more strong, shedding more bandwidth than appropriate...

In D22198#485757, @markj wrote:

Do you happen to have a test case for this? We don't have any ND6 regression tests at the moment. :(

I suspect that means that your BIOS pre-allocated enough bus numbers to the bridge to account for the VFs. Perhaps we should add a pcib method that recursively checks that a contiguous range of bus numbers is assigned to the bridge?

In D15337#333205, @mmacy wrote:

@rstone do you think you might be able to share the work that you or Jeff did on the frontend to adaptively lower the minRTO if the pipe was full?

In D11560#326314, @kib wrote:

I do not see such code in mlx4_en_rx.c. Could you point it out, please ?

In D11560#326054, @kib wrote:

Well, from my reading, your patch and this patch are quite similar, but this review provides two interesting optimizations.

• Use page-size fragments if MTU >= PAGE_SIZE, already discussed on lists for mlx5.
• Keep around mbufs and memory for unused fragments, if the received packed was shorter than MTU.

If these changes are transplanted into your patch, I believe the things start look almost identical. I want to confirm my understanding with Ryan.

The message that I got at BSDCan last year is that they wanted to see at least a full IETF draft before these changes could be merged in, which is why my work on it stalled. Has people's position on this changed?

FWIW, I had a short email conversation with the writers of that draft about a year ago and they went radio silent pretty quickly. I assumed that they weren't working on it anymore.

In D14559#308712, @asomers wrote:

Where is route.subr? And where did ipv4_move_subnet_route come from? I think you used the wrong base revision to generate the review.

Move non-fib test into separate test file

I've written some unit tests that covering these cases here:

I don't think that this conversation is going to go anywhere productive. Do you have specific technical objection to raise about this change?

In D14547#306224, @eugen_grosbein.net wrote:

Care to elaborate what are you trying to achieve by moving subnet route between interfaces?

Uploaded the wrong test to this review; fixed now

Move route(8) tests to their own atf test file

In D14547#305453, @eugen_grosbein.net wrote:

In D14547#305425, @rstone wrote:

In any case, even if the use-case is weird, the current behaviour is clearly buggy and needs to be fixed in some way.

Well, I guess so. But original configuration may be supposed to be "buggy" too (read: not supported). What if you use /16 mask for single interface only and assign IP from the same prefix to other interfaces using /32 mask?

In D14547#305139, @eugen_grosbein.net wrote:

And you do you insist in doing things manually while moving IP (first route change, then reassign IP) instead of just allowing system do all housekeeping for you by reassigning IP first?

Add ATF test cases for IPv4/IPv6 subnet route migration

In D14547#305375, @asomers wrote:

The fibs must actually exist. Set net.fibs=6 in /boot/loader.conf and reboot. Also, are you sure you need so many? I don't think there are any tests there that require more than two.

In D14547#305342, @asomers wrote:

In tests/sys/netinet/fibs_test.sh there are some ATF tests that check this kind of functionality. They do it on separate FIBs so they won't upset the system running the tests, and they use tap(4) and/or epair(4) interfaces. You should be able to copy one of those to create a regression test for this problem.

Check for NULL pointers

In D14291#300646, @ae wrote:

How it supposed to work for RADIX_MPATH case? I know RADIX_MPATH is currently broken, but anyway...

Clarify manpage update as per rgrimes' suggestion