Mostly looks great. A few general comments:

I can't prove my paranoia, so I think we should get this in, as it will be an improvement on balance.

I tend to think type / range checking something like this is overkill.. I'm OK with it as-is..

In D7393#155251, @kmacy wrote:

Are there further reservations to this change?

This is the API he has been working against for the bnxt driver so he needs this to go in before bnxt does.

Fix LINT-NOINET and a few line width problems

Update with some cleanups in reaction to comments from lstewart@

In D7251#151654, @mike-karels.net wrote:

I'm not sure my acceptance "counts" using mike-karels.net; if you add karels@, I can re-approve.

• Address ae's feedback

OK, the diff is back to the way it should be. Now to try to make arc apply my change to the correct, dependant review.

• Revert other files touched in mis-placed arc diff

• Revert mistplaced arc diff

OK, so my attempt at properly using arc has failed miserably.

address feedback from ae

Thanks for the valuable input. I will uprev with my fixes later today.

The code looks good to me; thanks for noticing that this was not in place.

In addition to helping the callout correctness, this seems to improve lock contention on tcbinfo, since this lock is now taken only when needed, not when entering every routine (thereby blocking writers, or blocking when a writer has the lock).

Looks good in terms of not killing perf. for the sorted case, and I'm fine with it as-is. However, maybe an else would be better than a goto?

In D6619#140530, @gnn wrote:

Why are we implementing new sorts in the TCP stack? The stack should depend on generic sorting functions placed in a library.

For Netflix workloads (~80Gb/s, ~80K connections), this seems to reduce the percentage of time spent in tcp_lro_sort() from ~1.1% down to 0.85%.

I'd really rather see new drivers for high performance hardware come in using the iflib interface. There will be a performance gain, and the code will be smaller and easier to maintain.

BTW, I tested a version of this patch in our Netflix code base. When serving roughly 80Gb/s with 80K TCP connections, the old method (qsort + tcp_lro_mbuf_compare_header) used 1.4% CPU, while the new (tcp_lro_sort) used 1.1% for LRO related sorting as measured by Intel Vtune. This test was done with a sysctl toggle to switch between qsort and the new sort.

Great work -- faster AND safer

So the real goal here is to be able to express to the LRO code that you have a hash that is usable as a hash (eg, not just an ordinal queue index), and yet warn the RSS code that it is not a toeplitz rss hash result. Is that correct?

Note that the hardware is only semi-RSS capable, which is why it is using an opaque flowid derived from the queue id.

Thanks, I'm good w/this version

The tcp_lro_entry_get() abstraction adds an extra compare to the critical path (the compare against NULL in the function itself, in addition to the same compare in the main routine). At least it does at the C level. Have you verified that the compiler is smart enough to continue to use a single compare?

In D5213#127689, @kmacy wrote:

Is this the final version, or is there further work to be done here?

Work continues in HEAD_MERGE/iflib branch at https://github.com/NextBSD/NextBSD.git

Ah, excellent. Is there any reason to keep this outdated review open? If you want to chuck it in the garbage pile, you can "Abandon" the revision.

@gallatin What do you think?

Very nice, thanks.

I'd forgotten that was what hps had named his flusher. But, the cool thing is that the existing tcp_lro_flush_all() does exactly what you want. All you need to do is get hps' approval to remove the __predict_false() from the check at the top of tcp_lro_flush_all().

I have no objection.

Internally at Netflix, we have a version of this patch with a slightly different name. This is because, in addition to avoiding code duplication, it also allows a bit more room to innovate with the LRO internals.

Right. Adjust updates the number of threads once SMP comes on line. Are you saying that that's still a problem?

In D5211#114965, @kmacy wrote:

In D5211#114963, @kmacy wrote:

In D5211#114961, @gallatin wrote:

In D5211#114950, @gallatin wrote:

Hmm.. Even when loading the module at boot on -current, I'm still seeing a panic:

I can confirm that moving back to SI_SUB_SMP in the DECLARE_MODULE macro fixes it enough to at least work with an ixl module loaded post-boot.

diff --git a/sys/net/iflib.c b/sys/net/iflib.c
index ab5f74c..37d970e 100644
--- a/sys/net/iflib.c
+++ b/sys/net/iflib.c
@@ -421,7 +421,7 @@ static moduledata_t iflib_moduledata = {
        NULL
 };
 
-DECLARE_MODULE(iflib, iflib_moduledata, SI_SUB_INIT_IF, SI_ORDER_ANY);
+DECLARE_MODULE(iflib, iflib_moduledata, SI_SUB_SMP, SI_ORDER_ANY);
 MODULE_VERSION(iflib, 1);
 
 MODULE_DEPEND(iflib, pci, 1, 1, 1);

Uhm ... did you pull in the corresponding update to the task queue patch? That's probably the real problem.

In D5211#114950, @gallatin wrote:

Hmm.. Even when loading the module at boot on -current, I'm still seeing a panic:

In D5211#114958, @erj wrote:

In D5211#114950, @gallatin wrote:

Hmm.. Even when loading the module at boot on -current, I'm still seeing a panic:

D5205 is supposed to fix this crash.

Hmm.. Even when loading the module at boot on -current, I'm still seeing a panic:

We still seem to be missing the device_register api required by iflib. Eg:

To follow up on my last comment, fixing this (patch appended) seems to result in the kernel actually sending down packets with csum offload, which then causes the ixl NIC to stop passing traffic, and to die occasionally with: "ixl0: Malicious Driver Detection event 0x02 on TX queue 5 pf number 0x00"

When linking, the mp_ring here conflicts with the one in the cxgbe driver if both iflib and cxgbe are included in the kernel.

I don't see a change to sys/conf/files to add iflib.c

I noticed this when trying to backport to stable..

Thanks for all the fixes, and sorry again to be a PITA..

I just did my first pass (i'll be back for more). But I have to say that this is pretty awesome.

Thanks for addressing my concerns.. Does anybody else want to comment?

It might be nice to make these general tunables that could be done centrally and apply to all drivers, but that's probably outside the scope of the review.

In D4994#106361, @hselasky wrote:

@gnn : Yes, Andrew is going to test it, probably next week.

General comments:

I may be misreading this, but it appears we are loosing the ability to set the initial pause state via a loader tunable. Would it be possible to preserve this feature by masking the IFM_ETHER_*PAUSE bits with the user's desired pause sate in the initial ifmedia_set() call?

o The device.h change is just whitespace & should probably be removed
o Have you measured the cost in terms of CPU overhead of doing this unzipping on a transmit-mostly workload (eg, busy web server, or even just net/iperf sender)? I'm assuming that it is in the noise, but it would be nice to get confirmation.

The related mlx4 review which introduces a use case for the rate limiting is https://reviews.freebsd.org/D3647