Ok, I updated this revision to just use the default value now instead of rounding down.

use the default value instead of rounding down

Modify _iflib_assert to verify driver parameters are power of 2

I would be fine modifying the iflib code to avoid the power of 2 limitation entirely, but I don't think there is significant gain for that, because we would have to add a configuration knob indicating the increment. Additionally, since some hardware likely does require power-of-2 values, we'd want to have a way for those drivers to express that restriction as well. power-of-2 restriction can't be expressed with just an increment value, so that becomes a lot trickier.

Yea, I'll add MPASS to check that the values are power of 2.

I have a couple of thoughts I had while writing this.

Document the new CONST_STRING sysctl macros

I like the addition of SYSCTL_ADD_CONST_STRING since it also ensures that we enforce that we don't have CTLFLAG_WR.

Use __DECONST instead of a poor reimplementation

In D19577#419330, @jhb wrote:

I think I'd rather have a SYSCTL_ADD_CONST_STRING or the like, but presumably you'd just get the warning again. The other hack is to cast through uintptr_t and it's what we normally do via the __DECONST() macro. I would just do that unconditionally perhaps:

char *__arg = __DECONST(arg, char *);

This version actually compilers without the warnings, but it *is* rather ugly to have to do it this way

Introduce a __drop_const macro to handle the warning

In D19577#419066, @jacob.e.keller_intel.com wrote:

Looks like the best 'fix' would be to use _Pragma to drop the warnings in SYSCTL_ADD_STRING. That will take a bit more work to get correct.

Looks like the best 'fix' would be to use _Pragma to drop the warnings in SYSCTL_ADD_STRING. That will take a bit more work to get correct.

This still doesn't quite work 100%, because it causes a -Wcast-qual warning....

Also cast isc_driver_version in SYSCTL_ADD_STRING

This needs some re-work before it's acceptable.

Uhh, oh I see why it's marked as non-const.... SYSCTL_ADD_STRING with a read-only flag doesn't know to take a const value....

Drivers can sort of get around this by using an explicit cast, but that triggers a -Wcast-qual warning.

Replace the 2048 magic number with MCLBYTES

In D19468#418973, @gallatin wrote:

I think this is a mostly theoretical issue, as you'll have had a chain which was impossible to send (eg, far too large, etc) even after defrag.

add an MPASS so we trigger a panic when in debug

Fix minor style nits

The intention here is to ensure that device drivers which need the size of the rx buffers can ensure they use the same size as iflib, (possibly incase we want to re-enable something like the old CONTIG_MALLOCWORKS again)

I'm definitely open to better/alternative solutions, and no, I don't actually have a live replication of the issue.

Move the remap++ to after the error check

I found this during code inspection while looking at a similar construction for a legacy device driver. It's possible this can never trigger, depending on how bus_dmamap_load_mbuf_sg returns EFBIG, but it definitely seemed plausible that it might happen, and potentially locking up a CPU is not a happy thing to do.

I think it would be useful to actually expose the ifl_buf_size calculation somehow, since hardware needs this information. Currently, most drivers just calculate the value in the same way that iflib does, but that seems error prone if we ever do change this in the future.

We probably should only check CSUM_TSO for TCP packets..

I think I see what's wrong... we check CSUM_TSO for IPPROTO_UDP and IPPROTO_SCTP. That doesn't make sense, since TSO is only for TCP

We are seeing tx hangs with UDP using multiple clients when offloads are disabled. Cannot reproduce on 12.0-RELEASE. Reproducible on X520 and X550.

The following pkt-gen command:

In D18545#395726, @shurd wrote:

In D18545#395423, @jacob.e.keller_intel.com wrote:

I'm not entirely certain how to reproduce the original setup that triggered the bug, so some help with that would be appreciated.

The easiest way to trigger the issue is to use net/pkt-gen to send a fixed number of packets. With the bug, pkt-gen would never complete because it wasn't notified that all packets were sent.

Stephen Hurd, this is what I believe to be the correct fix for this issue. I'm not entirely certain how to reproduce the original setup that triggered the bug, so some help with that would be appreciated.

I know I'm late commenting on this, but I think the correct fix is to initialize tx_cidx_processed to txr->desc_count -1 when we initialize.

In D17881#382258, @krzysztof.galazka_intel.com wrote:

@shurd - Turns out you've been faster. We've been working one this bug too. Our plan was to do it this way:

			if ((setmask & (IFCAP_RXCSUM | IFCAP_RXCSUM_IPV6)) !=
			    (oldmask & (IFCAP_RXCSUM | IFCAP_RXCSUM_IPV6)))
				setmask |= IFCAP_RXCSUM | IFCAP_RXCSUM_IPV6;

But it's too late for me today to figure out which has more sense ;)

If I fix the driver to enable IP checksums on TSOs even when CSUM_IP isn't set, *and* I enable IFLIB_TSO_INIT_IP, then this patch isn't necessary.

Actually, I think the reason we set NEED_CSUM_ZERO in some hardware is only because they need the IP sum zero'd during TSO, not because they need it for non-TSO packets....

Thanks! This looks great.

So this is a step in the right direction, but isn't a complete fix for the issue. We also need to enable the ip_sum workaround in this case as well. Possibly we can just move that check to after the TX_OFFLOAD4 and TX_OFFLOAD6 checks?

This doesn't seem like the right fix.

• taskqgroup: make tqg_name a constant
• taskqgroup: mark name paramter of taskqgroup_attach as const
• iflib: mark irq allocation name parameter as constant

Looks like this involves a bit more cleanup to the taskqgroup code (since ultimately the name gets passed there). due to the scope I think I'll split it into two commits.

Hmmm... I think there were. I'll double check this later today and see what I missed. I have a separate system for sending the reviews because my test system doesn't have direct access, I probably just forgot a few lines..

Remove the redundant message in iflib_device_register

Oops didn't mean to upload a new revision..

While we're cleaning this up, I'd either remove this print, or add a matching one for the tx setup, and remove the one in the caller. I realize that you can deduce which failed by how many prints you get, but that seems ... weird..

As mentioned on the other review, maybe remove the print in the caller, since it is now redundant..

Can we MFC this to stable/11? I originally found the bug on my FreeBSD 11.1 system.

I've got a copy of the github mirror and wasnt sure how to convert the git commits back to SVN revisions, so I referenced the git commits where this was broken in my summary.