Page MenuHomeFreeBSD

introduce new 'ice' driver for Intel E800 Ethernet controllers
ClosedPublic

Authored by erj on Oct 9 2019, 7:06 PM.

Details

Summary

The ice driver controls the Intel E800 Ethernet controllers. It is based
on the iflib framework. Initially, support for basic Tx/Rx functionality
is included.

Future work will enable virtual functions using the virtchnl interface and
the iavf virtual function driver.

Diff Detail

Lint
Lint OK
Unit
No Unit Test Coverage
Build Status
Buildable 31222
Build 28874: arc lint + arc unit

Event Timeline

erj added a reviewer: iflib.Oct 9 2019, 7:21 PM

I just looked for a short time.. more comments later..

sys/dev/ice/ice_common_txrx.h
95

Can you remove or properly comment these lines (eg, // is frowned upon). When I sent you this for ixl, I never intended those lines be kept.

Also, very sad/disappointed that ice has this horrible restriction that makes iflib's transmit routine more complex.

177

Cool. Can you talk about what can be tunneled?

197

Why a counter and not just a uint64_t?

Note that access to the tx/rx rings should be serialized at the point your touching counters. Counters burn 16 bytes per element per cpu, due to how UMA works.

337

I hate this.

Maybe I'm just old fashioned, but I'd check for the 2 most common cases first (good checksum, or no checksum at all), first and likely not even increment counters for them. Eg, find all the status0 bits that would be set for a good packet, if they are all set, set the flags in one go and the data to 0xffff and be done. No counters.

Then I'd decode all the fiddly stuff for the error cases and increment counters there.

I just looked for a short time.. more comments later..

Replied to some comments. Hopefully Eric can give more feedback on the counters thing. I dont' remember if we had a problem with uint64 or not.

sys/dev/ice/ice_common_txrx.h
95

Yea, I had forgotten these were still here.

I really don't like this either, but there's unfortunately nothing we can do about it now :(

197

Eric, can you recall why these counters are used instead of just using a u64? I don't remember offhand.

337

Makes some sense.

erj added a subscriber: erj.Oct 10 2019, 1:25 AM
erj added inline comments.
sys/dev/ice/ice_common_txrx.h
177

I'm not entirely sure, but the description for this field in the datasheet is almost exactly the same as it is in Fortville. The only difference in the field definition is that it looks like you can make Columbiaville calculate the tunneling UDP checksum, too.

197

From what I can remember, it was motivated by how we were storing these per-queue stats in the VSI struct instead of storing them in each queue struct. We're incrementing the stats in the hot path, so we didn't want to take some sort of per-VSI lock to protect the counters while they're being incremented. The counter(9) functionality looked like it would get around that requirement.

We could've stored each counter in each queue, but I think we were concerned about how much memory that could use and how it could be annoying to see what's incremented across many queues. Though, thinking about the per CPU memory usage, it might not be that bad to keep them in a per-queue struct. We also did have an implementation that actually periodically collected the stats from each queue (in the update_admin_status() function I think) and it added each queue counter to the VSI (so that we could just stick with uint64_t's), but I think that ran into some other problem. It might have been that we would've had to take a lock on each Tx/Rx ring to get the counter values, and that didn't seem like a good idea.

All that said, these were intended to help debug problems and validate that the hardware checksums were working; if it seems like there might be more harm than good or if they need to be reimplemented, it might be easier to just leave it out or have it compiled out upstream.

erj added inline comments.Oct 10 2019, 1:27 AM
sys/dev/ice/ice_common_txrx.h
337

It does seem like a good change to make, but I wonder if it'll have a positive performance impact.

gallatin added inline comments.Oct 10 2019, 12:07 PM
sys/dev/ice/ice_common_txrx.h
197

I really believe the counters should be stored per-ring, and summed without a lock.

Don't forget a manpage. :-)

Don't forget a manpage. :-)

Good point, thanks! I think we have some stuff already prepared but not included yet. I'll look into that.

Update ice driver review to address feedback

The review now includes updated code, including changes to address the
review comments from Drew regarding the checksum calculations and statistics.

Update ice driver code

erj commandeered this revision.Mar 20 2020, 7:08 PM
erj added a reviewer: jacob.e.keller_intel.com.
erj updated this revision to Diff 69729.Mar 20 2020, 7:09 PM
  • ice: Add missing includes to ice_iflib.h
  • ice: Correct filename used in Makefile
  • ice: Add files.amd64 entries for ice
  • ice: Add "device ice" as a kernel config option; add it to GENERIC
  • ice: Add ice module to sys/modules Makefile
  • ice: Remove SR-IOV files/option from Makefile
  • ice: Update ice driver code to a current version.
erj updated this revision to Diff 69730.Mar 20 2020, 7:12 PM
  • Remove extraneous line breaks from files.amd64
  • ice: Fix spelling mistake caught by arcanist
erj updated this revision to Diff 69935.Mar 27 2020, 9:37 PM
  • Add ddp to kernel and build process
  • ice: Use fancier ice module Makefile from OOT component
  • Move ice files.amd64 entries to appropriate place
  • ice: Update description in NOTES
erj added reviewers: adrian, np.Mar 27 2020, 9:43 PM
erj added a subscriber: adrian.

I'll add @adrian to see if I'm using the firmware(9) functionality correctly enough; it's similar to how it's used in the Wi-Fi drivers I think. Add other reviewers if you think it might be helpful.

erj updated this revision to Diff 71668.May 11 2020, 11:03 PM
  • ice-ddp: Use "SRCTOP" instead of "S" in module Makefile

Can we enable this in an unsupported state for other 64-bit archs (arm64. ppc64)? Just would like to see the files entry correct for such usage.

erj added a comment.May 12 2020, 6:49 PM

Can we enable this in an unsupported state for other 64-bit archs (arm64. ppc64)? Just would like to see the files entry correct for such usage.

Yeah, that's doable.

erj added a comment.May 13 2020, 10:34 PM

Can we enable this in an unsupported state for other 64-bit archs (arm64. ppc64)? Just would like to see the files entry correct for such usage.

Making it compile on arm64 was straightforward, but powerpc64 is telling me that "atomic_testandset_32" and "atomic_testandclear_32" don't exist. Are there equivalent atomic instructions available?

Having one other arch, arm64, compile seems like a reasonable litmus for the driver. I can look into the PPC stuff when AIC HW begins to appear on the market and it shouldn't block this.

erj updated this revision to Diff 71803.May 14 2020, 10:23 PM
  • ice: Add support for building on arm64 targets
erj updated this revision to Diff 71804.May 14 2020, 10:31 PM
  • ice: Add entries to arm64 NOTES file
emaste added inline comments.May 14 2020, 11:27 PM
sys/contrib/dev/ice/README
12

not sure what these special characters are (that Phab rendered as special colour !)

erj updated this revision to Diff 72090.May 21 2020, 8:32 PM
  • ice: Update MAINTAINERS file
  • ice: Remove strange characters in ice_ddp README
erj added inline comments.May 21 2020, 8:32 PM
sys/contrib/dev/ice/README
12

I don't know what those are, but I removed them.

We'll need to patch this to bring it in line with OOT, but is g2g for now.

This revision is now accepted and ready to land.May 21 2020, 9:39 PM
erj added a comment.May 22 2020, 8:25 PM

Any other comments before I commit this?

erj closed this revision.May 28 2020, 2:52 PM

This was committed in rS361541.