Page MenuHomeFreeBSD

Change smarter epoch support for NIC drivers to opt-in.
AcceptedPublic

Authored by glebius on Thu, Feb 13, 10:02 PM.

Details

Reviewers
hselasky
jeff
gallatin
shurd
bz
Group Reviewers
network
iflib
Summary

Although most of the drivers are epoch ready, due to peer pressure
switch over to opt-in instead of opt-out for epoch.

Instead of IFF_NEEDSEPOCH, provide IFF_KNOWSEPOCH. If driver marks
itself with IFF_KNOWSEPOCH, when ether_input() would not enter epoch
when processing its packets.

Now this will create recursive entrance in epoch in >90% network
drivers, but will guarantee safeness of the transition.

Mark tested drivers as IFF_KNOWSEPOCH.

Diff Detail

Lint
Lint OK
Unit
No Unit Test Coverage
Build Status
Buildable 29366
Build 27264: arc lint + arc unit

Event Timeline

glebius created this revision.Thu, Feb 13, 10:02 PM
hselasky accepted this revision.Thu, Feb 13, 10:22 PM

Looks good to me.

gallatin accepted this revision.Thu, Feb 13, 10:32 PM
This revision is now accepted and ready to land.Thu, Feb 13, 10:32 PM
jeff added a comment.Fri, Feb 14, 8:52 AM

What about the interrupt changes?

jeff accepted this revision.Fri, Feb 14, 8:53 AM
olivier added a subscriber: olivier.EditedFri, Feb 14, 2:00 PM

So, if I've correctly understood:

  • This will create recursive entrance in epoch with drivers that didn't support IFF_KNOWSEPOCH
  • iflib supports IFF_KNOWSEPOCH (because this patch add it to sys/net/iflib.c) but Chelsio didn't support it (because I don't see any modification to the Chelsio drivers here)

So theoretically, I shouldn't find performance difference on my forwarding lab using iflib drivers, but "perhaps" a small degradation with Chelsio because the recursive entrance:

  • My results show no performance impact with iflib neither with Chelsio.

Number of packets-per-second forwarded with a 10G Chelsio:

x FreeBSD r357884
+ FreeBSD r357884 and D23674
+--------------------------------------------------------------------------+
|x                                      +   +        x    + *+        x    |
|                   |__________________________A_____M____________________||
|                                          |_________A____M___|            |
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5     7301523.5       8641651       8316751     8202397.8     521321.49
+   5     8069901.5       8464468       8419314       8307376     187557.28
No difference proven at 95.0% confidence

Same but with Intel 1G:

x FreeBSD r357884
+ FreeBsD r357884 with D23674
+--------------------------------------------------------------------------+
|x                                                     +*  * x*   +        |
|                     |_________________________A__________M______________||
|                                                      |___A____|          |
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5        916923       1123971     1115717.5     1076802.5     89713.215
+   5       1100496       1138467       1114737     1116772.6     15947.773
No difference proven at 95.0% confidence
bz accepted this revision as: bz.Fri, Feb 14, 5:46 PM