Page MenuHomeFreeBSD

MFC r272514 Switch ipfw to use rmlock for runtime locking.

Authored by kbowling on Feb 2 2016, 3:32 AM.



Looking at one of our CDN nodes with pmcstat, ipfw incurs a significant penalty on uncontested rwlocks for each packet. The problem seems to be that each rw_rlock is actually a serializing mutex to prevent reader priority inversion.

This was fixed in head r272514

Test Plan

I can profile with and without the patch to provide some baselines if needed

Diff Detail

rS FreeBSD src repository
Lint Skipped
Unit Tests Skipped

Event Timeline

kbowling updated this revision to Diff 12938.Feb 2 2016, 3:32 AM
kbowling retitled this revision from to ipfw rmlocks.
kbowling updated this object.
kbowling edited the test plan for this revision. (Show Details)
kbowling added a reviewer: melifaro.
kbowling set the repository for this revision to rS FreeBSD src repository.
kbowling added subscribers: sbruno, network.
melifaro edited edge metadata.EditedFeb 2 2016, 5:10 AM

Please take a look on -HEAD ipfw version.
Along with some architectural changes, there are several performance oriented ones: rmlock for fast path, per-cpu rule counters, more compact rule structure and faster tables.
It could be merged to 10 (in fact, we even run in on 9/ in several places)

kbowling retitled this revision from ipfw rmlocks to MFC r272514 Switch ipfw to use rmlock for runtime locking..Feb 2 2016, 7:46 AM
kbowling updated this object.
kbowling edited edge metadata.
ae added a subscriber: ae.Feb 2 2016, 8:49 AM

AFAIR, you can not use the same struct rm_priotracker for different threads.

kbowling updated this revision to Diff 12946.Feb 2 2016, 9:14 AM

Thanks, sorry my patch was pretty brain dead. This updated patch is an MFC of r272514 against 10-STABLE.

Does this look better? There's one change vs HEAD.

ae added a comment.Feb 8 2016, 9:19 AM

Yes. We use this patch for two years. Also we have a lot of machines with the previous version of it, where rwmtx was just removed and pfil(4)'s rmlock is used instead.

gnn accepted this revision.Feb 22 2016, 11:15 PM
gnn added a reviewer: gnn.
This revision is now accepted and ready to land.Feb 22 2016, 11:15 PM
kbowling abandoned this revision.Mar 23 2016, 5:26 PM

Apparently this is too hard to commit because SVN