Page MenuHomeFreeBSD

Reduce contention on per-adapter lock.
ClosedPublic

Authored by jhb on Jun 25 2020, 9:24 PM.
Tags
None
Referenced Files
F150977812: D25457.id73669.diff
Sun, Apr 5, 6:48 AM
Unknown Object (File)
Thu, Apr 2, 3:44 AM
Unknown Object (File)
Tue, Mar 31, 10:18 PM
Unknown Object (File)
Sun, Mar 29, 12:24 PM
Unknown Object (File)
Thu, Mar 26, 4:11 PM
Unknown Object (File)
Tue, Mar 24, 3:45 AM
Unknown Object (File)
Tue, Mar 17, 4:57 AM
Unknown Object (File)
Sun, Mar 15, 10:28 AM
Subscribers

Details

Summary
  • Move temporary sglists into the session structure and protect them with a per-session lock instead of a per-adapter lock.
  • Retire an unused session field, and move a debugging field under INVARIANTS to avoid using the session lock for completion handling when INVARIANTS isn't enabled.
  • Use counter_u64 for per-adapter statistics.

Note that this helps for cases where multiple sessions are used
(e.g. multiple IPsec SAs or multiple KTLS connections). It does not
help for workloads that use a single session (e.g. a single GELI
volume).

Test Plan
  • tested with KTLS RX where it made a marginal perf improvement (29 Gbps to 30-31 Gbps). However, in pmcstat callchains, lock_delay due to lock contention went from 6% of overall time (of which 82% was for the adapter lock in ccr_process) to 3% of overall time (of which 3% was for the per-session lock in ccr_process)

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Not Applicable
Unit
Tests Not Applicable