Page MenuHomeFreeBSD

inpcb: take a pcbinfo database out from single lock
Needs ReviewPublic

Authored by glebius on Thu, Jul 9, 8:20 PM.

Details

Reviewers
markj
pouria
Group Reviewers
network
transport
Summary

Initialize all three hashes (exact, wild, load balance group) with a per-
bucket lock. Nothing changes for the packet lookup KPI - it still uses
SMR section for thread safety. But connect(2) and bind(2) operations gain
parallelism now.

The main concept is that as we lookup inpcb database for editing, we are
accumulating bucket locks necessary to accomplish the operation. Once all
lookups are complete and we are good to go, the inpcb is inserted (or
moved) and accumulated lock context is released.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 74661
Build 71544: arc lint + arc unit

Event Timeline

I got naive microbenchmark. It allocates ncpu threads, and each threads creates n UDP sockets that are connect(2)-ed to a random valid address:port. In a test virtual machine with 8 CPUs and 10000 sockets per thread I got 6x-7x speedup. If the test is adjusted to close(2) sockets when 50% allocation is reached, so that both insertions and removals are exercised at the same time, the speedup is roughly the same. I didn't yet do any real tests on hardware, but benefits are clear.

The patch doesn't yet address O(n) cycle when we are allocating ports to connect to a single remote IP address.