Page MenuHomeFreeBSD

SMR protection for inpcbs
ClosedPublic

Authored by glebius on Nov 16 2021, 9:24 PM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Mar 28, 5:24 PM
Unknown Object (File)
Mar 10 2024, 2:25 AM
Unknown Object (File)
Jan 30 2024, 8:29 AM
Unknown Object (File)
Jan 14 2024, 1:58 PM
Unknown Object (File)
Dec 23 2023, 3:31 AM
Unknown Object (File)
Dec 10 2023, 7:54 PM
Unknown Object (File)
Dec 2 2023, 1:50 AM
Unknown Object (File)
Nov 15 2023, 9:00 AM

Details

Summary

With introduction of epoch(9) synchronization to network stack the
inpcb database became protected by the network epoch together with
static network data (interfaces, addresses, etc). However, inpcb
aren't static in nature, they are created and destroyed all the
time, which creates some traffic on the epoch(9) garbage collector.

With SMR freed memory is reclaimed in page-sized batches, with
virtually zero overhead. Details:

  • The database is already build on CK lists, thanks to epoch(9).
  • For write access nothing changes, yet.
  • For a lookup in the database SMR section is now required. Once the desired inpcb is found we need to transition from SMR section to r/w lock on the inpcb itself, with a check that inpcb isn't yet freed. This requires some compexity, since SMR section itself is a critical(9) section. The complexity is hidden from KPI users in inp_smr_lock().
  • For a inpcb list traversal (a pcblist sysctl, or broadcast notification) also a new KPI is provided, that hides internals of the database - inp_next(struct inp_iterator *).

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 42880
Build 39768: arc lint + arc unit

Event Timeline

Given we don't seem to have a `man smr` could you at least expand SMR once in your title/description so people can find what they are looking for as it's virtually invisible to the world still (unless one finds the header file)?

Note: due to my poor phab skills the previous version was abandoned. Tried to add it to the stack of reviews, but failed. Previous version: https://reviews.freebsd.org/D32585

In D33022#745736, @bz wrote:

Given we don't seem to have a `man smr` could you at least expand SMR once in your title/description so people can find what they are looking for as it's virtually invisible to the world still (unless one finds the header file)?

Yes, absence of the manual page bugs me. Not even mention in uma(9). I will reword description.

inpcb garbage collector traffic in epoch context as seen under uperf -m connect.xml test.

in_pcbfree_deferred.png (548×1 px, 118 KB)

In D33022#745736, @bz wrote:

Given we don't seem to have a `man smr` could you at least expand SMR once in your title/description so people can find what they are looking for as it's virtually invisible to the world still (unless one finds the header file)?

Yes, absence of the manual page bugs me. Not even mention in uma(9). I will reword description.

I agree with all this we are sadly lacking a SMR manual page we really need one if this is going to start
being propagated and used in the kernel!!

This revision is now accepted and ready to land.Nov 17 2021, 7:40 PM
This revision now requires review to proceed.Nov 19 2021, 4:39 AM
This revision is now accepted and ready to land.Dec 2 2021, 7:45 PM