Page MenuHomeFreeBSD

simple preempt safe epoch API
ClosedPublic

Authored by mmacy on May 9 2018, 6:48 AM.
Tags
None
Referenced Files
Unknown Object (File)
Mar 14 2024, 7:47 AM
Unknown Object (File)
Jan 6 2024, 4:12 PM
Unknown Object (File)
Dec 21 2023, 1:23 AM
Unknown Object (File)
Dec 20 2023, 3:06 AM
Unknown Object (File)
Dec 4 2023, 5:47 PM
Unknown Object (File)
Nov 26 2023, 2:45 AM
Unknown Object (File)
Nov 25 2023, 5:34 PM
Unknown Object (File)
Nov 25 2023, 4:28 PM

Details

Summary

read locking is over used in the kernel to guarantee liveness, this API makes it easy to use EBR to provide liveness guarantees without a global atomic

See D15366 for the initial use case.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

hselasky added inline comments.
sys/conf/kern.pre.mk
80

Maybe OFED_CFLAGS does then no longer need to include -I$S/contrib/ck/include .

Generally I like it.
I'd like it a lot more if there were man.9 page(s) for the epoch API.
Some nits are all I see, but my CK foo is weak.

sys/conf/kmod.mk
126 ↗(On Diff #42300)

I don't think we need this BOTH places... But if we do, then I'll look into why... this duplication is getting out of hand.

sys/kern/subr_epoch.c
305

nit: blank line between functions...

sys/sys/epoch.h
4

Nit: Please consider removing this. It's not needed.

58

Nit: Most other files in the tree aren't double-spaced like this. Consider making it single spaced.

sys/sys/proc.h
246

is 255 nesting levels enough? It's hard to tell from the code.

sys/kern/subr_gtaskqueue.c
992 ↗(On Diff #42300)

Did an unrelated change leak in?

sys/sys/gtaskqueue.h
66 ↗(On Diff #42300)

Same question as above: This seems unrelated to the epoch change.

sys/sys/proc.h
246

sigqueue_t ends in an int, and td_flags is an int, so it looks like we have 4 bytes between them, one of which is used for td_lend_user_pri. So it seems like there is space for a u_short or uint16_t for this without changing the size of struct thread. Hopefully 64K levels would be enough for anyone.

sys/kern/subr_gtaskqueue.c
992 ↗(On Diff #42300)

It's a dependency. I'll just go and commit it separately today.

sys/sys/epoch.h
4

Will do.

sys/sys/proc.h
246

This is morally equivalent to an rlock having a single thread recurse 250 times on an rlock or simultaneously acquire 250 rlocks seems like behaviour that could only be a bug. I'll widen the field if you disagree.

sys/kern/subr_epoch.c
109

nits: Should this printf() only be for verbose boot? It seems a bit generic... maybe add in func or something? Also, two semi-colons at end of line.

151

Shouldn't this be eps[i]?

In D15365#323858, @imp wrote:

Generally I like it.
I'd like it a lot more if there were man.9 page(s) for the epoch API.
Some nits are all I see, but my CK foo is weak.

That's reasonable, but I'd like to see everything approved first otherwise it might be wasted effort. Also while you're on the topic, prod cognet to get the ck man pages installed on build. I asked him and he just said "I didn't for some reason".

Incorporate all feedback so far (except adding man pages)

sys/kern/subr_epoch.c
151

We're incrementing eps right above, the actual ck epoch record is a field in that.

This revision is now accepted and ready to land.May 9 2018, 7:39 PM
  • Forbid recursing in a second epoch
  • Replace the ad hoc block handling with turnstile
  • Forbid sleeping in an epoch section
This revision now requires review to proceed.May 9 2018, 9:54 PM
This revision is now accepted and ready to land.May 9 2018, 10:30 PM
  • add stress test
  • correctly handle case where thread on wait list is not blocked on a lock yet
  • correctly handle case where a thread blocked on a lock is unblocked
  • correctly handle case where the memory for a domain is not populated as can be the case if hw.physmem is set much less than the RAM in the system
  • add comments explaining the internals of epoch_block_handler
This revision now requires review to proceed.May 10 2018, 6:33 AM
  • Keep stats of paths taken in block handler
  • Add adaptive spinning
  • remove gratuitous assert
  • replace sleep with context switch when there are threads blocked on a turnstile
  • Add new testcase variation
  • Specify number of iterations as argument to runtest
sys/kern/subr_epoch.c
423

This might be misleading to someone not familiar with LKPI, clarify this as from LinuxKPI?

This revision is now accepted and ready to land.May 10 2018, 1:34 PM
sys/kern/kern_synch.c
150

Probably should do this in sched_bind() too since you rely on sched_pin().

sys/kern/subr_epoch.c
101

We could probably use a little more of this information included with smp.h or elsewhere. This type of topology information becomes handy as we get smarter about resource placement.