Page MenuHomeFreeBSD

callout(9): Fix a race between migration and callout_drain()
ClosedPublic

Authored by markj on Nov 18 2020, 2:10 PM.
Tags
None
Referenced Files
F134024375: D27266.id79699.diff
Thu, Oct 30, 3:02 AM
F133924463: D27266.diff
Wed, Oct 29, 9:49 AM
Unknown Object (File)
Sat, Oct 25, 6:43 AM
Unknown Object (File)
Fri, Oct 17, 8:50 AM
Unknown Object (File)
Wed, Oct 15, 12:49 AM
Unknown Object (File)
Wed, Oct 15, 12:48 AM
Unknown Object (File)
Wed, Oct 15, 12:48 AM
Unknown Object (File)
Wed, Oct 15, 12:48 AM
Subscribers

Details

Summary

Suppose a running callout re-arms itself, and before the callout
finishes running another CPU calls callout_drain() and goes to sleep.
softclock_call_cc() will wake up the draining thread, which may not run
immediately if there is a lot of CPU load. Furthermore, the callout is
still in the callout wheel so it can continue to run and re-arm itself.
Then, suppose that the callout migrates to another CPU before the
draining thread gets a chance to run. The draining thread is in this
loop in _callout_stop_safe():

while (cc_exec_curr(cc) == c) {
	CC_UNLOCK(cc);
	sleep();
	CC_LOCK(cc);
}

but after the migration, cc points to the wrong CPU's callout state.
Then the draining thread goes off and removes the callout from the
wheel, but does so using the wrong lock and per-CPU callout state.

Fix the problem by doing a re-lookup of the callout CPU after sleeping.
This is a minimal patch intended to be suitable for backporting.

Test Plan

syzkaller found this bug using kevent() and EVFILT_TIMER.

Diff Detail

Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 34877
Build 31899: arc lint + arc unit

Event Timeline

markj requested review of this revision.Nov 18 2020, 2:10 PM
markj created this revision.
markj added reviewers: hselasky, rrs.
hselasky added inline comments.
sys/kern/kern_timeout.c
1164

There is no need for a while () statement here. Change it into an if (). The block is always exited via a goto.

This revision is now accepted and ready to land.Nov 18 2020, 2:23 PM
markj marked an inline comment as done.

while -> if

This revision now requires review to proceed.Nov 18 2020, 2:30 PM
This revision is now accepted and ready to land.Nov 18 2020, 5:58 PM