callout: Wait for the softclock thread to switch before rescheduling
When a softclock thread prepares to go off-CPU, the following happens in
the context of the thread:
- callout state is locked
- thread state is set to IWAIT
- thread lock is switched from the tdq lock to the callout lock
- tdq lock is released
- sched_switch() sets td_lock to &blocked_lock
- sched_switch() releases old td_lock (callout lock)
- sched_switch() removes td from its runqueue
- cpu_switch() sets td_lock back to the callout lock
Suppose a timer interrupt fires while the softclock thread is switching
off, and callout_process() schedules the softclock thread. Then there
is a window between steps 5 and 8 where callout_process() can call
sched_add() while td_lock is &blocked_lock, but this is not correct
since the thread is not logically locked.
callout_process() thus needs to spin waiting for the softclock thread to
finish switching off (i.e., after step 8 completes) before rescheduling
it, since callout_process() does not acquire the thread lock directly.
Reported by: syzbot+fb44dbf6734ff492c337@syzkaller.appspotmail.com
Fixes: 74cf7cae4d22 ("softclock: Use dedicated ithreads for running callouts.")
Reviewed by: mav, kib, jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33709