The PowerPC context switch waited for the incoming thread to leave
blocked_lock before releasing the outgoing thread. If two CPUs selected
each other's running threads concurrently, both CPUs waited for the
other to perform the release and neither could progress. This appeared
as intermittent boot hangs under QEMU MTTCG when taskqueue threads were
bound to opposite CPUs.
8df2e5421468 ("powerpc: put the isync inside the TD_LOCK()...")
attempted to fix the same hang by moving isync into the polling loop.
That did not address the circular wait: isync cannot cause either CPU to
execute the release store after the loop. It only added context
synchronization to failed iterations and changed timing. Move isync back
to the successful exit, which preserves acquire ordering for subsequent
context loads.
Match the ordering used by other architectures: publish the outgoing
thread's new lock before waiting for the incoming thread. Apply the
correction to both 32-bit and 64-bit switch paths.
The 2015 change moved the release after the stack switch because the old
implementation continued through pmap_activate() and optional state
restoration on the outgoing thread's stack after making that thread
runnable. The switch path has since changed: after the release it only
polls wired td_lock state using registers, then immediately loads the
incoming PCB_SP before updating per-CPU state or making any calls.
cpu_switch() also runs with external and decrementer interrupts disabled
by spinlock_enter(). Thus this does not restore the post-release call
window that the 2015 change corrected.
Fixes: 53607fe3cc8d ("Fix an extremely subtle concurrency bug...")
Fixes: 7a49d964d3f9 ("Merge r278429 from ppc64:")
Fixes: 8df2e5421468 ("powerpc: put the isync inside the TD_LOCK()...")
MFC after: 2 weeks
MFC to: stable/14, stable/15
Sponsored by: FreeBSD Foundation