Page MenuHomeFreeBSD

powerpc64: give each CPU its own KDB trap stack
Needs ReviewPublic

Authored by pkubaj on Mon, Oct 5, 1:28 PM.

Details

Reviewers
None
Group Reviewers
PowerPC
Summary

Kernel mode program, trace, breakpoint and machine check exceptions go
through dbtrap, which builds the trap frame on trapstk before calling
db_trap_glue() and, when KDB does not want the trap, restores every
register from that frame and re-enters the normal trap path. There
was a single trapstk for the whole machine. Two CPUs taking such a
trap at the same time built their frames on top of each other and one
of them continued with the other's registers and stack pointer.

DTrace fbt probes are exactly these traps, so enabling a probe on a
function that runs concurrently on several CPUs corrupted the kernel
within seconds: an fbt probe on pmap_nofault() during a parallel
compile on a POWER9 pseries guest panicked with "stack overflow
detected" from pmap_nofault() itself.

Size trapstk for MAXCPU and pick the stack by cpuid. The array moves
to .bss so that it does not take space in the kernel image.

MFC after: 1 week

Test Plan

POWER9 pseries guest with 96 vCPUs. With an fbt probe on pmap_nofault()
enabled during a parallel compile the stock kernel panicked within
seconds with "stack overflow detected" raised from pmap_nofault(): the
KDB trap frames of two CPUs overlapped on the single trapstk. With this
change the same probe runs for the length of the compile, and the fbt
exercises of the later changes in this series (tens of thousands of
hits per second on 96 CPUs, repeated enable/disable rounds) all ran on
kernels carrying it.

Caveat: the panic was observed before the fbt entry-path exclusion of a
separate change existed, so that run does not isolate the two defects;
the defect itself is plain in trap_subr64.S, where trapstk has no
per-CPU indexing.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 77716
Build 74599: arc lint + arc unit