Page MenuHomeFreeBSD

powerpc/radix: take the pmap lock in mmu_radix_sync_icache()
ClosedPublic

Authored by pkubaj on Tue, Sep 1, 6:06 PM.
Tags
None
Referenced Files
F170063945: D59311.diff
Thu, Sep 3, 10:05 AM
F169911556: D59311.id185585.diff
Wed, Sep 2, 10:52 PM
F169854119: D59311.id185574.diff
Wed, Sep 2, 6:47 PM
Unknown Object (File)
Wed, Sep 2, 10:22 AM
Unknown Object (File)
Wed, Sep 2, 4:05 AM
Unknown Object (File)
Tue, Sep 1, 9:04 PM
Unknown Object (File)
Tue, Sep 1, 8:51 PM
Unknown Object (File)
Tue, Sep 1, 8:41 PM
Subscribers

Details

Summary

mmu_radix_sync_icache() walks the page tables with an unlocked
pmap_extract() and passes the result straight to PHYS_TO_DMAP(), checking
only that it is non-zero. Nothing keeps the mapping - or the page table
page holding it - alive across that window: if another thread of the same
process tears a mapping down concurrently, the page table page can be
freed and reused, so pmap_extract() reads arbitrary memory and returns a
bogus physical address. __syncicache() then dereferences an unmapped
direct map address and the kernel takes a data storage interrupt:

fatal kernel trap:
   exception       = 0x300 (data storage interrupt)
   virtual address = 0xc003317ca6022a00
   dsisr           = 0x40000000
   srr0            = 0xc000000000f59460 (__syncicache)
   lr              = 0xc000000000f23588 (mmu_radix_sync_icache)
          pid = 23878, comm = skyframe-evaluator-
panic: data storage interrupt trap

The faulting addresses decode to physical addresses far beyond installed
memory (~140 TB and ~900 TB on a 256 GB machine), i.e. translations that
never existed.

The hash MMU implementation of the same method, moea64_sync_icache(),
already holds PMAP_LOCK() across the loop; do the same here.
mmu_radix_extract() does not acquire the pmap lock itself, so this
introduces no recursion.

JIT workloads reach this path constantly: ppc_instr_emulate() calls
pmap_sync_icache() on the faulting address for the SIGILL "second chance"
retry, so a multithreaded JVM executing freshly written code races against
its own threads' mmap/munmap. Every panic observed here was in a JVM
thread.

MFC after: 1 week

Test Plan

POWER9 (radix MMU), a poudriere loop building science/py-tensorflow, which
is a heavily threaded bazel/JVM workload. Before the change the machine
panicked twice within ten minutes, every time in a JVM thread. After it,
13 consecutive builds and more than 10 hours of uptime with no panic, on
both 15.1-RELEASE and 16.0-CURRENT; one of those builds ran for 4h11m and
completed successfully.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 76387
Build 73270: arc lint + arc unit