Page MenuHomeFreeBSD

rtld: Annotate .rtld_start on powerpc and powerpc64
ClosedPublic

Authored by pkubaj on Thu, Oct 15, 9:36 AM.
Tags
None
Referenced Files
F176096271: D60740.id189958.diff
Fri, Oct 16, 2:02 AM
F176095328: D60740.id189967.diff
Fri, Oct 16, 1:51 AM
F176083458: D60740.diff
Thu, Oct 15, 11:17 PM
F176077708: D60740.id189958.diff
Thu, Oct 15, 10:01 PM
F176061235: D60740.id189958.diff
Thu, Oct 15, 6:05 PM
F176055186: D60740.id189967.diff
Thu, Oct 15, 4:42 PM
F176045761: D60740.id.diff
Thu, Oct 15, 2:53 PM
F176045759: D60740.id189958.diff
Thu, Oct 15, 2:53 PM
Subscribers

Details

Summary

Mark the link register as undefined at the entry point, as was done for amd64 (D40794), aarch64 (D40844), i386 (D40949) and riscv (D46406).

.rtld_start calls the program's _start with blrl, so unwinding the main thread of a dynamically linked program reaches it, but its FDE (from the .cfi_startproc in _ENTRY()) has no rule for the return address. LLVM libunwind then reuses the stale LR value of the innermost frame as the return address (https://reviews.llvm.org/D83573, which the base copy reverts in rG5866c369e4fd for PR 256864), unwinds into the argument and environment strings above the initial stack frame and crashes. On powerpc64 this breaks every main thread backtrace in programs that link their own libunwind. ClickHouse, for example, crashes whenever clickhouse-client or clickhouse-local throws an exception on its main thread, because the exception collects a stack trace.

Static binaries and new threads are not affected, since exec_setregs() and cpu_set_upcall() start them with LR cleared.

Test Plan

Built ld-elf.so.1 into a temporary objdir, unmodified and with this change, and ran a program that walks its own stack with LLVM libunwind (unw_getcontext() + unw_step(), the copy bundled with ClickHouse) through each of them directly (./ld-elf.so.1 ./prog), with a small and with a 2.5 kB environment:

rtldpowerpc64, main @ rG7cf02164a517powerpc64le, releng/15.1
installedSIGSEGV after _rtld_startSIGSEGV after _rtld_start
rebuilt, unmodifiedSIGSEGV after _rtld_startSIGSEGV after _rtld_start
with this changestops at _rtld_startstops at _rtld_start

Then ran a full make buildworld on powerpc64 (main @ rG7cf02164a517) with this change and repeated the test with the resulting ld-elf.so.1 and ld-elf32.so.1, the latter with the program built with -m32:

rtld64-bit32-bit
installedSIGSEGV after _rtld_startstops after .rtld_start, but only because the return address computed for it is a stack address (0xffffdb30) with no FDE
from this buildworldstops at _rtld_start, return address 0stops at .rtld_start, return address 0

The same walk in a new thread and in a static binary already stops cleanly. backtrace(3) from base returns the same frames with either rtld.

objdump --dwarf=frames shows DW_CFA_undefined: reg65 (the return address column) in the FDE of _rtld_start in ld-elf.so.1 and of .rtld_start in ld-elf32.so.1.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable