Page MenuHomeFreeBSD

rtld-elf: Fix loading libraries with ORIGIN flag (like LLVM) via LD_LIBRARY_PATH_FDS
Needs ReviewPublic

Authored by greg_unrelenting.technology on Jan 5 2020, 9:16 PM.

Details

Summary

Dynamically loading LLVM from LD_LIBRARY_PATH_FDS is important for Capsicum-sandboxing 3D applications. Unfortunately it fails.

Test case:

#include <stdio.h>
#include <dlfcn.h>
#include <sys/capsicum.h>

int main() {
	printf("fd %d\n", openat(AT_FDCWD, "/usr/local/llvm90/lib", O_DIRECTORY));
	cap_enter();
	dlopen("libLLVM-9.so", RTLD_LAZY);
}
cc -o capllvm capllvm.c && LD_LIBRARY_PATH_FDS=3 ./capllvm

Without the patch results in:

fd 3                                                                                                                                   
ld-elf.so.1: Fatal error
openat(3,"libLLVM-9.so",O_RDONLY|O_CLOEXEC|O_VERIFY,00) = 4 (0x4)
[…]
__getcwd(0x800228008,1024)                       ERR#94 'Not permitted in capability mode'
fstatat(AT_FDCWD,"/",0x7fffffffab38,0x0)         ERR#94 'Not permitted in capability mode'
* frame #0: 0x000000080021be14 ld-elf.so.1`getcwd(pt="/usr/local/llvm60/lib/libLLVM-9.so", size=1024) at getcwd.c:79
  frame #1: 0x000000080021949e ld-elf.so.1`realpath [inlined] realpath1(path="#3/libLLVM-9.so", resolved="/usr/local/llvm60/lib/libLLVM-9.so") at realpath.c:71
  frame #2: 0x000000080021946c ld-elf.so.1`realpath(path="#3/libLLVM-9.so", resolved="/usr/local/llvm60/lib/libLLVM-9.so") at realpath.c:226
  frame #3: 0x00000008002104ab ld-elf.so.1`digest_dynamic2 [inlined] rtld_dirname_abs(path="#3/libLLVM-9.so", base="/usr/local/llvm60/lib/libLLVM-9.so") at rtld.c:3992
  frame #4: 0x00000008002104a3 ld-elf.so.1`digest_dynamic2 [inlined] obj_resolve_origin(obj=0x0000000800224c08) at rtld.c:1407
  frame #5: 0x0000000800210483 ld-elf.so.1`digest_dynamic2(obj=0x0000000800224c08, dyn_rpath=0x0000000000000000, dyn_soname=0x000000080515d0d8, dyn_runpath=0x000000080515d018) at rtld.c:1415
  frame #6: 0x0000000800211bb0 ld-elf.so.1`load_object [inlined] digest_dynamic(obj=<unavailable>, early=0) at rtld.c:1437
  frame #7: 0x0000000800211b8f ld-elf.so.1`load_object [inlined] do_load_object(fd=<unavailable>, name="libLLVM-9.so", path=<unavailable>, sbp=<unavailable>, flags=<unavailable>) at rtld.c:2565
  frame #8: 0x0000000800211ae6 ld-elf.so.1`load_object(name="libLLVM-9.so", fd_u=<unavailable>, refobj=<unavailable>, flags=<unavailable>) at rtld.c:2524
  frame #9: 0x00000008002109de ld-elf.so.1`dlopen_object(name="libLLVM-9.so", fd=<unavailable>, refobj=0x0000000800224008, lo_flags=2, mode=1, lockstate=<unavailable>) at rtld.c:3394
  frame #10: 0x000000080020d77f ld-elf.so.1`rtld_dlopen(name="libLLVM-9.so", fd=-1, mode=<unavailable>) at rtld.c:3356

(where did the llvm60 path come from btw?? somewhere in the library?)

With the patch, it works. (Actual Wayland-EGL/GLES applications were tested as well.)

If a different way of fixing this is better, feel free to suggest (or just do) it.

Diff Detail

Repository
rS FreeBSD src repository
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

kib added a comment.Jan 6 2020, 1:20 AM

How do you define 'work' ? Show the ktrace of the test with patched rtld, I doubt that the library is loaded. Rtld never parses "#NNNN" in a path as a reference to a directory descriptor.

In D23043#505145, @kib wrote:

How do you define 'work' ? Show the ktrace of the test with patched rtld, I doubt that the library is loaded.

Full test with calling a function, with ktrace: P350

Rtld never parses "#NNNN" in a path as a reference to a directory descriptor.

Well it tries to obj_resolve_origin on that path (which is the only path it has) for libLLVM, seems like because of this:

% readelf -a /usr/local/llvm90/lib/libLLVM-9.so | rg -i orig
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../lib:/usr/local/lib]
 0x000000000000001e (FLAGS)              ORIGIN
 0x000000006ffffffb (FLAGS_1)            Flags: ORIGIN

I guess ideally the runpath with fd-relative $ORIGIN references would be fully supported, but in this case it refers to the directory that's already on LD_LIBRARY_PATH_FDS so everything's fine, we just have to not crash.

kib added a comment.Jan 6 2020, 5:17 PM

So lets untangle this.

First, there is a bug in rtld as is. Instead of killing the process, if $ORIGIN for dlopened object cannot be resolved, we should return an error. I fixed this in D23053.

Second, what you do, i.e. converting $ORIGIN into fake path, is beyond even a hack. It breaks the feature effectively ignoring the $ORIGIN at all.

It is not clear me how to make $ORIGIN work in the cap environment at all, and the use of dotdot in the rpath makes it even less feasible to try to find a solution.

effectively ignoring the $ORIGIN at all

That's exactly what I want, an option to ignore it instead of getting an error. For sandboxing, we often already have all these directories in LD_LIBRARY_PATH_FDS. It could be enabled with a new environment variable like LD_IGNORE_ORIGIN or something

emaste added a subscriber: emaste.Jan 10 2020, 4:31 PM