rtld-elf: Fix loading libraries with ORIGIN flag (like LLVM) via LD_LIBRARY_PATH_FDS

[val_packett.cool]

Summary

Dynamically loading LLVM from LD_LIBRARY_PATH_FDS is important for Capsicum-sandboxing 3D applications. Unfortunately it fails.

Test case:

#include <stdio.h>
#include <dlfcn.h>
#include <sys/capsicum.h>

int main() {
	printf("fd %d\n", openat(AT_FDCWD, "/usr/local/llvm90/lib", O_DIRECTORY));
	cap_enter();
	dlopen("libLLVM-9.so", RTLD_LAZY);
}

cc -o capllvm capllvm.c && LD_LIBRARY_PATH_FDS=3 ./capllvm

Without the patch results in:

fd 3                                                                                                                                   
ld-elf.so.1: Fatal error

openat(3,"libLLVM-9.so",O_RDONLY|O_CLOEXEC|O_VERIFY,00) = 4 (0x4)
[…]
__getcwd(0x800228008,1024)                       ERR#94 'Not permitted in capability mode'
fstatat(AT_FDCWD,"/",0x7fffffffab38,0x0)         ERR#94 'Not permitted in capability mode'

* frame #0: 0x000000080021be14 ld-elf.so.1`getcwd(pt="/usr/local/llvm60/lib/libLLVM-9.so", size=1024) at getcwd.c:79
  frame #1: 0x000000080021949e ld-elf.so.1`realpath [inlined] realpath1(path="#3/libLLVM-9.so", resolved="/usr/local/llvm60/lib/libLLVM-9.so") at realpath.c:71
  frame #2: 0x000000080021946c ld-elf.so.1`realpath(path="#3/libLLVM-9.so", resolved="/usr/local/llvm60/lib/libLLVM-9.so") at realpath.c:226
  frame #3: 0x00000008002104ab ld-elf.so.1`digest_dynamic2 [inlined] rtld_dirname_abs(path="#3/libLLVM-9.so", base="/usr/local/llvm60/lib/libLLVM-9.so") at rtld.c:3992
  frame #4: 0x00000008002104a3 ld-elf.so.1`digest_dynamic2 [inlined] obj_resolve_origin(obj=0x0000000800224c08) at rtld.c:1407
  frame #5: 0x0000000800210483 ld-elf.so.1`digest_dynamic2(obj=0x0000000800224c08, dyn_rpath=0x0000000000000000, dyn_soname=0x000000080515d0d8, dyn_runpath=0x000000080515d018) at rtld.c:1415
  frame #6: 0x0000000800211bb0 ld-elf.so.1`load_object [inlined] digest_dynamic(obj=<unavailable>, early=0) at rtld.c:1437
  frame #7: 0x0000000800211b8f ld-elf.so.1`load_object [inlined] do_load_object(fd=<unavailable>, name="libLLVM-9.so", path=<unavailable>, sbp=<unavailable>, flags=<unavailable>) at rtld.c:2565
  frame #8: 0x0000000800211ae6 ld-elf.so.1`load_object(name="libLLVM-9.so", fd_u=<unavailable>, refobj=<unavailable>, flags=<unavailable>) at rtld.c:2524
  frame #9: 0x00000008002109de ld-elf.so.1`dlopen_object(name="libLLVM-9.so", fd=<unavailable>, refobj=0x0000000800224008, lo_flags=2, mode=1, lockstate=<unavailable>) at rtld.c:3394
  frame #10: 0x000000080020d77f ld-elf.so.1`rtld_dlopen(name="libLLVM-9.so", fd=-1, mode=<unavailable>) at rtld.c:3356

(where did the llvm60 path come from btw?? somewhere in the library?)

With the patch, it works. (Actual Wayland-EGL/GLES applications were tested as well.)

If a different way of fixing this is better, feel free to suggest (or just do) it.

[kib]

How do you define 'work' ? Show the ktrace of the test with patched rtld, I doubt that the library is loaded. Rtld never parses "#NNNN" in a path as a reference to a directory descriptor.

[val_packett.cool]

In D23043#505145, @kib wrote:

How do you define 'work' ? Show the ktrace of the test with patched rtld, I doubt that the library is loaded.

Full test with calling a function, with ktrace: P350

Rtld never parses "#NNNN" in a path as a reference to a directory descriptor.

Well it tries to obj_resolve_origin on that path (which is the only path it has) for libLLVM, seems like because of this:

% readelf -a /usr/local/llvm90/lib/libLLVM-9.so | rg -i orig
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../lib:/usr/local/lib]
 0x000000000000001e (FLAGS)              ORIGIN
 0x000000006ffffffb (FLAGS_1)            Flags: ORIGIN

I guess ideally the runpath with fd-relative $ORIGIN references would be fully supported, but in this case it refers to the directory that's already on LD_LIBRARY_PATH_FDS so everything's fine, we just have to not crash.

[kib]

So lets untangle this.

First, there is a bug in rtld as is. Instead of killing the process, if $ORIGIN for dlopened object cannot be resolved, we should return an error. I fixed this in D23053.

Second, what you do, i.e. converting $ORIGIN into fake path, is beyond even a hack. It breaks the feature effectively ignoring the $ORIGIN at all.

It is not clear me how to make $ORIGIN work in the cap environment at all, and the use of dotdot in the rpath makes it even less feasible to try to find a solution.

[val_packett.cool]

effectively ignoring the $ORIGIN at all

That's exactly what I want, an option to ignore it instead of getting an error. For sandboxing, we often already have all these directories in LD_LIBRARY_PATH_FDS. It could be enabled with a new environment variable like LD_IGNORE_ORIGIN or something