Modify the kernel linker to handle ifunc call site relocations.

[markj]

Summary

The linker code currently assumes that ifunc relocations are of type
R_[*]_IRELATIVE, but with -z ifunc-noplt this is no longer true.
In particular, we may also have to apply PC-relative relocations.
Such relocations require a symbol lookup. The default symbol lookup
function, elf_lookup(), calls a kern_linker.c function which performs
a kobj call, which can't work early during boot.

Address these problems:

• Add a custom symbol lookup function which simply invokes the ifunc resolver and otherwise returns ENOENT if the symbol is not of type GNU_IFUNC. Add a lookup callback to relocate_file1() so that we can use this custom lookup routine early during boot. I like this approach because it helps minimize the amount of kernel code that may be executed for ifunc resolution is performed.
• Apply all relocations in the kernel executable during boot, rather than trying to filter ifunc relocations. I believe we can safely do this on at least x86.
• Don't apply ifunc relocations in a separate pass when loading a .so kernel module. elf_reloc_internal() no longer has an easy way of determining whether a given relocation applies to an ifunc or not, and we cannot simply apply all relocations twice since they may not be idempotent.

Test Plan

I tested on i386 and amd64, with and without -zifunc-noplt.

[kib]

sys/kern/link_elf_obj.c

1555–1556

So this and previous condition body calls are identical now ? How is it supposed to work ?

Let me explain a fundamental issues with ifuncs: we must resolve all non-ifunc relocations when ifuncs resolvers are called. Otherwise, resolvers cannot access global symbols like cpu_feature and cannot call other functions. This is why two-stage resolution is coded there.

[markj]

sys/kern/link_elf_obj.c

1555–1556

Oops. But this means that the else case can simply be removed, so the logic becomes "apply relocations against GNU_IFUNC symbols iff 'ifuncs' is true." And link_elf_reloc_local() is never called with ifuncs == true when i386 and amd64 aren't defined.

[kib]

sys/kern/link_elf_obj.c

1555–1556

We need to handle both IRELATIVE and STB_LOCAL with target symbol having ifunc type there. Second (now identical) part handled IRELATIVE. This was done after normal relocs processed.

[markj]

sys/kern/link_elf_obj.c

1555–1556

Sorry, I still don't see the problem here once the else case is removed. We are applying relocations in two passes, first with ifunc == false and then with ifunc == true. Note that elf_reloc_local() now handles IRELATIVE relocations.

[kib]

sys/kern/link_elf_obj.c

1555–1556

Yes, elf_reloc_local() does handle IRELATIVE, but when ? From what I see, they are handled during the first pass, which is too early.

[markj]

Attempt to fix handling of ifunc relocations in kernel modules.

We must process all non-ifunc relocations first, where an ifunc relocation
is a relocation targetting a symbol of type STT_GNU_IFUNC, or of type
R_*_IRELATIVE. Add the MD predicate function elf_is_ifunc_reloc() for this
purpose.

[markj]

sys/kern/link_elf_obj.c

1555–1556

I see, I had forgotten that IRELATIVE relocations do not reference a symbol. I believe the latest upload handles this properly. If you agree with the approach I'll add stub elf_is_ifunc_reloc() predicates to the other platforms.

[kib]

sys/kern/link_elf.c

1213

Don't this expression excludes relocations with the referenced symbol type GNU_IFUNC always, even when ifunc == true ?

[markj]

sys/kern/link_elf.c

1213

I don't see how. We are comparing "ifuncs" with the expression "symbol_type() == GNU_IFUNC || elf_is_ifunc_reloc()". That is, relocations referencing GNU_IFUNC symbols are processed iff ifuncs == true.

[kib]

This looks fine then.

sys/kern/link_elf.c

1213

Ah, I see. I mis-readed the braces.

[markj]

• Add elf_is_ifunc_reloc() for non-x86 platforms.