Page MenuHomeFreeBSD

Make linuxulator handle ELFOSABI_NONE ELF executables
AbandonedPublic

Authored by trasz on Jun 21 2019, 7:09 PM.
Tags
None
Referenced Files
Unknown Object (File)
Feb 28 2024, 5:37 AM
Unknown Object (File)
Feb 20 2024, 6:56 AM
Unknown Object (File)
Jan 24 2024, 12:47 PM
Unknown Object (File)
Dec 23 2023, 3:15 AM
Unknown Object (File)
Dec 22 2023, 10:31 PM
Unknown Object (File)
Nov 15 2023, 1:27 PM
Unknown Object (File)
Nov 12 2023, 5:01 AM
Unknown Object (File)
Nov 10 2023, 10:54 AM

Details

Summary

Make linuxulator able to handle ELFOSABI_NONE ELF executables. Previously it was neccessary to either brand them as Linux using brandelf(1), or by adjusting kern.elf{32,64}.fallback_brand. Binaries that are correctly branded, either in the ELF header or with an ABI note, are not affected.

This can be enabled with 'sysctl compat.linux.exec_unbranded=0'. It fixes ldd(1) from Centos and Ubuntu.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 24974
Build 23695: arc lint + arc unit

Event Timeline

In which way does it fix ldd, and why ? [As usual, you do not bother to explain]

Without this commit, Linux ldd is broken - it errors out due to not being able to exec rtld. No idea why rtld is like this, but, well, it is - both in Centos and Ubuntu.

It probably also fixed Android binaries; I hadn’t tested it this time.

Without this commit, Linux ldd is broken - it errors out due to not being able to exec rtld. No idea why rtld is like this, but, well, it is - both in Centos and Ubuntu.

It probably also fixed Android binaries; I hadn’t tested it this time.

So the issue is that shared objects do not specify ABI ? Does it mean that with the change, direct execution mode for our rtld (and then our ldd) is either broken or only works by chance ?

Many many many Linux binaries are ELFOSABI_NONE.

The Elf64_Brandinfo definitions directly above the added lines allow executables with ELFOSABI_NONE to run when interp_path matches. You can see one was added for musl libc, before that one was added, Alpine Linux did not work.
But that's for normal executables.

why rtld is like this

I guess because rtld can't have *itself* set as its interpreter, as that would be an infinite loop.

This is our stuff, but I guess Linux is the same:

% file /bin/ls
/bin/ls: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.0 (1300030), FreeBSD-style, stripped
% file /libexec/ld-elf.so.1
/libexec/ld-elf.so.1: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, stripped

I guess an alternative patch would be checking for when we're launching from interp_path when matching these Elf64_Brandinfo things..

In D20720#448098, @kib wrote:

Without this commit, Linux ldd is broken - it errors out due to not being able to exec rtld. No idea why rtld is like this, but, well, it is - both in Centos and Ubuntu.

It probably also fixed Android binaries; I hadn’t tested it this time.

So the issue is that shared objects do not specify ABI ? Does it mean that with the change, direct execution mode for our rtld (and then our ldd) is either broken or only works by chance ?

No, our rtld and ldd work just fine, because they are marked ELFOSABI_FREEBSD, as reported by "elfdump -e". This change applies to binaries marked ELFOSABI_NONE.

In D20720#448101, @greg_unrelenting.technology wrote:

Many many many Linux binaries are ELFOSABI_NONE.

The Elf64_Brandinfo definitions directly above the added lines allow executables with ELFOSABI_NONE to run when interp_path matches. You can see one was added for musl libc, before that one was added, Alpine Linux did not work.
But that's for normal executables.

But the musl entry still says ELFOSABI_LINUX, doesn't it?

why rtld is like this

I guess because rtld can't have *itself* set as its interpreter, as that would be an infinite loop.

Okay, but why is the rtld binary marked ELFOSABI_NONE instead of ELFOSABI_LINUX,
like other binaries?

This is our stuff, but I guess Linux is the same:

% file /bin/ls
/bin/ls: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.0 (1300030), FreeBSD-style, stripped
% file /libexec/ld-elf.so.1
/libexec/ld-elf.so.1: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, stripped

Note that both say "FreeBSD" and not "SYSV".

I guess an alternative patch would be checking for when we're launching from interp_path when matching these Elf64_Brandinfo things..

But that wouldn't work for jails.

In D20720#448101, @greg_unrelenting.technology wrote:

Many many many Linux binaries are ELFOSABI_NONE.

The Elf64_Brandinfo definitions directly above the added lines allow executables with ELFOSABI_NONE to run when interp_path matches. You can see one was added for musl libc, before that one was added, Alpine Linux did not work.
But that's for normal executables.

But the musl entry still says ELFOSABI_LINUX, doesn't it?

Yes, that doesn't mean it won't match with NONE.

https://github.com/freebsd/freebsd/blob/fc870a6df90c3876ec348720e21e74beb8b70d92/sys/kern/imgact_elf.c#L378-L395 "Lacking a known brand, search for a recognized interpreter"

why rtld is like this

I guess because rtld can't have *itself* set as its interpreter, as that would be an infinite loop.

Okay, but why is the rtld binary marked ELFOSABI_NONE instead of ELFOSABI_LINUX,
like other binaries?

Others aren't either. In some distros I've tried (including Alpine), *everything* is NONE.

Linux doesn't care about ABI markers, because Linux thinks the whole world is Linux and nothing else deserves to exist :D

In D20720#448424, @greg_unrelenting.technology wrote:
In D20720#448101, @greg_unrelenting.technology wrote:

Many many many Linux binaries are ELFOSABI_NONE.

The Elf64_Brandinfo definitions directly above the added lines allow executables with ELFOSABI_NONE to run when interp_path matches. You can see one was added for musl libc, before that one was added, Alpine Linux did not work.
But that's for normal executables.

But the musl entry still says ELFOSABI_LINUX, doesn't it?

Yes, that doesn't mean it won't match with NONE.

https://github.com/freebsd/freebsd/blob/fc870a6df90c3876ec348720e21e74beb8b70d92/sys/kern/imgact_elf.c#L378-L395 "Lacking a known brand, search for a recognized interpreter"

Ah, true.

why rtld is like this

I guess because rtld can't have *itself* set as its interpreter, as that would be an infinite loop.

Okay, but why is the rtld binary marked ELFOSABI_NONE instead of ELFOSABI_LINUX,
like other binaries?

Others aren't either. In some distros I've tried (including Alpine), *everything* is NONE.

Okay, but in CentOS and Ubuntu most of things are marked correctly, the rtld seems
to be only binary there not marked as Linux. Oh well.

Linux doesn't care about ABI markers, because Linux thinks the whole world is Linux and nothing else deserves to exist :D

Still - this patch should fix it for static Alpine binaries and at least fix ldd for others.

Ping? Does the patch look ok?

Ping? Does the patch look ok?

No, it does not look ok. It lacks analysis of impact of the change on other ABIs.

Which other ABIs?

Any ABI supported by FreeBSD. Your patch potentially affects selection of the the ABI for any binary without the brand.

Well, yes, that’s precisely what it’s intended to do.

Well, yes, that’s precisely what it’s intended to do.

It would be even fine if we knew the side effects in advance. I can only repeat, such change should be accompanied by the analysis of effects on other ABIs.

In D20720#452276, @kib wrote:

Well, yes, that’s precisely what it’s intended to do.

It would be even fine if we knew the side effects in advance. I can only repeat, such change should be accompanied by the analysis of effects on other ABIs.

Okay, but what do you mean by that? Apart from what I've done, which is looking at various binaries in FreeBSD, both base and some packages, to verify they are all properly branded?

In D20720#452276, @kib wrote:

Well, yes, that’s precisely what it’s intended to do.

It would be even fine if we knew the side effects in advance. I can only repeat, such change should be accompanied by the analysis of effects on other ABIs.

Okay, but what do you mean by that? Apart from what I've done, which is looking at various binaries in FreeBSD, both base and some packages, to verify they are all properly branded?

What you did is just a way to start examine the change and see that it does not break things big outright.

What is needed is understanding how does the change affects selection of the ABI for binary, i.e. explanation which binaries suddenly start being classified for different brand.

That said, we already have sysctl kern.elf{32,64}.fallback_brand and brandelf(8).

In D20720#452395, @kib wrote:
In D20720#452276, @kib wrote:

Well, yes, that’s precisely what it’s intended to do.

It would be even fine if we knew the side effects in advance. I can only repeat, such change should be accompanied by the analysis of effects on other ABIs.

Okay, but what do you mean by that? Apart from what I've done, which is looking at various binaries in FreeBSD, both base and some packages, to verify they are all properly branded?

What you did is just a way to start examine the change and see that it does not break things big outright.

What is needed is understanding how does the change affects selection of the ABI for binary, i.e. explanation which binaries suddenly start being classified for different brand.

That said, we already have sysctl kern.elf{32,64}.fallback_brand and brandelf(8).

Okay. So, it will make Linuxulator execute ELF files marked as OSABI_NONE (also called 'SYSV'), unless they have an ELF note that indicates another ABI.

trasz edited the summary of this revision. (Show Details)
trasz edited the summary of this revision. (Show Details)

Add sysctl to disable the new behaviour.

trasz added a reviewer: emaste.

You added more people to the review, but was completely silent on both my points:

  • we already have sysctl kern.elf{32,64}.fallback_brand, which can be set by rc.d/linux or rc.d/abi. There is simply no need to disturb the brands selection process and add more knobs to it.
  • I pointed out that you in effect add a wildcard for matching, which consequences needs to be analyzed before applying, but you did not.

So I object against this fiddling, suggest adding some code to rc.d to set fallback brand.

It's hard to set the sysctl from a script (there's no way to figure out what value it should be set to), but generally: doing that would be working around a bug instead of fixing it. We should ship software that works correctly by default, without the need for twiddling some options. Currently we don't: we ship broken ELF activator that can't recognize common Linux executables.

And sure I have analyzed the consequences: there's a description above. The impact is minimal: it doesn't affect systems without linuxulator loaded, and even in rare cases where someone, for some reason, had an ancient, unbranded FreeBSD executable it can be worked around by tweaking the sysctl or by using brandelf(1).

trasz edited the summary of this revision. (Show Details)

Default to disabled, add description in the man page.

Based on feedback from Konstantin, it's too risky to make it enabled
by default. Based on discussion with Brooks, defaulting to enable _could_
have made sense for Linux jails - which are not a thing right now. And
when they do, this could be turned into a jail option.