Optionally build/link programs against the compat ABI.

[brooks]

Summary

Compilation/linkage is controlled by two make knobs:

WANT_COMPAT - Prefer to use the compat ABI, fall back to native.
NEED_COMPAT - Use the compat ABI or fail to build.

Supported values are "32", "soft", and "any". The latter meaning pick
the first[0] supported compat ABI.

This can be used to provide test binaries for compat ABIs or to link
ABI-specific programs.

[0] We currently support only one compat ABI at a time, but this may
change in the future and some code in this commit is structured to ease
that change.

[brooks]

To provide a little more background: We have this feature in CheriBSD to complement a set of (build|install|...)cheri targets forked from the old build32, etc targets before Makefile.libcompat was created. We're in the process of killing that off, but find this functionality useful. Among other things, it lets us build tests for compatibility libraries. It can also aid in bringing up a new compat layer (or base system ABI if you coerce all the programs to build for a compat layer).

I've got this broken into ~5 commits in my tree, but it's small enough that a single review seemed sufficient.

share/mk/src.libnames.mk

541 ↗

(On Diff #63257)

I didn't update these because I've not found it to be required and I'm not sure what they are for. I'm happy to do so if it makes sense.

[jhb]

So one thought I had had when looking at this as well (in the context of trying to update libcheri to use Makefile.libcompat which we won't bother with, but meant reading much of this stuff) is that I really feel like we should install /usr/lib<foo> into the normal sysroot instead of a separate sysroot. This means builds of programs can always use the same --sysroot (and it also makes it possible to use the same --sysroot from the obj tree for different ABIs when building out-of-tree programs).

One other note is that when you move the libcompat builds earlier before everything, the labels (4.2.1 or whatever) are now wrong and need to be adjusted. We didn't bother for libcheri, but you would see the relative order being wrong if you save a buildworld / installworld log and grep for the stages.

[brooks]

• Re-number stages as part of reordering.

[brooks]

In D22023#481075, @jhb wrote:

So one thought I had had when looking at this as well (in the context of trying to update libcheri to use Makefile.libcompat which we won't bother with, but meant reading much of this stuff) is that I really feel like we should install /usr/lib<foo> into the normal sysroot instead of a separate sysroot. This means builds of programs can always use the same --sysroot (and it also makes it possible to use the same --sysroot from the obj tree for different ABIs when building out-of-tree programs).

This seems like a good idea, but I think I'd prefer to avoid it in this batch of changes.

[jhb]

In D22023#481108, @brooks wrote:

In D22023#481075, @jhb wrote:

So one thought I had had when looking at this as well (in the context of trying to update libcheri to use Makefile.libcompat which we won't bother with, but meant reading much of this stuff) is that I really feel like we should install /usr/lib<foo> into the normal sysroot instead of a separate sysroot. This means builds of programs can always use the same --sysroot (and it also makes it possible to use the same --sysroot from the obj tree for different ABIs when building out-of-tree programs).

This seems like a good idea, but I think I'd prefer to avoid it in this batch of changes.

That's fair. I haven't implemented it either, was just a thought I had for the future.

[bdrewery]

This is great!

Maybe update share/mk/bsd.README and/or build(7) to make it known.

[bdrewery]

We have use for this at work as well for FreeBSD amd64/32 ABI compat testing since we have our own ABI. Though we just use FreeBSD 10 dist files for the purpose this feature is still nice.

[brooks]

Preparatory commits done in rS353592 rS353593 rS353594, rS353595.