Page MenuHomeFreeBSD

Fix build of stand/ when building world with ASAN
ClosedPublic

Authored by arichardson on Jul 5 2021, 11:33 AM.

Details

Summary

The userboot/test program links against the default userspace libraries
(e.g. shared libgcc_s.so) that will be instrumented if WITH_ASAN is set.
All other programs link against libsa instead of libc and therefore can't
use the sanitizer runtime library. To fix the stand/ build with
sanitizers, we disable MK_ASAN/MK_UBSAN if -nostdlib is found in the
LDFLAGS (i.e. we are using libsa instead of libc).

Diff Detail

Repository
rG FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

A couple of weeks ago I was faulting through similar problems while trying to add a WITH_ADDRESS_SANITIZER knob. I didn't need this particular change though, and I don't really understand it - doesn't the libgcc_s dependency come from the ASAN runtime?

A couple of weeks ago I was faulting through similar problems while trying to add a WITH_ADDRESS_SANITIZER knob. I didn't need this particular change though, and I don't really understand it - doesn't the libgcc_s dependency come from the ASAN runtime?

The problem here is that for MK_ASAN==yes libgcc_s is built with -fsanitize=address and therefore has references to the __asan_foo functions.
The problem here is that I add MK_ASAN=no to stand/Makefile.inc in https://reviews.freebsd.org/D31043, so -fsanitize=address is not passed as part of LDFLAGS.

I am not familiar at all with this part of the tree so I don't know what this test program is supposed to do, but I assume building it with ASAN might cause breakage. I can try removing MK_ASAN=no from the Makefile.inc but I am not sure that will work correctly.
Maybe it would also be possible to avoid passing -lgcc_s here?

Different approach (stop forcing ASAN off for this directory)

drop MK_MSAN (this does not work yet)

A couple of weeks ago I was faulting through similar problems while trying to add a WITH_ADDRESS_SANITIZER knob. I didn't need this particular change though, and I don't really understand it - doesn't the libgcc_s dependency come from the ASAN runtime?

The problem here is that for MK_ASAN==yes libgcc_s is built with -fsanitize=address and therefore has references to the __asan_foo functions.
The problem here is that I add MK_ASAN=no to stand/Makefile.inc in https://reviews.freebsd.org/D31043, so -fsanitize=address is not passed as part of LDFLAGS.

I am not familiar at all with this part of the tree so I don't know what this test program is supposed to do, but I assume building it with ASAN might cause breakage. I can try removing MK_ASAN=no from the Makefile.inc but I am not sure that will work correctly.
Maybe it would also be possible to avoid passing -lgcc_s here?

The test program seems to only be used by tools/boot/lua-test.sh. I still don't see how anything in userboot is linking libgcc_s.

In fact I think you could enable ASAN/UBSAN for all of userboot, no?

The test program seems to only be used by tools/boot/lua-test.sh. I still don't see how anything in userboot is linking libgcc_s.

In fact I think you could enable ASAN/UBSAN for all of userboot, no?

The other directories seem to use libsa instead of libc, so I don't think we can enable ASAN there. libgcc_s is linked by default unless you pass -nostdlib.

userboot is just a user program that's linked into bhyve in a funky, but not too funky way.

stand/Makefile.inc
4 ↗(On Diff #91797)

I think userboot* will work.

7 ↗(On Diff #91797)

I think this whole thing likely should be near the top of defs.mk instead of Makefile.inc.
That will still get it applied to the whole tree, but we can more easily make userboot an exception.

New (hopefully less ugly) approach

arichardson retitled this revision from Fix build of stand/userboot when building world with ASAN to Fix build of stand/ when building world with ASAN.Jul 19 2021, 11:58 AM
arichardson edited the summary of this revision. (Show Details)
arichardson edited the summary of this revision. (Show Details)

drop debug print

This revision is now accepted and ready to land.Jul 19 2021, 2:12 PM