Fix build of stand/ when building world with ASAN

[arichardson]

Summary

The userboot/test program links against the default userspace libraries
(e.g. shared libgcc_s.so) that will be instrumented if WITH_ASAN is set.
All other programs link against libsa instead of libc and therefore can't
use the sanitizer runtime library. To fix the stand/ build with
sanitizers, we disable MK_ASAN/MK_UBSAN if -nostdlib is found in the
LDFLAGS (i.e. we are using libsa instead of libc).

[markj]

A couple of weeks ago I was faulting through similar problems while trying to add a WITH_ADDRESS_SANITIZER knob. I didn't need this particular change though, and I don't really understand it - doesn't the libgcc_s dependency come from the ASAN runtime?

[arichardson]

In D31047#698403, @markj wrote:

A couple of weeks ago I was faulting through similar problems while trying to add a WITH_ADDRESS_SANITIZER knob. I didn't need this particular change though, and I don't really understand it - doesn't the libgcc_s dependency come from the ASAN runtime?

The problem here is that for MK_ASAN==yes libgcc_s is built with -fsanitize=address and therefore has references to the __asan_foo functions.
The problem here is that I add MK_ASAN=no to stand/Makefile.inc in https://reviews.freebsd.org/D31043, so -fsanitize=address is not passed as part of LDFLAGS.

I am not familiar at all with this part of the tree so I don't know what this test program is supposed to do, but I assume building it with ASAN might cause breakage. I can try removing MK_ASAN=no from the Makefile.inc but I am not sure that will work correctly.
Maybe it would also be possible to avoid passing -lgcc_s here?

[arichardson]

Different approach (stop forcing ASAN off for this directory)

[arichardson]

drop MK_MSAN (this does not work yet)

[markj]

In D31047#698407, @arichardson wrote:

In D31047#698403, @markj wrote:

A couple of weeks ago I was faulting through similar problems while trying to add a WITH_ADDRESS_SANITIZER knob. I didn't need this particular change though, and I don't really understand it - doesn't the libgcc_s dependency come from the ASAN runtime?

The problem here is that for MK_ASAN==yes libgcc_s is built with -fsanitize=address and therefore has references to the __asan_foo functions.
The problem here is that I add MK_ASAN=no to stand/Makefile.inc in https://reviews.freebsd.org/D31043, so -fsanitize=address is not passed as part of LDFLAGS.

I am not familiar at all with this part of the tree so I don't know what this test program is supposed to do, but I assume building it with ASAN might cause breakage. I can try removing MK_ASAN=no from the Makefile.inc but I am not sure that will work correctly.
Maybe it would also be possible to avoid passing -lgcc_s here?

The test program seems to only be used by tools/boot/lua-test.sh. I still don't see how anything in userboot is linking libgcc_s.

In fact I think you could enable ASAN/UBSAN for all of userboot, no?

[arichardson]

In D31047#698536, @markj wrote:

The test program seems to only be used by tools/boot/lua-test.sh. I still don't see how anything in userboot is linking libgcc_s.

In fact I think you could enable ASAN/UBSAN for all of userboot, no?

The other directories seem to use libsa instead of libc, so I don't think we can enable ASAN there. libgcc_s is linked by default unless you pass -nostdlib.

[imp]

userboot is just a user program that's linked into bhyve in a funky, but not too funky way.

stand/Makefile.inc

4 ↗

(On Diff #91797)

I think userboot* will work.

7 ↗

(On Diff #91797)

I think this whole thing likely should be near the top of defs.mk instead of Makefile.inc.
That will still get it applied to the whole tree, but we can more easily make userboot an exception.

[arichardson]

New (hopefully less ugly) approach

[arichardson]

drop debug print

[imp]

Looks good. Thanks

[tsoome]

Thanks!