Enable WITHOUT_INET* WITHOUT_CRYPT WITHOUT_OPENSSL, WITHOUT_FILE

[editor_callfortesting.org]

Summary

This diff for FreeBSD 12.0-ALPHA4 aims to enable the following FreeBSD build options:

WITHOUT_INET
WITHOUT_INET_SUPPORT
WITHOUT_INET6
WITHOUT_INET6_SUPPORT
WITHOUT_CRYPT
WITHOUT_OPENSSL
WITHOUT_DYNAMICROOT
WITHOUT_FILE

I have tested it with moderately random mixes of each (i.e. WITHOUT_INET and WITHOUT_INET6_SUPPORT) plus clean builds.

Known shortcoming: tests/sys/Makefile should really exclude only /usr/src/tests/sys/geom/class/eli but doing so appears to break the pre-build the clean up.

Subject matter experts please review and test it.

Why? I have experimented with "build options" since I discovered FreeBSD Jail around 2003 with stunningly frustrating results because of their lack of maintenance. Please test all build options you touch. Please.

Test Plan

Apply the patch and limit the build option survey to:

OPLIST="
WITHOUT_ZONEINFO
WITHOUT_CRYPT
WITHOUT_OPENSSL
WITHOUT_INET
WITHOUT_INET6
WITHOUT_INET_SUPPORT
WITHOUT_INET6_SUPPORT
WITHOUT_DYNAMICROOT
WITHOUT_FILE
"

Consider various arrangements of these under normal build and installworld.

Be sure to try installworld too!

[emaste]

For any future uploads please include full context (e.g. from git diff -U9999 or equivalent).

I believe at least some of these should be addressed by fixing whatever failed rather than by excluding parts from the build; do you have a list of the errors you encountered with the various combinations? i.e., I would be interested in seeing how ping and dhclient fail under DYNAMICROOT.

[rgrimes]

tools/tools/build_option_survey/option_survey.sh

121

Please break this out into a seperate diff and review, I'll get that pushed into the tree ASAP as this is a trivial issue. Same with the above rsync removel. Those 2 fixes can just go in, mixing this with a bunch of other issues makes it un necessarily complicated.

[editor_callfortesting.org]

For any future uploads please include full context (e.g. from git diff -U9999 or equivalent).

And from SVN?

I'll post the errors!

[editor_callfortesting.org]

I believe at least some of these should be addressed by fixing whatever failed rather than by excluding parts from the build; do you have a list of the errors you encountered with the various combinations? i.e., I would be interested in seeing how ping and dhclient fail under DYNAMICROOT.

From a separate "build option survey" build of WITHOUT_DYNAMICROOT:

• all_subdir_secure ---

cc -target x86_64-unknown-freebsd12.0 --sysroot=/usr/obj/usr/src/tools/tools/bui
ld_option_survey/_.tst/usr/src/amd64.amd64/tmp -B/usr/obj/usr/src/tools/tools/bu
ild_option_survey/_.tst/usr/src/amd64.amd64/tmp/usr/bin -pg -O2 -pipe -I/usr/
src/crypto/openssl -DTERMIOS -DANSI_SOURCE -DOPENSSL_THREADS -DDSO_DLFCN -DHAVE_
DLFCN_H -DL_ENDIAN -DOPENSSL_IA32_SSE2 -DAES_ASM -DBSAES_ASM -DVPAES_ASM -DECP_N
ISTZ256_ASM -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -
DMD5_ASM -DGHASH_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DWHIRLPOOL_ASM -I/usr
/obj/usr/src/tools/tools/build_option_survey/_.tst/usr/src/amd64.amd64/secure/li
b/libcrypto -I/usr/src/crypto/openssl/crypto -I/usr/src/crypto/openssl/crypto/as
n1 -I/usr/src/crypto/openssl/crypto/evp -I/usr/src/crypto/openssl/crypto/modes -
g -MD -MF.depend.x509_cmp.po -MTx509_cmp.po -std=gnu89 -fstack-protector-strong
-Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variab
le -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-un
used-function -Wno-enum-conversion -Wno-unused-local-typedef -Wno-address-of-pac
ked-member -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Wno-parenth
eses -Qunused-arguments -c /usr/src/crypto/openssl/crypto/x509/x509_cmp.c -o x
509_cmp.po

• all_subdir_sbin ---

cc: error: linker command failed with exit code 1 (use -v to see invocation)

• [dhclient.full] Error code 1

make[4]: stopped in /usr/src/sbin/dhclient

[editor_callfortesting.org]

emaste,

From another 12.0-ALPHA4 build:

• all_subdir_bin ---

objcopy --only-keep-debug expr.full expr.debug

• all_subdir_lib ---

cc: error: linker command failed with exit code 1 (use -v to see invocation)

• all_subdir_bin ---

istine/share/colldef/../../tools/tools/locale/etc/final-maps/map.UTF-8 /usr/obj/
usr/srcpristine/amd64.amd64/share/colldef/ca_AD.UTF-8

• all_subdir_bin ---
• expr.1.gz ---

gzip -cn /usr/srcpristine/bin/expr/expr.1 > expr.1.gz

• all_subdir_sbin ---

/usr/bin/ld: error: unable to find library -lcasper

• all_subdir_lib ---
• e_gammaf.po ---
• all_subdir_sbin ---
• all_subdir_sbin/dhclient/tests ---
• option-domain-search_test ---

(cd /usr/srcpristine/sbin/dhclient/tests && DEPENDFILE=.depend.option-domain-se
arch_test NO_SUBDIR=1 make -f /usr/srcpristine/sbin/dhclient/tests/Makefile _RE
CURSING_PROGS=t PROG=option-domain-search_test )

• dhclient.full ---

/usr/bin/ld: error: unable to find library -lcap_syslog
cc: error: linker command failed with exit code 1 (use -v to see invocation)

• all_subdir_bin ---
• expr.debug ---
• all_subdir_sbin ---
• [dhclient.full] Error code 1

make[4]: stopped in /usr/srcpristine/sbin/dhclient

[editor_callfortesting.org]

Adding the 'svnlite diff -x -U99999' full diff. Not yet broken it up.

[emaste]

Ah,

/usr/bin/ld: error: unable to find library -lcap_syslog

casper services are only available as shared libraries.

Looking at dhclient's Makefile I see .if ${MK_CASPER} != "no" && !defined(RESCUE); we ought to incorporate MK_DYNAMICROOT into that as well. Something like

.if ${MK_DYNAMICROOT} == "no"
.warning dhclient built without libcasper support
.elif ${MK_CASPER} != "no" && !defined(RESCUE)
LIBADD+=        casper
...

[emaste]

Add @oshogbo for comment on the dhclient libcasper issue

[emaste]

tools/tools/build_option_survey/mkhtml.sh

222–224

Agree with @rgrimes this should just go in now, separately.

[editor_callfortesting.org]

I have updated the diff to remove the build option survey components and have submitted a separate diff to rgrimes for his review and review by his mentors. This is based on 12.0-ALPHA4.

[editor_callfortesting.org]

This comment was removed by editor_callfortesting.org.

[editor_callfortesting.org]

I have split out the build option survey components and WITHOUT_DYNAMICROOT.

Huge thanks to rgrimes for getting the "BOS" patch reviewed and committed, and to emaste for posting a separate WITHOUT_DYNAMICROOT. patch that I have built and installed:

https://reviews.freebsd.org/D17074

I have removed both of those components from this diff.

WITHOUT_FILE, WITHOUT_CRYPT!OPENSSL and WITHOUT_INET* all are quite intertwined, with the possible exception of WITHOUT_FILE, though it applies to *SVN*, which in turn depends on OPENSSL and INET*

[oshogbo]

@emaste The part about Casper LGTM.

[emaste]

Some more comments inline - as mentioned earlier we should try to address the root causes, as in D17074. That said, I'm fine with this approach pending root cause fixes (given that it doesn't build today), but want to make sure we have some way to track these outstanding issues such that they'll get fixed after the release.

lib/Makefile

146

Ideally we'd build a ssl-less libfetch if the OPENSSL knob is off rather than avoiding building libfetch altogether, and the same for other cases (where feasible). That will take more investigation though.

206

libfetch being added 2nd time here

tests/sys/Makefile

17–18

can we reduce the scope of this to only skipping one (or a few) tests rather than all of geom/?

usr.sbin/syslogd/Makefile

16

these are probably better handled by having one of these be dependent on the other - i.e., WITHOUT_INET_SUPPORT should imply WITHOUT_INET I think. @bz ?

[bz]

I should probably add that I never liked having two options, but it's been inherited from what we had (I think v6 had it and I did the same for legacy). But since our option space has been way too fine grained to my believe anyway for too many things, which is a totally different issue.

usr.bin/Makefile

222–223

This looks wrong to me. See comment further below on the syslogd Makefile.

usr.sbin/syslogd/Makefile

16

I think the new check is bogus. WITHOUT_INET{,6} means do not build the program at all; that also sets (hopefully still) WITHOUT_INET{,6}_SUPPORT.

WITHOUT_INET{,6}_SUPPORT means don't build the address family component into the program/library/.. but still build the program/library/..

So in theory the previous Makefile was correct (also for v6 below). If the code does not build any longer, please fix the code.

[rgrimes]

In D17040#364538, @emaste wrote:

Some more comments inline - as mentioned earlier we should try to address the root causes, as in D17074. That said, I'm fine with this approach pending root cause fixes (given that it doesn't build today), but want to make sure we have some way to track these outstanding issues such that they'll get fixed after the release.

I would prefer we only do the root cause fixes, as if we bypass the failing build the root cause fix well never happen. These things have been broken for a long time, and with out Michael Dexter driving this issue to get them fixed they would never even be addressed. If we apply his simple fixes we loose that pressure from him to have these fixes get in the tree.

@ed can we get this added to the CI work load as a once a week type run? IIRC it takes a 16 core machine about 3 days to run build_option_survey.sh

[ed]

In D17040#364634, @rgrimes wrote:

@ed can we get this added to the CI work load as a once a week type run? IIRC it takes a 16 core machine about 3 days to run build_option_survey.sh

This was intended for @emaste, right?

[editor_callfortesting.org]

This is partially obsoleted by https://reviews.freebsd.org/D19418

Great work bz@!

[editor_callfortesting.org]

Fixed by bz@ in 2019 and re-fixed in 2021