Paths

Table of Contentst

Differential D51935

build: remove certctl requirement for host OpenSSL libs on macOS
ClosedPublic
Actions

Authored by kevans on Aug 16 2025, 8:08 PM.

Details

Reviewers

brooks
imp
jrtc27
ngie

Commits

rGaef16fc30bb2: build: remove certctl requirement for host OpenSSL libs on macOS

Summary

Some platforms, like macOS, do not expose headers for the system's
libcrypto for public consumption. libcrypto is relatively heavy and
needs to know, e.g., the host system's endianness, so we scope the build
down to macOS where OpenSSL headers are known to not be present and we
can be reasonably certain that most of the systems today that would be
cross-building are little endian.

We still don't bother if building WITHOUT_OPENSSL since the end result
is expected to be used by OpenSSL, but perhaps we could revisit that
independently in case one, e.g., brings their own implementation.

Reported by: jrtc27
Fixes: 81d8827ad875 ("certctl: Reimplement in C")

Diff Detail

Repository

rG FreeBSD src repository

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

kevans created this revision.Aug 16 2025, 8:08 PM

Herald added subscribers: emaste, bdrewery. · View Herald TranscriptAug 16 2025, 8:08 PM

kevans requested review of this revision.Aug 16 2025, 8:08 PM

Harbormaster completed remote builds in B66288: Diff 160437.Aug 16 2025, 8:08 PM

Clean cross-build here: https://github.com/freebsd/freebsd-src/actions/runs/17012332378/job/48230034448?pr=1814

jrtc27 added inline comments.Aug 16 2025, 9:04 PM

Makefile.inc1
2719	Can we not just set these appropriately based on (or modify the conditions themselves to use) BOOTSTRAPPING in libcypto's Makefiles?

kevans added inline comments.Aug 16 2025, 9:19 PM

Makefile.inc1
2716	whoops- the FreeBSD bit is leftover from local debugging, will remove.
2719	Sure, I don't have a strong opinion here -- I had mostly just found that it only needed a few tweaks and we largely had knobs to control them, so I tried to minimize the diff to libcrypto's build. I'll shift toward the direction you've indicated when I get back home.

@cperciva has fixed the cross-build by reverting the certctl rewrite, but better
to be prepared for its return. Push the bootstrap requirements into libcrypto's
Makefile, which is likely better from a maintenance perspective and makes the
difference between bootstrap and non-bootstrap libcrypto a little more clear.

Harbormaster completed remote builds in B66295: Diff 160447.Aug 17 2025, 2:21 AM

Please land this with this one bug fixed (you can add my Reviewed by: for that modified version of this patch), assuming you've verified it does fix the build still

secure/lib/libcrypto/Makefile.common
9

Fixes: 81d8827ad875 ("certctl: Reimplement in C")

Don't know if it should reference the re-landed version https://reviews.freebsd.org/rGc340ef28fd384b567e35882d04ce17fa31b7384f?

Whoops, ack; will fix, rerun through GitHub and push in a couple hours

CC @ngie for visibility, too, but the parts we're modifying here look to be in areas not touched by D52009

I wouldn't say building libcrypto is small necessarily... I need to understand why certctl is needed during the build process instead of later on (as part of the install process).

share/mk/bsd.endian.mk
28–31

I need to understand why certctl is needed during the build process instead of later on (as part of the install process).

It is during install, but that still happens on the possibly-not-FreeBSD host

In D51935#1188110, @ngie wrote:

I wouldn't say building libcrypto is small necessarily... I need to understand why certctl is needed during the build process instead of later on (as part of the install process).

certctl is used as part of distributeworld/installworld. Seems like a prime candidate to build as part of build-tools or cross-tools instead of prebuilding another copy/set of libraries.

In D51935#1188113, @ngie wrote:

In D51935#1188110, @ngie wrote:

I wouldn't say building libcrypto is small necessarily... I need to understand why certctl is needed during the build process instead of later on (as part of the install process).

certctl is used as part of distributeworld/installworld. Seems like a prime candidate to build as part of build-tools or cross-tools instead of prebuilding another copy/set of libraries.

That's what happens (bootstrap-tools), but it needs a libcrypto to build against, which is what this is doing.

Could you try this please?

% git diff ../../Makefile.inc1
diff --git a/Makefile.inc1 b/Makefile.inc1
index cf32248b6b9d..1180af17486e 100644
--- a/Makefile.inc1
+++ b/Makefile.inc1
@@ -2776,7 +2776,6 @@ bootstrap-tools: ${_bt}-links .PHONY
     ${_strfile} \
     usr.bin/dtc \
     ${_cat} \
-    ${_certctl} \
     ${_kbdcontrol} \
     ${_elftoolchain_libs} \
     ${_libkldelf} \
@@ -2943,6 +2942,7 @@ cross-tools: .MAKE .PHONY
 .for _tool in \
     ${LOCAL_XTOOL_DIRS} \
     ${_ar} \
+    ${_certctl} \
     ${_clang_libs} \
     ${_clang} \
     ${_lld} \

In D51935#1188115, @ngie wrote:

Could you try this please?

% git diff ../../Makefile.inc1
diff --git a/Makefile.inc1 b/Makefile.inc1
index cf32248b6b9d..1180af17486e 100644
--- a/Makefile.inc1
+++ b/Makefile.inc1
@@ -2776,7 +2776,6 @@ bootstrap-tools: ${_bt}-links .PHONY
     ${_strfile} \
     usr.bin/dtc \
     ${_cat} \
-    ${_certctl} \
     ${_kbdcontrol} \
     ${_elftoolchain_libs} \
     ${_libkldelf} \
@@ -2943,6 +2942,7 @@ cross-tools: .MAKE .PHONY
 .for _tool in \
     ${LOCAL_XTOOL_DIRS} \
     ${_ar} \
+    ${_certctl} \
     ${_clang_libs} \
     ${_clang} \
     ${_lld} \

I don't see where we'd build a host libcrypto anywhere between bootstrap-tools and there to fix this, but I've started a run with that anyways to see: https://github.com/freebsd/freebsd-src/actions/runs/17056345162/job/48354662107?pr=1814

Indeed, that run has also failed because we don't have ssl headers available on the host

ngie added inline comments.Aug 19 2025, 12:57 AM

secure/lib/libcrypto/Makefile
2	Or just define `-DLIBCRYPTO_WITHOUT_SUBDIRS`, like other portions of `Makefile.inc1` do?
secure/lib/libcrypto/Makefile.common
19	This doesn't seem like it would work properly if the variables seen above were defined as empty values. Hmm..

kevans added inline comments.Aug 19 2025, 1:05 AM

secure/lib/libcrypto/Makefile
2	The original version of this diff did, but in earlier comments Jessica had pondered a bit on going in this direction. I think it's more useful in some sense to be here for folks less familiar with our build system; a simple grep of BOOTSTRAPPING can point out how the build differs between the host build and the actual target build, rather than having to grok the the earlier stages of buildworld and follow the breadcrumbs back to libcrypto. My other thought here in changing it was that it'd be easier on maintenance if we had an explicit marker on things that were actually used in the bootstrap phase if you need to, e.g., make a judgement call bout whether these knobs were useful. Makefile.inc1 can be overlooked quite easily, it seems.

jrtc27 added inline comments.Aug 19 2025, 1:19 AM

secure/lib/libcrypto/Makefile
2	LIBCRYPTO_WITHOUT_SUBDIRS is for teasing apart _prebuild_libs and _generic_libs. It is arguably a hack that should be done another way, as it bakes very specific knowledge about libcrypto in a very generic place at the top-level. But, at the end of the day, it's not about restricting the total set of things that get built, just about doing so in a careful order. Whereas for BOOTSTRAPPING we really do want to restricting things. It is generally best to keep BOOTSTRAPPING checks close to where they take effect, otherwise you can end up with "effect at a distance" where it's very confusing to understand how things are actually working. This way, Makefile.inc1 knows it wants a bootstrap libcrypto, and it's up to libcrypto to deal with how to make that work. Separation of concerns, abstraction, all that good stuff.
share/mk/bsd.endian.mk
28–31	It's not about configuration. That's what their ISA is. Yes, big-endian arm64 does kind of exist in theory, but that is not hardware they have built.

In D51935#1188118, @kevans wrote:

Indeed, that run has also failed because we don't have ssl headers available on the host

Yes, I see the pickle you're dealing with now.

I'll get out of the way. This is not super ideal, but no less ideal than the current implementation I suppose.

It might be a good idea to split off the bsd.endian.mk change from the other changes BTW so it can be more easily MFCed.

Makefile.inc1
2716–2719	This is needed, regardless of the OS, in order to compile with the version of libcrypto that the version of the application is supposed to compile with. For example, the sources in tree require OpenSSL 3.5 but the host provides OpenSSL 1.1.

This revision is now accepted and ready to land.Aug 19 2025, 2:07 AM

jrtc27 added inline comments.Aug 19 2025, 2:19 AM

Makefile.inc1
2716–2719	Assuming that _ is meant to be a newline, it's not needed on Linux, and I'm not sure it is on FreeBSD either? Won't the .depend file pick up the headers changing and rebuild it after you've upgraded your host's OpenSSL on the next build? AFAIK we don't take steps for other bootstrap tools to handle soname bumps of the host system across versions.

In D51935#1188136, @ngie wrote:

In D51935#1188118, @kevans wrote:

Indeed, that run has also failed because we don't have ssl headers available on the host

Yes, I see the pickle you're dealing with now.

I'll get out of the way. This is not super ideal, but no less ideal than the current implementation I suppose.

I mean, "not super ideal" is an apt description for FreeBSD's build system in general :)

It might be a good idea to split off the bsd.endian.mk change from the other changes BTW so it can be more easily MFCed.

I suppose you could, though I'd be surprised if we grew more uses for this. I view most of the current uses as a bit gratuitous; CFLAGS uses can instead be done based on preprocessor macros. But there's a whole host of changes I'd make to how we deal with make variables and flags if I had the time...

kevans added inline comments.Aug 19 2025, 2:27 AM

Makefile.inc1
2716–2719	Right, I'm not particularly worried about out-of-date OpenSSL for certctl's usage; we're mostly using it for convenient der/pem/x509 parsing and not really a wide array of features. IMO it's a bridge we should cross if we find a compelling need for some API that hasn't been widely available, because getting a libcrypto build on Linux doesn't feel very fun at the moment.