secure/lib/lib{crypto,ssl}: regenerate Makefiles

[ngie]

Summary

This change regenerates the Makefiles in an automated manner using the
output from gmake and some semi-complex templating logic written in
python using the Jinja2 templating framework.

Sources and headers have been moved to more appropriate areas and the
Makefiles have been partitioned in a way to better represent how the
upstream build functions in terms of the [intermediate] provided
libraries, e.g., libcommon.a, libdefault.a, liblegacy.a, etc.

This corrects the legacy provider by ensuring that legacy algorithms are
no longer loaded as part of the base/default providers, but instead are
only loaded as part of the legacy provider (where they should be). This
helps ensure that algorithms deemed legacy (generally insecure) aren't
enabled in libcrypto, etc, in the default configuration. Loading the
legacy provider (added in 3.0) has required one of the following be met:

• The administrator must specify the needed config file entries to load the base and legacy providers (if legacy is needed).
• Developers must explicitly load the base and legacy providers.

Relnotes: yes

Test Plan

• I compared and contrasted the outputs from the Makefiles (before and after) in order to confirm that the delta was to be expected versus the upstream provided Makefile/buildinfo files.

• legacy provider differences (SRCS): https://reviews.freebsd.org/P669
• libcrypto differences (INCS): https://reviews.freebsd.org/P667
• libcrypto differences (SRCS): https://reviews.freebsd.org/P667
• libssl differences (SRCS): https://reviews.freebsd.org/P666
• The changes work on amd64 after param_idx.c is re-added to secure/lib/libcrypto/modules/legacy/Makefile.

[ngie]

Chase D51663: update the Makefiles again for algorithms change

[ngie]

Update with working content

This compiles on amd64 at least. make tinderbox's in progress.

I'll work on polishing up the directions, templates, etc, for
general consumption.

[ngie]

secure/lib/libcrypto/Makefile

1338–1344

These should never be installed. Not sure why my Makefile generator didn't handle this properly...

1492

For some odd reason this isn't showing up in DT_NEEDED when I compile this library, which is resulting in downstream issues with libssl, etc 🤔 (they claim they need libpthread, even though libcrypto _should_ be getting linked to libpthread).. I'm not sure why yet.

[ngie]

Do not install internal headers

[khorben]

Should we involve secteam@ regarding the extra algorithms enabled?
I am not aware of any security concern there as of today, but I think we had stuck with OpenSSL's own list of default algorithms so far for the base system.

• https://en.wikipedia.org/wiki/ARIA_(cipher)
• https://en.wikipedia.org/wiki/SM9_(cryptography_standard)

AFAICT this should probably also be reflected in crypto/openssl/include/configuration.h, by removing the defines OPENSSL_NO_ARIA, OPENSSL_NO_SM2, OPENSSL_NO_SM3, and OPENSSL_NO_SM4.

[ngie]

Hey there!

In D52009#1191377, @khorben wrote:

Should we involve secteam@ regarding the extra algorithms enabled?
I am not aware of any security concern there as of today, but I think we had stuck with OpenSSL's own list of default algorithms so far for the base system.

...

AFAICT this should probably also be reflected in crypto/openssl/include/configuration.h, by removing the defines OPENSSL_NO_ARIA, OPENSSL_NO_SM2, OPENSSL_NO_SM3, and OPENSSL_NO_SM4.

I removed that portion of the earlier change -- it was making life a lot more painful as far as fixing the upgrade was concerned. Enabling these algorithms should be its own separate change.

[jhb]

I think it would be useful (if possible) to reformat the list of existing SRCS as a separate commit. It will be clearer what the differences from the regeneration are when it comes to reviewing those, and will also be easier to bisect if needed.

[ngie]

In D52009#1193198, @jhb wrote:

I think it would be useful (if possible) to reformat the list of existing SRCS as a separate commit. It will be clearer what the differences from the regeneration are when it comes to reviewing those, and will also be easier to bisect if needed.

I don't have a ton of time to deliver this before 15.0-RELEASE, so I'm going to manually "massage" the Makefiles in order to deliver something working for 3.5.1, then pivot to 3.5.2 (which has been out for over a month now).

[ngie]

In D52009#1197108, @ngie wrote:

In D52009#1193198, @jhb wrote:

I think it would be useful (if possible) to reformat the list of existing SRCS as a separate commit. It will be clearer what the differences from the regeneration are when it comes to reviewing those, and will also be easier to bisect if needed.

I don't have a ton of time to deliver this before 15.0-RELEASE, so I'm going to manually "massage" the Makefiles in order to deliver something working for 3.5.1, then pivot to 3.5.2 (which has been out for over a month now).

Done as part of D52554 (the Makefile update part -- not the reformatting part).

secure/lib/libcrypto/modules/legacy/Makefile

1

I need to add params_idx.c -- otherwise this module won't load.
See also: D52554#1200787

[ngie]

secure/lib/libcrypto/Makefile

1492

This wasn't an issue with later iterations on this change.