Besides my comment above, I wonder if we want to apply a different namespace to the SBOM files generated for base (in D56474 and D60352), or if it should be up to the user to set the correct PKG_CONFIG_LIBDIR. With pkgbase coming, we may want to be able to express dependencies on components from the base system anyway.
Should I change the naming in the base system?
Should we align names between ports and the base system? I can picture cases like expat/libexpat/libbsdxml, curl/libcurl, openssl{36,41,42} etc to be tricky.

This adds copyright information for lib/libexpat (aka libbsdxml).

In D56474#1380498, @stephane.rochoy_stormshield.eu wrote:

In D56474#1379752, @khorben wrote:

The reasoning is to have one SBOM per package and per third-party project. In practice this is not really possible, because if a project exposes a library in addition to itself (e.g., libssl and libcrypto for OpenSSL) we still need to easily be able to express the dependency on OpenSSL, or at least on a specific component of OpenSSL.

In the current version of this proposal for the generation framework, dependencies are tracked automatically. When including internal libraries in this mechanism, a single project like wpa_supplicant gets 16 SBOM files in addition to itself (e.g., libwpatls, libwpapasn, libwpadrivers...). From what I understand of the INTERNALLIB flag, these libraries are not meant to be re-used by other projects. Therefore I have reflected this here, by excluding such libraries from the automatic dependency list (by using _INTERNALLIBS as per src.libnames.mk).

In this example, wpa_supplicant still gets its corresponding SBOM, and is shipped in a single package without an additional library package, so I think it makes sense to stick to a single SBOM.

With this said, if your project wants to express a dependency on an internal library specifically, it is possible to override SBOM_REQUIRES to list it there. As things currently are, this will probably require the creation of a corresponding .pc file for this library, which can be done with the creation of a Makefile.sbom file in the directory of that library.

Let me know if this is still not clear, if it looks good enough for now, or if I should modify anything in any way.

Do you know if INTERNALLIBS also encompass the libprivate* libs? Because this libs should be exposed in the SBOM IMO.

In D56474#1379216, @stephane.rochoy_stormshield.eu wrote:

In D56474#1378998, @khorben wrote:

Dependencies on internal libraries are now ignored; in the context of SBOMs, they are useless duplicates of information.

I may have missed some aspect of the reasoning here. Could you explain how we express dependency on internal libraries then?

Dependencies on internal libraries are now ignored; in the context of SBOMs, they are useless duplicates of information.

This update adds support for libraries installing multiple instances under different names, through the use of SYMLINKS. This is the case for lib/ncurses/ncurses for instance, which builds and installs libncursesw, but also aliases it as libncurses, libcursesw, and libcurses.

In D56474#1375885, @mat wrote:

In D56474#1375309, @khorben wrote:

In D56474#1369872, @mat wrote:

1. the framework would generate a ${PKGBASE}.pc file during staging, put it in, say, /usr/local/share/sbom/pc and add it to the package.

I don't think this intermediate ${PKGBASE}.pc file needs to be kept.

2. a pkg trigger would run a script on install to run bomtool on the pc file and generate a .spdx file in /usr/local/share/sbom/spdx-2.2, and run spdxtool to generate the .jsonld file in /usr/local/share/sbom/spdx-3.0.1.
3. another pkg trigger would remove the .jsonld and the .spdx files on deinstall.

Why not simply generate the .jsonld and .spdx files during package creation, and include them in the plist?
Then there is no need for any triggers.

Well, say, port devel/foo 1.0 builds a binary that depends on libbar.so that comes from devel/bar 1.0. If the SBOM files are generated at build time, the ones for foo will contain the version of bar that foo was built with.
Now bar goes to 1.1 where abi is still compatible, and the so version stays the same, foo will probably be rebuilt on the builders (but that will probably change in the near future) you run pkg upgrade, and only bar gets upgraded, then you have foo 1.0 and bar 1.1, foo's SBOM files still reference bar 1.0.

Note that this is not a corner case example, it happens something like hundreds of time at each package builds.

So, we either need triggers that rebuild all the SBOM files from the .pc files after each install/upgrade, or a new port that does the generation and that runs from a crontab, or that you run when you need the SBOM files.

It seems better to have a trigger do this so the SBOM say consistent with what is installed.

In D56474#1369872, @mat wrote:

In D56474#1369338, @khorben wrote:

In D56474#1369278, @mat wrote:

I have been thinking about sbom for ports a lot the past week, and this gives me a good idea of which way I should head into.

Awesome!

As a package set is built incrementally, I am wondering if the sbom of a port needs to include the software it has been built with, for example, a port may need ninja, yacc or gettext to build, but not to run.

This bug is being fixed in pkg instead at https://github.com/freebsd/pkg/issues/2755, with an ongoing review for the fix at https://github.com/freebsd/pkg/pull/2754.

In D59670#1369462, @ivy wrote:

i'm curious why we have files with spaces in on the release to begin with. where do those come from?

In D56474#1369278, @mat wrote:

I have been thinking about sbom for ports a lot the past week, and this gives me a good idea of which way I should head into.

Adding pkg for review, since this might really be a bug with the -o METALOG option in pkg(8).

• Fixed an issue looking up dependencies between SBOM files
• Added an SBOM file for libpkgconf to illustrate and test dependencies
• Added support for SCRIPTS and SHLIB

With this new version of the SBOM generation framework, I have introduced the following changes:

In D56474#1336929, @ivy wrote:

In D56474#1317500, @markj wrote:

I don't really see how this is going to stay up-to-date: this seems to depend on developers remembering to update the file each time they add a dependency.

i agree and i object to landing this as-is; this simply isn't a workable solution.

when the original SBOM proposal was announced, i was disappointed when FF declared that pkgbase was out-of-scope for that work, but it seems like this and pkgbase and trying to solve similar problems when it comes to dependencies. can we not find a solution that works for both without requiring manual upkeep?

No longer build bomtool(1) and spdxtool(1) with NO_SHARED in the tooling phase. (Reported by imp@, thanks!)

This should fix the cross-compilation of libpkgconf while building bomtool(1) and spdxtool(1) as tools.

I just saw that this breaks cross-compilation, when building libpkgconf early for bomtool(1) and spdxtool(1) as tools for SBOM generation:

Generate SPDX files in version 3.0.1 as well by default, now that spdxtool(1) has been imported into the base system.
Sets the correct version of FreeBSD automatically into the SBOM files generated for FreeBSD's own software.

Removed use of NO_SHARED for the test binaries.

In D58034#1333641, @markj wrote:

Are the tests all passing now?

Copied missing data.
Registered the missing test directories in etc/mtree/BSD.tests.dist.
Forced a C locale to avoid the UTF-8 test. (Original test file removed from our import because of UTF-8 filename)

Now copying most of the test data required: only 2 tests do not pass on my local setup (the spdxtool binary is required and I have not installed it yet).

Also hooked the serialisation test for spdxtool.

Added the SPDX license header.
Added discovery mechanism for the location of the test executables.

Imported more tests from the 2.9.93 release.
Some still remain to be integrated, like for spdxtool specifically.

Fixed inconsistent indentation in etc/mtree/BSD.tests.dist.

• Update wording for packages/pkgconf/pkgconf.ucl, as per Mark's suggestion.

• List SRCS and INCS with one line per file, as per Mark's suggestion.
• Clarify where the information was gathered from.

Build a missing file for spdxtool(1).

Imported in ebcc78644e0cbc4141807c0887ef8d902cb91bb4.

Removed a test folder in tests/lib1 which broke the generation of source packages (UTF-8).

In D57628#1321573, @markj wrote:

How does one verify that a given executable is compiled with FORTIFY_SOURCE > 0?

Imported adjustments from Tuukka.

Remove redundant code, as suggested by des@. (Thanks!)

Removed redundant braces.

Simplified the code obtaining LOCALBASE as per @des' suggestion. (Thanks!)

• Avoid file conflicts in the -lib32 base packages (no duplicate installation of SPDX files)
• Rename the PCFILE variable to SBOMFILE to avoid confusion with PCFILES
• Fix typo in libmd.pc (thanks stephane.rochoy_stormshield.eu for the heads up!)

Use a stack buffer of MAXPATHLEN for localbase, as suggested by @des.

This update takes care of PERSONALITY_PATH in addition to PKG_DEFAULT_PATH.

In D56404#1309826, @markmi_dsl-only.net wrote:

What is the relationship of this type of change to use of ${PREFIX} when ports install themselves and it is not the /usr/local default value? (There is also a linux compatibility related standard default for PREFIX and folks can specify things like /opt .) There was a later change to use /usr/local/ when LOCALBASE was undefined.

But in response to that last change there is now from eduardo@freebsd.org :

This change doens't honour PREFIX when it is set to other path than
default "/usr/local/"

(Due to the message Nuno picked to reply to I ended up listed as the "To:", des and the lists Cc:'d.)

I figured I could at least ask the question here of the folks involved in the original review so that the question is not completely lost.

In D56474#1308969, @ivy wrote:

a couple of comments:

• i'm not sure this should be in release/, since it doesn't seem specific to building releases. iiuc, any src build will include the SBOM data if enabled.

In D56474#1309051, @ziaee wrote:

One thing we may want to investigate is building the SBOM files into the ELF headers instead. Sony has compiler extensions to do that -- https://github.com/sony/esstra/tree/poc/rust-llvm

• The SPDX files are now placed in their respective base packages.
• The source .pc files from illuusio were imported into release/sbom.

LGTM, thanks!

In D56591#1296034, @khorben wrote:

In D56591#1295928, @khorben wrote:

The latest update (with WARNS?=3) built correctly with clang; I am still testing with GCC 14.

It built correctly with GCC 14 as well.

Reverted the copyright to myself, and the license to ISC.

In D56589#1296262, @ngie wrote:

In D56589#1296252, @khorben wrote:

I gave the copyright back to pkgconf, as the comment and description primarily come from the pkgconf project.

While there, I also updated the SPDX-License-Identifier to pkgconf, as per the discussion here, and confirmed at https://spdx.org/licenses/pkgconf.html.

Does that mean that you need to credit DuckDuckGo or Google whenever you find a result?

I gave the copyright back to pkgconf, as the comment and description primarily come from the pkgconf project.

Updated the copyright, license text, and added the licenses = [ "pkgconf" ] as per ngie's advice; thank you!

In D56591#1295928, @khorben wrote:

The latest update (with WARNS?=3) built correctly with clang; I am still testing with GCC 14.