release: Add 2 image layers for almost all base.txz packages excluding toolchain, and one with toolchain

[dch]

Summary

Contains almost everything of use in a container except:

• *lib32
• *dev
• *dbg
• *src
• anything expecting hardware such as device config tools

[dch]

David Chisnall has pointed out the usefulness of having a "kitchen sink" container image.
He's currently doing an article for the FreeBSD Journal, and it would be handy if we can
build a sink for him to put his toys in...

Doug: is this ok for you?
Colin: this would add ~ 110MiB / architecture to our builds, using amd64 + arm64 as reference

I should have a thermite patch soon.

[theraven]

Thanks. It would also be nice to have an intermediate layer that is all of this except the toolchain, since the toolchain is very large but not needed for a lot of these use cases.

Off-topic question: I note that pkg-bootstrap is still in the packages for pkgbase. Are we really going to ship pkgbase without pkg?

release/tools/oci-image-full.conf

16 ↗

(On Diff #158932)

Bikeshedding, but bhyve is probably a fairly niche use case for containers, so could lose it here.

27 ↗

(On Diff #158932)

Aren't these tools for directly accessing devices?

33 ↗

(On Diff #158932)

Duplicated line?

[dfr]

I don't have strong opinions on the package list other than to note that clang is fairly useless without the *dev packages. As David notes, we could probably make a case for two images here, 'kitchen sink without toolchain' and another one containing toolchain etc. using kitchen sink as base.

Regardless of image contents, we need to make sure adding large image(s) won't break the download.freebsd.org mirrors.

[dch]

I'll also remove:

• ntp
• ppp
• rdma
• syscons-data
• vt-data
• wpa

I'll split this into a compiler & full image as suggested
What should the dependency be? compiler needs full, or other
way around?

We can bikeshed the naming, but first I'll get it working.

release/tools/oci-image-full.conf

27 ↗

(On Diff #158932)

I don't know all of these, so if you see one (like ctl!) let me know I'll remove it.

[dch]

update after initial feedback, remove a few more hardware-y packages

split into 2 images still to come.

[dch]

woops actually nuke bhyve this time

[dfr]

In D51471#1175361, @dch wrote:

I'll also remove:

• ntp
• ppp
• rdma
• syscons-data
• vt-data
• wpa

I'll split this into a compiler & full image as suggested
What should the dependency be? compiler needs full, or other
way around?

We can bikeshed the naming, but first I'll get it working.

I think the ordering should be static, dynamic, runtime, kitchensink, toolchain which would allow access to the larger functionality of kitchensink without the weight of toolchain bits. Not sure about names but probable not "kitchensink".

[dch]

untested, proposed split. clang / lld / lldb seem intertwined.

I'm still waiting on builders to complete.

[theraven]

This split looks good to me. I'd also be happy if these were just pushed to public container registries, rather than to mirrors.

[dch]

• https://pkg.skunkwerks.at/FreeBSD:15:amd64/oci-images/
• https://pkg.skunkwerks.at/FreeBSD:15:aarch64/oci-images/

which yields:

...
# podman load -i=/usr/obj/usr/src/amd64.amd64/release/container-image-base.txz
# podman load -i=/usr/obj/usr/src/amd64.amd64/release/container-image-toolchain.txz
...
# podman image ls -a
REPOSITORY                       TAG                 IMAGE ID      CREATED        SIZE
localhost/freebsd-static         15.0-CURRENT-amd64  b88071fed72c  4 minutes ago  3.55 MB
localhost/freebsd-dynamic        15.0-CURRENT-amd64  16dda0e57402  4 minutes ago  13.8 MB
localhost/freebsd-runtime        15.0-CURRENT-amd64  2a2c470af4c8  4 minutes ago  32.7 MB
localhost/freebsd-base           15.0-CURRENT-amd64  34c5cd3c2e57  3 minutes ago  174 MB
localhost/freebsd-toolchain      15.0-CURRENT-amd64  18b5ec58d6e6  3 minutes ago  367 MB

how does that look?

A functional test will have to wait til later.

[dch]

I like the idea of just publishing these bigger image to mirrors.

$ l (fd -tf container-image /usr/obj) |grep -v meta
-rw-r--r--  1 root operator   40M Jul 23 11:47 /usr/obj/usr/src/amd64.amd64/release/container-image-base.txz
-rw-r--r--  1 root operator  4.6M Jul 23 11:46 /usr/obj/usr/src/amd64.amd64/release/container-image-dynamic.txz
-rw-r--r--  1 root operator   11M Jul 23 11:46 /usr/obj/usr/src/amd64.amd64/release/container-image-runtime.txz
-rw-r--r--  1 root operator  738K Jul 23 11:46 /usr/obj/usr/src/amd64.amd64/release/container-image-static.txz
-rw-r--r--  1 root operator  103M Jul 23 11:47 /usr/obj/usr/src/amd64.amd64/release/container-image-toolchain.txz
-rw-r--r--  1 root operator   38M Jul 23 11:47 /usr/obj/usr/src/arm64.aarch64/release/container-image-base.txz
-rw-r--r--  1 root operator  4.3M Jul 23 11:46 /usr/obj/usr/src/arm64.aarch64/release/container-image-dynamic.txz
-rw-r--r--  1 root operator  9.9M Jul 23 11:46 /usr/obj/usr/src/arm64.aarch64/release/container-image-runtime.txz
-rw-r--r--  1 root operator  738K Jul 23 11:46 /usr/obj/usr/src/arm64.aarch64/release/container-image-static.txz
-rw-r--r--  1 root operator   96M Jul 23 11:47 /usr/obj/usr/src/arm64.aarch64/release/container-image-toolchain.txz

[emaste]

I'm not sure "base" is the best name -- to me it has some suggestion of being a small image you can add other things to. Can't come up with a better name that implies "everything but the toolchain" though.

[dch]

rename 3-headed dog at gates of hell

[emaste]

thinking some more, base is probably OK because it mirrors the base system, but should that then be the one including the toolchain? I guess it's fine like this

[dfr]

In D51471#1176700, @emaste wrote:

thinking some more, base is probably OK because it mirrors the base system, but should that then be the one including the toolchain? I guess it's fine like this

The toolchain is bigger than everything else put together so it seemed to make sense to have an image which has all the toys except the compiler/includes/libs. I don't have a good idea for what it should be called. Base is ok but isn't the same content as base.txz - not sure if that matters.

[dch]

picking this up post holidays, it seems:

• we need to add a +compiler flavour - before? or after? "full" ?
• is toolchain a good enough name, or compiler ?
• is my package list above complete, correct & usable? (clang, ctf-tools, lld, lldb, toolchain)

Which image (full or compiler) has to be the last one? Do we need to ship some portion,
like runtime libraries, from compiler, for the packages in full to work?

• we should try to have the naming match up with FreeBSD-base (or whatever the pkgbase nomenclature thread on ML comes up with)

[dfr]

I don't have strong opinions on the package list other than to note that clang is fairly useless without the *dev packages. As David notes, we could probably make a case for two images here, 'kitchen sink without toolchain' and another one containing toolchain etc. using kitchen sink as base.

Regardless of image contents, we need to make sure adding large image(s) won't break the download.freebsd.org mirrors.

In D51471#1190340, @dch wrote:

picking this up post holidays, it seems:

• we need to add a +compiler flavour - before? or after? "full" ?
• is toolchain a good enough name, or compiler ?
• is my package list above complete, correct & usable? (clang, ctf-tools, lld, lldb, toolchain)

Which image (full or compiler) has to be the last one? Do we need to ship some portion,
like runtime libraries, from compiler, for the packages in full to work?

• we should try to have the naming match up with FreeBSD-base (or whatever the pkgbase nomenclature thread on ML comes up with)

Firstly, we should figure out whether it is worthwhile to have two images, i.e one with kitchen sink except toolchain and one with added toolchain. The argument for is that toolchain packages are large and against is the added complexity and the problem of choosing names.

If we choose to have the two images, I think "toolchain" is the right name since this term is already used to describe the tools and support files needed for building software. The toolchain image will be the last one since it is a strict superset of "full" and therefore will be built with "full" as its base image.

[dch]

pick some arbitrary names - no-compiler and full.

[dch]

actually use what dfr@ proposed in his comment.

[dch]

I'm very confused about pkgbase -dev packages.

For example, sshd:

$ ldd /usr/sbin/sshd
/usr/sbin/sshd:
        libprivatessh.so.5 => /usr/lib/libprivatessh.so.5 (0x32d96f455000)
...
$ pkg which /usr/lib/libprivatessh.so
/usr/lib/libprivatessh.so was installed by package FreeBSD-ssh-dev-15.0.1.20250906130029

But in practice, sshd doesn't seem to depend on this - I removed the FreeBSD-ssh-dev
package, restarted sshd and can connect -- neither client nor server were broken.

@dfr according to what you say above, all the -dev packages should go in toolchain package.
But if we actually need them in general for running e.g. ssh or whatever, then they need
to be pulled in elsewhere.

TLDR do I add all of these -dev and -man packages to toolchain or full?

[dfr]

In D51471#1199744, @dch wrote:

I'm very confused about pkgbase -dev packages.

For example, sshd:

$ ldd /usr/sbin/sshd
/usr/sbin/sshd:
        libprivatessh.so.5 => /usr/lib/libprivatessh.so.5 (0x32d96f455000)
...
$ pkg which /usr/lib/libprivatessh.so
/usr/lib/libprivatessh.so was installed by package FreeBSD-ssh-dev-15.0.1.20250906130029

But in practice, sshd doesn't seem to depend on this - I removed the FreeBSD-ssh-dev
package, restarted sshd and can connect -- neither client nor server were broken.

@dfr according to what you say above, all the -dev packages should go in toolchain package.
But if we actually need them in general for running e.g. ssh or whatever, then they need
to be pulled in elsewhere.

TLDR do I add all of these -dev and -man packages to toolchain or full?

If anywhere, toolchain. I'm in the middle of building your latest patch so that I do some simple testing with the toolchain image to see if it needs extra bits.

[dfr]

release/Makefile.oci

22

release/tools/oci-image-toolchain.conf

15

At a bare minimum, we need FreeBSD-clibs-dev and FreeBSD-runtime-dev to allow building a C hello world program. Ideally, this image should be able to build anything which uses include files and libraries from the base system so a better option would be FreeBSD-*-dev.

[dfr]

My version of the diff which adds all the -dev pacakges {F128913448}

[emaste]

@dch the .so file is a symlink to the .so.5 -- the symlink is only needed for development, the shared object itself is required by the built binary.

[dch]

Jan's list of packages he excludes

# small
*-dbg
*-dev
*-lib32
FreeBSD-bootloader
FreeBSD-bsnmp
FreeBSD-clang
FreeBSD-cxgbe-tools
FreeBSD-dtrace
FreeBSD-kernel-*
FreeBSD-lld
FreeBSD-lldb
FreeBSD-mlx-tools
FreeBSD-rescue
FreeBSD-src*
FreeBSD-tests

Tiny removes man as well

# tiny
*-man

[dch]

I could not find a good word that means "not-quite-full excluding toolchain".

• Roll in dfr's changes
• Switch from "full" -> "jail" for image name of the 'base-no-toolchain' flavour

# podman run -it --dns 9.9.9.9  localhost/freebsd-toolchain:15.0-ALPHA2-amd64
# cat mnml.c
#include <stdio.h>
int main() {
   // printf() displays the string inside quotation
   printf("Hello, World!");
   return 0;
}
# cc mnml.c
# ls
a.out   mnml.c
# ./a.out
Hello, World!#

[emaste]

This looks fine to me. It can switch to pkg set metapackages but I'd go with this for MFC, and then migrate to metapackages.

[dch]

Use new, new, notoolchain name. YOLO!

[imp]

release/tools/oci-image-toolchain.conf

5

Should this be toolchain instead of notoolchain?

[jrtc27]

release/tools/oci-image-toolchain.conf

5

toolchain is the image this file is building, which is notoolchain plus the packages below

[imp]

release/tools/oci-image-toolchain.conf

5

Oh. Missed BASE. never mind

[jamie_catflap.org]

In D51471#1176700, @emaste wrote:

thinking some more, base is probably OK because it mirrors the base system, but should that then be the one including the toolchain? I guess it's fine like this

Obviously, the full version with toolchain should be called "kitchensink", and the one without the toolchain called "everything-but-the-kitchen-sink"

/coat

[theraven]

This passed beyond the point where I could add any meaningful feedback a while ago, but I wanted to just chime in briefly to thank everyone who worked on it. I'm really looking forward to playing with these images!