release: Rework oci image handling using metalogs
Needs ReviewPublic
Actions

Authored by dfr on Sep 19 2025, 1:56 PM.

Details

Reviewers

dch
cperciva

Summary

This uses the metalog to keep track of exactly what is being added in
each image and makes it possible to share the layers instead of
squashing them together in each image.

This is close to being able to support unprivileged building of OCI
images, currently blocked by pkg requiring root privileges for
installing packages, even if the metalog is enabled.

MFC after: 1 day

Diff Detail

Repository

rG FreeBSD src repository

Lint

Lint Skipped

Unit

Tests Skipped

Build Status

Buildable 67165
Build 64048: arc lint + arc unit

Event Timeline

dfr created this revision.Sep 19 2025, 1:56 PM

Herald added subscribers: jrtc27, imp. · View Herald TranscriptSep 19 2025, 1:56 PM

dfr requested review of this revision.Sep 19 2025, 1:56 PM

Harbormaster completed remote builds in B67165: Diff 162413.Sep 19 2025, 1:56 PM

You might want to look at ~imp/armv7-pkgbase-14.3-exp.sh which generates a bootable image w/o root for armv7 with the correct perms using pkgbase + pkg to create the system.

I'm sure it misses many .conf files, etc that are installed / generated (*pwd.db I found already). I'm surprised I didn't add passwd though.

I wrote the above as PoC for making nanobsd be able to generate images like this, stealing bits and pieces from different places (including your OCI scripts, which acted as one of the kicks in the butt to get my work moving again).

-o INSTALL_AS_USER=yes

was what I needed, though I also needed a lot of extra 'ignore os release' stuff

-o OSVERSION=1403000 \
-o IGNORE_OSVERSION=yes \

for this to build on 16...

And yes, I know the script is an absolute mess... It works and I'll be refactoring it into nanobsd today.

The cool thing is I can build a new image in about 20s and boot it in qemu in another 30...

In D52617#1201983, @imp wrote:
You might want to look at ~imp/armv7-pkgbase-14.3-exp.sh which generates a bootable image w/o root for armv7 with the correct perms using pkgbase + pkg to create the system.

I'm sure it misses many .conf files, etc that are installed / generated (*pwd.db I found already). I'm surprised I didn't add passwd though.

I wrote the above as PoC for making nanobsd be able to generate images like this, stealing bits and pieces from different places (including your OCI scripts, which acted as one of the kicks in the butt to get my work moving again).
-o INSTALL_AS_USER=yes

I think this is the secret option which should allow no-root for OCI image builds. Thanks Warner!

imp added inline comments.Sep 19 2025, 2:50 PM

release/tools/oci-image-static.conf
19	Why do you need to run the mtrees here? I've not noticed the need and am wondering what I'm missing.
22	I've also noticed something in the pkgbase packages does this bit too... and I ran into the same need for adjusting METALOG
30	My image also have termcap.small installed in them... I know from ye-olden-times of hacking on nanobsd this was needed (In fact, I created termcap.small), but I'm curious why you need it now.

dfr added inline comments.Sep 19 2025, 3:05 PM

release/tools/oci-image-static.conf
19	I'm struggling to remember exactly why I started doing this. I think it was to ensure that all the directories had the right permissions or something. Possibly not needed at all.
22	I'm pretty sure I copied it from pkgbase
30	It is mainly to keep the images small. The static image only needs passwd, certs, tz and termcap. Installing the full termcap isn't needed and would add maybe 10% to the size.