release: Rework oci image handling using metalogs

[dfr]

Summary

This uses the metalog to keep track of exactly what is being added in
each image and makes it possible to share the layers instead of
squashing them together in each image.

This is close to being able to support unprivileged building of OCI
images, currently blocked by pkg requiring root privileges for
installing packages, even if the metalog is enabled.

MFC after: 1 day

[imp]

You might want to look at ~imp/armv7-pkgbase-14.3-exp.sh which generates a bootable image w/o root for armv7 with the correct perms using pkgbase + pkg to create the system.

I'm sure it misses many .conf files, etc that are installed / generated (*pwd.db I found already). I'm surprised I didn't add passwd though.

I wrote the above as PoC for making nanobsd be able to generate images like this, stealing bits and pieces from different places (including your OCI scripts, which acted as one of the kicks in the butt to get my work moving again).

-o INSTALL_AS_USER=yes

was what I needed, though I also needed a lot of extra 'ignore os release' stuff

-o OSVERSION=1403000 \
-o IGNORE_OSVERSION=yes \

for this to build on 16...

And yes, I know the script is an absolute mess... It works and I'll be refactoring it into nanobsd today.

The cool thing is I can build a new image in about 20s and boot it in qemu in another 30...

[dfr]

In D52617#1201983, @imp wrote:

You might want to look at ~imp/armv7-pkgbase-14.3-exp.sh which generates a bootable image w/o root for armv7 with the correct perms using pkgbase + pkg to create the system.

I'm sure it misses many .conf files, etc that are installed / generated (*pwd.db I found already). I'm surprised I didn't add passwd though.

I wrote the above as PoC for making nanobsd be able to generate images like this, stealing bits and pieces from different places (including your OCI scripts, which acted as one of the kicks in the butt to get my work moving again).

-o INSTALL_AS_USER=yes

I think this is the secret option which should allow no-root for OCI image builds. Thanks Warner!

[imp]

release/tools/oci-image-static.conf

19

Why do you need to run the mtrees here? I've not noticed the need and am wondering what I'm missing.

22

I've also noticed something in the pkgbase packages does this bit too... and I ran into the same need for adjusting METALOG

30

My image also have termcap.small installed in them... I know from ye-olden-times of hacking on nanobsd this was needed (In fact, I created termcap.small), but I'm curious why you need it now.

[dfr]

release/tools/oci-image-static.conf

19

I'm struggling to remember exactly why I started doing this. I think it was to ensure that all the directories had the right permissions or something. Possibly not needed at all.

22

I'm pretty sure I copied it from pkgbase

30

It is mainly to keep the images small. The static image only needs passwd, certs, tz and termcap. Installing the full termcap isn't needed and would add maybe 10% to the size.