Page MenuHomeFreeBSD

bsdinstall: do pkgbase installations with the "script" command
ClosedPublic

Authored by asomers on Wed, Apr 29, 7:50 PM.
Tags
None
Referenced Files
F157580656: D56717.id177607.diff
Sat, May 23, 1:27 AM
F157541950: D56717.diff
Fri, May 22, 3:16 PM
Unknown Object (File)
Fri, May 22, 5:48 AM
Unknown Object (File)
Tue, May 19, 10:00 AM
Unknown Object (File)
Tue, May 19, 9:28 AM
Unknown Object (File)
Mon, May 18, 1:47 PM
Unknown Object (File)
Mon, May 18, 8:30 AM
Unknown Object (File)
Mon, May 18, 8:24 AM

Details

Summary

"bsdinstall script" will now do a pkgbase installation unless the
script file contains "PKGBASE=no". The system components to install can
be specified in the COMPONENTS variable, and have the same names as
those used in the interactive installer.

MFC: 2 weeks
PR: 290375
Sponsored by: ConnectWise

Test Plan

Ran bsdinstall manually. Also, built a DVD and installed a VM from that image.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Should this not do the same as 9134ed157388f3e34374322a5de06449a031f1ec?

It's not the same thing. That older revision changed the default interactive installation to pkgbase. What my change is is for the non-interactive installations. That is, the installations that the CD will do if you put an /etc/installerconfig file on the CD.

We're trying to avoid making pkgbase a proper user facing noun

We're trying to avoid making pkgbase a proper user facing noun

What do you suggest instead? Is there a different phrase to use, or should I make pkgbase mandatory in bsdinstall and remove mention from the man page?

Should this not do the same as 9134ed157388f3e34374322a5de06449a031f1ec?

It's not the same thing. That older revision changed the default interactive installation to pkgbase. What my change is is for the non-interactive installations. That is, the installations that the CD will do if you put an /etc/installerconfig file on the CD.

My point is, that commit made the default depend on what's on the installation media. Should this change not do the same, rather than make it always pkgbase?

Should this not do the same as 9134ed157388f3e34374322a5de06449a031f1ec?

It's not the same thing. That older revision changed the default interactive installation to pkgbase. What my change is is for the non-interactive installations. That is, the installations that the CD will do if you put an /etc/installerconfig file on the CD.

My point is, that commit made the default depend on what's on the installation media. Should this change not do the same, rather than make it always pkgbase?

Well, it does still depend on the installation media. But it's based on the /etc/installerconfig file. If /etc/installerconfig specifies distsets, but the installation media doesn't include those, then I suppose the installation might be broken. Are you suggesting that a single installerconfig file ought to be usable for either pkgbase or distset-based installation, and which is actually done ought to be based on whether the media includes base packages? I actually tried that first. But it doesn't work, because there's no 1:1 correspondence between distsets and pkgbase components.
So if you want a distset installation, you must specify DISTRIBUTIONS=... and PKGBASE=no in the installerconfig. Otherwise, you must specify COMPONENTS=... .

We're trying to avoid making pkgbase a proper user facing noun

What do you suggest instead? Is there a different phrase to use, or should I make pkgbase mandatory in bsdinstall and remove mention from the man page?

I recently learned that the official term is freebsd-base(7).

However, we (NanoBSD GSoC) are (also?) using the NOPKGBASE nomenclature (double negative is worse). Regardless, we plan to align with the outcome of this review.
We have yet to test this approach of only using sets and report back.

So if you want a distset installation, you must specify DISTRIBUTIONS=... and PKGBASE=no in the installerconfig. Otherwise, you must specify COMPONENTS=... .

Could we not do

  1. If COMPONENTS is set and packages are on the install media, use packages
  2. If DISTRIBUTIONS is set and dist sets are on the install media, use dist sets
  3. Error

So if you want a distset installation, you must specify DISTRIBUTIONS=... and PKGBASE=no in the installerconfig. Otherwise, you must specify COMPONENTS=... .

Could we not do

  1. If COMPONENTS is set and packages are on the install media, use packages
  2. If DISTRIBUTIONS is set and dist sets are on the install media, use dist sets
  3. Error

I don't think we should, because both of those variables are optional. DISTRIBUTIONS, if unset, defaults to "base.txz kernel.txz". COMPONENTS effectively defaults to "minimal pkg kernel". So we can't require one of those two to be set. I suppose we could do:

  1. If COMPONENTS is set and packages are on the install media, use packages
  2. if COMPONENTS is set and packages are not on the install media, error
  3. if DISTRIBUTIONS is set and distsets are on the install media, use those
  4. if DISTRIBUTIONS is set and distsets are not on the install media, error
  5. if packages are on the install media, use those
  6. if distsets are on the install media, use those
  7. error

I recently learned that the official term is freebsd-base(7).

that's not the official term for "pkgbase", it's just the name of the manpage. (it's also not a very good name, but no one could come up with a better suggestion at the time.)

when referring to pkgbase in documentation we usually say something like "base system packages". for a variable name, you could use NO_BASE_PACKAGES, or if you're only installing the base system, NO_PACKAGES.

edit: or use something like METHOD=packages/METHOD=distsets.

usr.sbin/bsdinstall/bsdinstall.8
330

when adding set support to bsdinstall, the consensus was that we should install "base" by default to match what the distset installer does. i think, to avoid confusion, it might be better if all installation methods work the same way.

Actually @emaste we can't do what I just suggested, because either packages or distsets might need to be fetched over the network. It's valid for neither to be on the installation media. So if you don't like the PKGBASE=no variable, then the best proposal I can come up with is:

  • If COMPONENTS is set in /etc/installerconfig, use packages
  • If DISTRIBUTIONS is set, use distsets
  • If both are set, error
  • If neither is set, use packages with the default COMPONENTS.
usr.sbin/bsdinstall/bsdinstall.8
330

That's basically what happens now. If DISTRIBUTIONS is unset, it defaults to "base.txz kernel.txz". Or merely "base.txz" for a jail. But for a pkgbase install, COMPONENTS effectively defaults to "minimal pkg kernel" or "minimal-jail pkg".

  • Remove "pkgbase" from user-facing documentation.
usr.sbin/bsdinstall/bsdinstall.8
330

right, but the default for all other cases (including the interactive pkgbase installer) is to install "base" by default. to avoid confusion, i think it would be better if all installation methods used the same default, and if we want minimal to be the default, we should change that in the interactive pkgbase installer as well. otherwise, i think this is going to cause friction for people who don't care about pkgbase specifically and just want to update their distset config to use pkgbase.

  • For non-interactive installations, default COMPONENTS to "base
usr.sbin/bsdinstall/bsdinstall.8
330

Hmm. Yes I made a mistake. When doing an interactive pkgbase install, "minimal pkg kernel" are the mandatory package sets. But the defaults, as you mentioned, are "base kernel-dbg". I'll change the noninteractive behavior to match.

usr.sbin/bsdinstall/bsdinstall.8
330

Minor nit: I'm not sure how we should wrap this for grep, but I usually trying to avoid breaking up a word which is made of words, like "base system packages". However, it seems useful to have install and script on the same line.

334

I can understand we want to keep it as close as to what it currently is. And now I agree that this should be the way to go.
In the pull request bug report you asked me if I had a different take on it: so, originally my approach was, given that they're all packages, regardless of their repo, a user could define something like:

PARTITIONS=DEFAULT
PACKAGES="FreeBSD-set-base FreeBSD-kernel-generic puppet" # Or PKG_LIST
#METHOD="packages" # This is the default

#!/bin/sh
sysrc ifconfig_DEFAULT=DHCP
sysrc sshd_enable=YES

This gives the user the power to choose a FreeBSD-kernel-minimal, for instance, and an à la carte set (or non-set) of components. We then check that a FreeBSD-kernel-* and (FreeBSD-set-base or FreeBSD-set-minimal) are present.
But in retrospect, it makes more sense to just replace DISTRIBUTIONS with COMPONENTS, and remove .txz for each distribution set from the existing installerconfig file.
Regarding the order of relevance, I do not quite follow:
If neither DISTRIBUTIONS nor COMPONENTS is set, then it defaults to "base.txz kernel.txz" for distribution sets, and "base" (and kernel(s) plural/singular?) for components.
If the user has no METHOD, by default is packages, that is COMPONENTS. I can't see an error in this regard, unless of course, there is a typo.
The only thing I do not agree with, is that if you download the CD (not the DVD, the CD does not contain the distribution sets), the distribution sets should be automatically downloaded from the Internet (assuming there is proper connectivity). The same for packages not included in the CD/DVD or installation media (a scripted netinstall, so to speak).

usr.sbin/bsdinstall/bsdinstall.8
480

I would hope.

Actually @emaste we can't do what I just suggested, because either packages or distsets might need to be fetched over the network.

Oh right, of course.

  • If COMPONENTS is set in /etc/installerconfig, use packages
  • If DISTRIBUTIONS is set, use distsets
  • If both are set, error
  • If neither is set, use packages with the default COMPONENTS.

At a high level this approach sounds reasonable to me. IMO we ought to either
a) use the same component/dist variable and a separate installation type variable, if we have a have a 1:1 mapping (e.g., we could strip .txz if installing pkgbase)
b) determine the installation type based on which variable is set

  • Man page changes requested by @ziaee
  • Eliminate the METHOD variable

@emaste I went with option B since there isn't a 1:1 mapping between COMPONENTS and DISTRIBUTIONS. @jlduran the current Lua code doesn't make it possible to install both base and non-base packages using the same variable. It could maybe happen in the future, but not for now. And it's not the way that the interactive installer works. So an installerconfig that installs non-base packages still needs to look something like this:

export COMPONENTS="base debug lib32"
...
#! /bin/sh
...
/usr/sbin/pkg install -y sysutils/py-salt

i haven't tested this, but the changes as described seem reasonable to me.

for installing extra packages: when we added set support to bsdinstall, we also changed it so that, e.g. when you don't install sshd, bsdinstall doesn't prompt to start sshd. one suggestion was that, instead, if the user selects sshd, bsdinstall should install it in addition to the selected sets. i think, if someone wanted to work on making it possible to install arbitrary packages, it would be worth doing it in a way that works for both the interactive and script installer, which might address both issues.

This revision is now accepted and ready to land.Mon, May 11, 8:53 PM

Yes, thank you for the feedback and thank you for working on this.
I think from the user's perspective, this is much better!

usr.sbin/bsdinstall/bsdinstall.8
669

One quick question: I noticed in your example you used -y:

/usr/sbin/pkg install -y sysutils/py-salt

But the documentation omits -y, since this is a scripted installation, the user can assume ASSUME_ALWAYS_YES=YES is set in the environment?
I was wondering if this holds true?

usr.sbin/bsdinstall/bsdinstall.8
669

In fact, no, it isn't true. If you're referring to the "pkg install puppet" in the man page, I copied that from another section of the same man page. But ASSUME_ALWAYS_YES isn't set, and the "pkg install puppet" prompts for confirmation. I'll add a "-y" there before I commit. Good catch.