distributeworld is used to generate install media, so it makes no
sense to check the host database since the install media can be
generated from any box, regardless of the version of FreeBSD it's
running.
Details
Details
Diff Detail
Diff Detail
- Repository
- rS FreeBSD src repository - subversion
- Lint
Lint Not Applicable - Unit
Tests Not Applicable
Event Timeline
Comment Actions
I'm not sure what led to putting these changes where they're at in the makefile, it seems a bit unrelated to the things immediately surrounding it. It should be noted that for this change to work properly, the new lines must appear before line 833 (pre-patched, or 838 with this patch included). It might make more sense to move this change to just before line 833; that would make it a lot more clear what the real effect of this change is.
| Makefile.inc1 | ||
|---|---|---|
| 377 ↗ | (On Diff #46018) | This comment focuses on the current pain and glosses over most of what DB_FROM_SRC actually implies. A comment that more directly indicates what's going on might be something like # When generating install media, do not allow user and group information # from the build host to affect the contents of the distribution. |
Comment Actions
Hrm, I didn't intend for my previous comments to include the "request changes" flag, don't consider my comments to be blocking, I was just throwing them out there to see what other people think.
Comment Actions
Change the wording of the comment and put it just before the first
usage of DB_FROM_SRC.
Comment Actions
I'm 'accepting' this revision in the sense that I think this is the right way to achieve unconditional use of users and groups from the source being installed, for the distributeworld target. I can't say whether doing so is a good idea or not in general, because I don't know much about the distributeworld target.
If distributeworld is truly used only by the freebsd project to generate images for distribution media, then it probably makes sense. If the same target is used by end users to generate local images for site-wide distribution, then I can envision theoretical scenarios where this would be the wrong thing to do. For example, there could be a site-wide user/group database such as ldap, and that could theoretically differ in terms of name<->idnumber associations even for the 'standard' ids < 1000. A user doing something like that would see this as a regression, one that cannot be undone with local command line args or env vars. This might seem like a far-fetched scenario, but I've heard that the freebsd cluster machines do something like this.
In other words, maybe it would be good to get re@ and/or clusteradm@ to eyeball this too.
Comment Actions
So it seems there are still issues even after this fix, now I'm seeing:
sh /usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/release/scripts/mm-mtree.sh -m /usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/release/.. -F "TARGET_ARCH=amd64 TARGET=amd64 " -D "/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/release/dist/base"
*** Creating the temporary root environment in /var/tmp/temproot.GeKDEEWO
*** /var/tmp/temproot.GeKDEEWO ready for use
*** Creating and populating directory structure in /var/tmp/temproot.GeKDEEWO
cd /usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/etc; MACHINE_ARCH=amd64 MACHINE=amd64 CPUTYPE= CC="cc -target x86_64-unknown-freebsd12.0 --sysroot=/usr/home/osstest/build.125801.build-amd64-freebsd/obj/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/amd64.amd64/tmp -B/usr/home/osstest/build.125801.build-amd64-freebsd/obj/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/amd64.amd64/tmp/usr/bin" CXX="c++ -target x86_64-unknown-freebsd12.0 --sysroot=/usr/home/osstest/build.125801.build-amd64-freebsd/obj/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/amd64.amd64/tmp -B/usr/home/osstest/build.125801.build-amd64-freebsd/obj/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/amd64.amd64/tmp/usr/bin" CPP="cpp -target x86_64-unknown-freebsd12.0 --sysroot=/usr/home/osstest/build.125801.build-amd64-freebsd/obj/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/amd64.amd64/tmp -B/usr/home/osstest/build.125801.build-amd64-freebsd/obj/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/amd64.amd64/tmp/usr/bin" AS="as" AR="ar" LD="ld" LLVM_LINK="" NM=nm OBJCOPY="objcopy" RANLIB=ranlib STRINGS= SIZE="size" PATH=/usr/home/osstest/build.125801.build-amd64-freebsd/obj/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/amd64.amd64/tmp/legacy/usr/sbin:/usr/home/osstest/build.125801.build-amd64-freebsd/obj/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/amd64.amd64/tmp/legacy/usr/bin:/usr/home/osstest/build.125801.build-amd64-freebsd/obj/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/amd64.amd64/tmp/legacy/bin:/usr/home/osstest/build.125801.build-amd64-freebsd/obj/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/amd64.amd64/tmp/usr/sbin:/usr/home/osstest/build.125801.build-amd64-freebsd/obj/usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/amd64.amd64/tmp/usr/bin:/sbin:/bin:/usr/sbin:/usr/bin make METALOG= distrib-dirs
for file in /usr/share/doc/usd/10.exref /usr/share/doc/usd/11.edit /usr/share/doc/usd/12.vi /usr/share/doc/usd/13.viref; do if [ -f /usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/release/dist/base/${file} ]; then rm -f /usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/release/dist/base/${file}; fi; done
mtree -deU -i -f /usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/etc/mtree/BSD.root.dist -p /usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/release/dist/base/
mtree -deU -i -f /usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/etc/mtree/BSD.var.dist -p /usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/release/dist/base/var
mtree: unknown user `ntpd'
mtree: failed at line 49 of the specification
*** Error code 1
Stop.
make[3]: stopped in /usr/home/osstest/build.125801.build-amd64-freebsd/freebsd/etc
*** Error code 1
Stop.
make[2]: stopped in /usr/home/osstest/build.125801.build-amd64-freebsd/freebsd
*** Error code 1This is part of the 'ftp' target in the release directory that runs the release/scripts/mm-mtree.sh. I see that there are snapshots still being created without issues, so I wonder what runes are used inside of the cluster to generate the install media. Is something different from 'make -C release ftp' used to generate the install sets?
Comment Actions
Added it to what? IMO, it's a really bad idea to paper over this mess, it just means we have to suffer all this pain again next time a user is added to the base system.
Comment Actions
Sorry for the incomplete statement. Added to the build host so 'installworld' to the target chroot directories did not error.
I could have just upgraded the build host, but chose to apply a band-aid for this one case.
Comment Actions
Either one of those things is exactly what I mean by papering over the problem. Upgrading the build host is a way of saying that building images for a given version is only supported if the build host is running the version of the image you're trying to build. Is that a documented requirement of the release build process?
Comment Actions
No. In fact, it extremely rare for me to upgrade the builder during a release cycle, for example.
That said, I have run into this in the past, and had to use a similar band-aid.
Comment Actions
Upgrading the build host is a way of saying that building images for a given version is only supported if the build host is running the version of the image you're trying to build.
Indeed, it may be the case that building on an older host often causes no trouble (i.e., because we don't frequently add new user IDs), but there's currently an implicit requirement. There are a few of these issues and they've existed for ages. We can address them as they're encountered, but we also ought to take a holistic look at the implicit dependencies.