Paths

Table of Contentst

M1: CI Scripts for Developers
AbandonedPublic
Actions

Authored by bofh on Feb 27 2023, 11:12 PM.

Details

Reviewers

lwhsu
imp
manu
cperciva
gjb
gnn
kp
emaste
olivier
jrm

Summary

This is the first(1/6) installment of scripts of bringing CI for developers in the base tree. This patch brings the following changes:

Changes in new ci directory:
- Create a new directory ci where all ci related scripts/configs/Makefile will be kept
  - Create a firstboot_ci rc script which will change configurations or more likely enable/disable relevant tests after the testvm is booted. In reference to our jenkins scripts this replaces most of the replace/append of meta scripts.
  - Merge the original rc.local script from jenkins scripts
  - Adds multiple scripts mostly the test selections into `ci/scripts/test/subr' from our jenkins scripts
  - Adds multiple scripts mostly the run-* into `ci/scripts/test/subr' from our jenkins scripts. These scripts kickoffs the actual testing once the test vm boots into the testing environment.

Changes in existing release directory:
- Add 3 new vars in Makefile
  - WITH_COMPRESSED_CLOUDWARE: We will often need to share the same images in between jobs both locally and other CI environments where it is required to move the artifacts from one job to the cache host and download again in another job. This will help reduce the amounts to wait for transferring and also the required space in the cache server as there might be a limit on how much we can cache.
  - WITH_PACKAGESYSTEM: Currently we have tried to reproduce a distdir on different hosts so that we can use the same distdir on different jobs but the file flags and modes are hard to reproduce in between different jobs so it's easier to use packagesystem from the release system. However those artifacts are only used to create the bootonly, disc1 and dvd image and are not used at all in the VM images. This variable adds the option to use the packagesystem artifacts to create vm-images.
  - OBJDIR: As we will use packagesystem rather than installworld so we need to make sure that the mk-vmimage.sh and vmimage.subr knows about .OBJDIR.This variable is used to pass that to the other helper scripts.
- Changes in Makefile.vm
  - Add new CLOUDWARE image CI. Initially I had a discussion with @manu about using BASIC-CI however their console requirements are different to us and hence I have created this new version of CLOUDWARE image.
  - Refactor the code to accommodate the new variables WITH_COMPRESSED_CLOUDWAREand WITH_PACKAGESYSTEM. Also utilize PACKAGESYSTEM_ARGS to pass the argument of OBJDIR to the other helper scripts.
- Changes in scripts/mk-vmimage.sh
  - Add getopts arg P to accommodate passing the environment variable WITH_PACKAGESYSTEM
  - Add getopts arg D to accommodate passing the environment variable OBJDIR
- Changes in new tools/ci.conf
  - Configuration file for the CI CLOUDWARE image.
- Changes in tools/vmimage.subr
  - Based on the value of PACKAGESYSTEM modify vm_install_base to install from packagesystem artifacts; otherwise use the traditional installworld
  - During vm_extra_pkg_rmcache we need to mount and unmount devfs

Although mentioned we cannot yet issue the command make testvm and generate the images. These changes are trivial before we can go into that state.

Test Plan

In the host system we need linux support as we need to install some linux-* packages.

sysrc linux_enable=YES
service linux start

Then we can start build, packagesystem and create the image as following:

JFLAG=$(sysctl -n hw.ncpu)
make -j $JFLAG -DNO_CLEAN buildworld TARGET="amd64" TARGET_ARCH="amd64"
make -j $JFLAG -DNO_CLEAN buildkernel TARGET="amd64" TARGET_ARCH="amd64"

cd /usr/src/release
make clean
make -DNOPORTS -DNOSRC -DNODOC TARGET="amd64" TARGET_ARCH="amd64" packagesystem
make -DNO_PKG -DNOPORTS -DNOSRC -DNODOC -DWITH_COMPRESSED_CLOUDWARE -DWITH_CLOUDWARE -DWITH_PACKAGESYSTEM TARGET="amd64" TARGET_ARCH="amd64" CLOUDWARE="CI" cloudware-release

We should have a file ci.raw.xz in ${.OBJDIR}

Diff Detail

Repository

rG FreeBSD src repository

Lint

Lint Skipped

Unit

Tests Skipped

Event Timeline

bofh created this revision.Feb 27 2023, 11:12 PM

Herald added a reviewer: gnn. · View Herald TranscriptFeb 27 2023, 11:12 PM

bofh requested review of this revision.Feb 27 2023, 11:12 PM

Herald added a reviewer: kp. · View Herald TranscriptFeb 27 2023, 11:12 PM

lwhsu added a reviewer: emaste.Feb 28 2023, 1:20 AM

Does using firstboot_ci guarantee it is the *last* thing in booting?

Context not available.

For any future updates to this review please generate diffs with full context, e.g. git diff -U999999 ... or git show -U999999 ...

Can I share this with some folks that have expressed an interest in all things CI?

In D38815#884335, @imp wrote:

Can I share this with some folks that have expressed an interest in all things CI?

This is of course welcomed. We're seeking for comments and feedback from different CI setups and hope what in the base can be a shared foundation for them.

manu added inline comments.Mar 2 2023, 6:43 AM

release/Makefile.vm
33	I don't think that it's a correct descritpion, yes it's used by the CI at freedesktop and was made for it but it can be used for anything. It's just an image with ssh and serial setup

bofh added a reviewer: olivier.Mar 2 2023, 1:04 PM

In D38815#883644, @lwhsu wrote:

Does using firstboot_ci guarantee it is the *last* thing in booting?

I need to recheck this. I am having a bad time running bhyve on 13.1 with a head guest due to zfs incompatibility.

In D38815#884334, @emaste wrote:

Context not available.

For any future updates to this review please generate diffs with full context, e.g. git diff -U999999 ... or git show -U999999 ...

Noted.

bofh added inline comments.Mar 2 2023, 4:30 PM

release/Makefile.vm
33	Yes. I just had a discussion with @lwhsu on today's CI meeting and will look into merging into one. I am just not sure whether if the additional packcages installed will create a regression for other CI tests. Because the rc scripts by default boots and starts the tests. Can you have a look at the `release/tools/ci.conf` file? I will try to optionize the usage of the rc script so that developers can run different tests of their choice or scripts.

bofh added a reviewer: jrm.Mar 2 2023, 4:31 PM

bofh added inline comments.

release/scripts/mk-vmimage.sh
37	By the by this won't make into the commit. This is just for my help for debugging.
release/tools/ci.conf
29	@manu Here are the list of packages.

emaste added inline comments.Mar 2 2023, 4:52 PM

ci/scripts/test/rc/firstboot_ci
28	Where is `$meta` getting set?
ci/scripts/test/rc/rc.local
11	Would be nice to have a comment explaining these bits/settings
28	odd indentation in this block vs above
release/tools/vmimage.subr
54	strange indendation

@emaste I just figured out man style. Is it alright to use the editor configs?

ci/scripts/test/rc/firstboot_ci
28	Oops. It should be `$metadir`
ci/scripts/test/rc/rc.local
11	@lwhsu Can you advise? There were no comments on why we needed this.
28	Noted. Will fix.

bofh edited the summary of this revision. (Show Details)Mar 2 2023, 5:17 PM

lwhsu added inline comments.Mar 2 2023, 5:31 PM

ci/scripts/test/rc/rc.local
11	This is added for showing (hopefully) useful information on console on panic, along with the backtrace. `reset` is for terminating the bhyve vm, I'm not sure this is applicable for all the environments...
release/scripts/mk-vmimage.sh
3	Just noticed this, I think 2023 can be added.
4	And remove this one :)

guest-patmaddox added a subscriber: guest-patmaddox.Mar 26 2023, 7:13 PM

zlei added a subscriber: zlei.Apr 19 2023, 2:37 PM

The size of this review has made it a little hard to review. Might make sense to make more, small reviews to help the turn-around time.

ci/scripts/test/rc/firstboot_ci
12	Maybe having a quick sentence here that this is for XXXX testing...
28	could this iterate over $metadir and do all the scripts it found?
ci/scripts/test/rc/rc.local
34	can drop mips64 And it might be good to have a Bugzilla bug and reference it here...
ci/scripts/test/subr/disable-notyet-tests.sh
3	This looks like it could be a series of function calls that do this disabling... would be less verbose and error prone to changes as we go through time.
ci/scripts/test/subr/run-kyua-ltp.sh
36	why not append to this file instead of doing the sed re-writing above?
release/tools/vmimage.subr
65	drop mergemaster reference.. and support... it's deprecated so we don't need it anymore, ,right? I know it's there now...

Herald added a subscriber: jrtc27. · View Herald TranscriptMay 17 2023, 3:34 PM

Apologies for my silence on this review. I was a bit ... out when it was created, and did not see it until just today. I will try to get some comments in by the end of the week.

zlei added inline comments.May 18 2023, 8:24 AM

release/tools/ci.conf
80	For current and stable/13, by default `net.add_addr_allfibs` is `0`. Will this change be MFC'd into stable/12 ?

nyan_myuji.xyz added a subscriber: nyan_myuji.xyz.May 25 2023, 8:58 PM

emaste added inline comments.Jun 7 2023, 1:47 AM

ci/scripts/test/rc/firstboot_ci
12	Yes a brief description of the purpose of this script would be useful
ci/scripts/test/rc/rc.local
34	Yes reference to a bug would be good. The issue is that our kernel doesn't know how to signal the shutdown?
38	odd shell style(9)?
ci/scripts/test/subr/disable-disks-tests.sh
3	It seems a bit strange to me that we have all of these tiny per-test files
ci/scripts/test/subr/run-kyua-ltp.sh
10–12	it seems a little strange to me that these dirs aren't created elsewhere.
ci/scripts/test/subr/run-kyua-tcttestsuite.sh
12–13	this seems a little magic, is the list of vtbd# devices described somewhere?
release/Makefile
36–37	It seems like these changes could be submitted separately in advance

delphij added a subscriber: delphij.Jun 26 2023, 4:39 PM

delphij added inline comments.

ci/scripts/test/rc/firstboot_ci
29	(Minor issue) I think these should be installed to the image, instead of referencing /usr/src paths.
ci/scripts/test/subr/run-kyua-ltp.sh
41–46	Maybe use the output from mdconfig instead of hardcoding /dev/md0
ci/scripts/test/subr/run-kyua-tcttestsuite.sh
12	Can we use tmpfs for this? (it appears like we only need it as a scratch space and it's discarded afterward).
ci/scripts/test/subr/run-kyua-zfs.sh
12	Can we use tmpfs for this? (it appears like we only need it as a scratch space and it's discarded afterward).