Change Details

This is the first(1/6) installment of scripts of bringing CI for developers in the base tree. This patch brings the following changes: - Changes in new `ci` directory: - Create a new directory `ci` where all ci related scripts/configs/Makefile will be kept - Create a `firstboot_ci` rc script which will change configurations or more likely enable/disable relevant tests after the testvm is booted. In reference to our jenkins scripts this replaces most of the replace/append of meta scripts. - Merge the original rc.local script from jenkins scripts - Adds multiple scripts mostly the test selections into `ci/scripts/test/subr' from our jenkins scripts - Adds multiple scripts mostly the run-* into `ci/scripts/test/subr' from our jenkins scripts. These scripts kickoffs the actual testing once the test vm boots into the testing environment. - Changes in existing `release` directory: - Add 3 new vars in `Makefile` - **`WITH_COMPRESSED_CLOUDWARE`:** We will often need to share the same images in between jobs both locally and other CI environments where it is required to move the artifacts from one job to the cache host and download again in another job. This will help reduce the amounts to wait for transferring and also the required space in the cache server as there might be a limit on how much we can cache. - **`WITH_PACKAGESYSTEM`:** Currently we have tried to reproduce a distdir on different hosts so that we can use the same distdir on different jobs but the file flags and modes are hard to reproduce in between different jobs so it's easier to use `packagesystem` from the release system. However those artifacts are only used to create the bootonly, disc1 and dvd image and are not used at all in the VM images. This variable adds the option to use the packagesystem artifacts to create vm-images. - **'OBJDIR':** As we will use `packagesystem` rather than `installworld` so we need to make sure that the `mk-vmimage.sh` and `vmimage.subr' knows about `.OBJDIR`.This variable is used to pass that to the other helper scripts. - Changes in `Makefile.vm` - Add new CLOUDWARE image `CI`. Initially I had a discussion with @manu about using `BASIC-CI` however their console requirements are different to us and hence I have created this new version of CLOUDWARE image. - Refactor the code to accommodate the new variables `WITH_COMPRESSED_CLOUDWARE`and `WITH_PACKAGESYSTEM`. Also utilize `PACKAGESYSTEM_ARGS` to pass the argument of `OBJDIR` to the other helper scripts. - Changes in `scripts/mk-vmimage.sh` - Add getopts arg `P` to accommodate passing the environment variable `WITH_PACKAGESYSTEM` - Add getopts arg `D` to accommodate passing the environment variable `OBJDIR` - Changes in new `tools/ci.conf` - Configuration file for the `CI` CLOUDWARE image. - Changes in `tools/vmimage.subr` - Based on the value of `PACKAGESYSTEM` modify `vm_install_base` to install from `packagesystem` artifacts; otherwise use the traditional `installworld` - During `vm_extra_pkg_rmcache` we need to mount and unmount `devfs` Although mentioned we cannot yet issue the command `make testvm` and generate the images. These changes are trivial before we can go into that state.

This is the first(1/6) installment of scripts of bringing CI for developers in the base tree. This patch brings the following changes: - Changes in new `ci` directory: - Create a new directory `ci` where all ci related scripts/configs/Makefile will be kept - Create a `firstboot_ci` rc script which will change configurations or more likely enable/disable relevant tests after the testvm is booted. In reference to our jenkins scripts this replaces most of the replace/append of meta scripts. - Merge the original rc.local script from jenkins scripts - Adds multiple scripts mostly the test selections into `ci/scripts/test/subr' from our jenkins scripts - Adds multiple scripts mostly the run-* into `ci/scripts/test/subr' from our jenkins scripts. These scripts kickoffs the actual testing once the test vm boots into the testing environment. - Changes in existing `release` directory: - Add 3 new vars in `Makefile` - **`WITH_COMPRESSED_CLOUDWARE`:** We will often need to share the same images in between jobs both locally and other CI environments where it is required to move the artifacts from one job to the cache host and download again in another job. This will help reduce the amounts to wait for transferring and also the required space in the cache server as there might be a limit on how much we can cache. - **`WITH_PACKAGESYSTEM`:** Currently we have tried to reproduce a distdir on different hosts so that we can use the same distdir on different jobs but the file flags and modes are hard to reproduce in between different jobs so it's easier to use `packagesystem` from the release system. However those artifacts are only used to create the bootonly, disc1 and dvd image and are not used at all in the VM images. This variable adds the option to use the packagesystem artifacts to create vm-images. - **`OBJDIR`:** As we will use `packagesystem` rather than `installworld` so we need to make sure that the `mk-vmimage.sh` and `vmimage.subr` knows about `.OBJDIR`.This variable is used to pass that to the other helper scripts. - Changes in `Makefile.vm` - Add new CLOUDWARE image `CI`. Initially I had a discussion with @manu about using `BASIC-CI` however their console requirements are different to us and hence I have created this new version of CLOUDWARE image. - Refactor the code to accommodate the new variables `WITH_COMPRESSED_CLOUDWARE`and `WITH_PACKAGESYSTEM`. Also utilize `PACKAGESYSTEM_ARGS` to pass the argument of `OBJDIR` to the other helper scripts. - Changes in `scripts/mk-vmimage.sh` - Add getopts arg `P` to accommodate passing the environment variable `WITH_PACKAGESYSTEM` - Add getopts arg `D` to accommodate passing the environment variable `OBJDIR` - Changes in new `tools/ci.conf` - Configuration file for the `CI` CLOUDWARE image. - Changes in `tools/vmimage.subr` - Based on the value of `PACKAGESYSTEM` modify `vm_install_base` to install from `packagesystem` artifacts; otherwise use the traditional `installworld` - During `vm_extra_pkg_rmcache` we need to mount and unmount `devfs` Although mentioned we cannot yet issue the command `make testvm` and generate the images. These changes are trivial before we can go into that state.

This is the first(1/6) installment of scripts of bringing CI for developers in the base tree. This patch brings the following changes: - Changes in new `ci` directory: - Create a new directory `ci` where all ci related scripts/configs/Makefile will be kept - Create a `firstboot_ci` rc script which will change configurations or more likely enable/disable relevant tests after the testvm is booted. In reference to our jenkins scripts this replaces most of the replace/append of meta scripts. - Merge the original rc.local script from jenkins scripts - Adds multiple scripts mostly the test selections into `ci/scripts/test/subr' from our jenkins scripts - Adds multiple scripts mostly the run-* into `ci/scripts/test/subr' from our jenkins scripts. These scripts kickoffs the actual testing once the test vm boots into the testing environment. - Changes in existing `release` directory: - Add 3 new vars in `Makefile` - **`WITH_COMPRESSED_CLOUDWARE`:** We will often need to share the same images in between jobs both locally and other CI environments where it is required to move the artifacts from one job to the cache host and download again in another job. This will help reduce the amounts to wait for transferring and also the required space in the cache server as there might be a limit on how much we can cache. - **`WITH_PACKAGESYSTEM`:** Currently we have tried to reproduce a distdir on different hosts so that we can use the same distdir on different jobs but the file flags and modes are hard to reproduce in between different jobs so it's easier to use `packagesystem` from the release system. However those artifacts are only used to create the bootonly, disc1 and dvd image and are not used at all in the VM images. This variable adds the option to use the packagesystem artifacts to create vm-images. - **'`OBJDIR':**`:** As we will use `packagesystem` rather than `installworld` so we need to make sure that the `mk-vmimage.sh` and `vmimage.subr'` knows about `.OBJDIR`.This variable is used to pass that to the other helper scripts. - Changes in `Makefile.vm` - Add new CLOUDWARE image `CI`. Initially I had a discussion with @manu about using `BASIC-CI` however their console requirements are different to us and hence I have created this new version of CLOUDWARE image. - Refactor the code to accommodate the new variables `WITH_COMPRESSED_CLOUDWARE`and `WITH_PACKAGESYSTEM`. Also utilize `PACKAGESYSTEM_ARGS` to pass the argument of `OBJDIR` to the other helper scripts. - Changes in `scripts/mk-vmimage.sh` - Add getopts arg `P` to accommodate passing the environment variable `WITH_PACKAGESYSTEM` - Add getopts arg `D` to accommodate passing the environment variable `OBJDIR` - Changes in new `tools/ci.conf` - Configuration file for the `CI` CLOUDWARE image. - Changes in `tools/vmimage.subr` - Based on the value of `PACKAGESYSTEM` modify `vm_install_base` to install from `packagesystem` artifacts; otherwise use the traditional `installworld` - During `vm_extra_pkg_rmcache` we need to mount and unmount `devfs` Although mentioned we cannot yet issue the command `make testvm` and generate the images. These changes are trivial before we can go into that state.