release: Improper cleanup while removing pkgcache

[bofh]

Summary

During the VM image building procedures devfs is mounted and unmounted couple of times specially when installing extra pkgs. However during the last stage in the vm_extra_pkg_rmcache stage the devfs is neither mounted nor unmounted. However when pkg cleanup is called somehow /dev/null is mounted in the file system. And despite being checked and called through the cleanup function this is actually not removed or unmounted. I am not sure whether if this is a problem with pkg or the script itself. However I believe this actually does not happen when re@ is building their images in a clean chrooted environment. This is most probably due to the fact that their is a chroot environment(Final Build Staged) inside another(CHROOTDIR) chroot environment.

In the following testplan the detailed information on how to produce this problem is mentioned. I have also faced a similar problem while I was working with the CI scripts for the developers which is still under review here: https://reviews.freebsd.org/D43786

In this occasion I have entirely override the function with this same patch. Otherwise I get the following error:

pkg: Cannot open dev/null

And also the installed pkgs are not cleaned from /var/cache/pkg in the staged dir.

Test Plan

To reproduce this error:

# cd <SRC>
# make -j64 buildworld
# make -j64 buildkernel
# cd release
# make -DWITH_CLOUDWARE CLOUDWARE_TYPES=VAGRANT cloudware-release
# ll /usr/obj/root/Projects/FreeBSD/src/amd64.amd64/release/cw-vagrant-virtualbox-ufs/dev
total 1
-rw-r--r--  1 root wheel uarch   53B Mar  3 17:29 null

[bapt]

pkg -c chroot clean -y -a

Will do the same without the need of mounting /dev other bonus, no emulator is needed

[bofh]

In D44198#1008346, @bapt wrote:

pkg -c chroot clean -y -a

Will do the same without the need of mounting /dev other bonus, no emulator is needed

As per the instructions I have tried with the following:

vm_extra_pkg_rmcache() {
        ls -la ${DESTDIR}/dev
        if [ -e ${DESTDIR}/usr/local/sbin/pkg ]; then
                env ASSUME_ALWAYS_YES=yes \
                        /usr/local/sbin/pkg -c ${DESTDIR} clean -y -a
        fi
        ls -la ${DESTDIR}/dev

        return 0
}

And the problem still persists. And I can see that devfs was mounted in between this pkg clean and was not properly unmounted. /dev/null still persists after the pkg clean. I have also tried with the base /usr/sbin/pkg resulting in same situation.