Calling sync did not cause the changes in root filesystem to be written
to disk. Changing kern_sync function to call VFS_SYNC with MNT_WAIT
option causes the syscall to wait for I/O operations and fixes
The bug could be easily reproduced by writing to a file in root
filesystem, calling sync and then forcefully powering off the machine
(e.g. by pulling out the power cable). After rebooting the newly
written file was absent.
During testing this bug was noticed to only happen on UFS partitions
mounted as rootfs. ZFS was not affected and UFS filesystems mounted
in different directories also did not seem to be affected. Powering
off normally did sync the data properly.
The bug was observed on aarch64 and amd64 but probably also affects
After some further testing it seems that the problem was introduced
somewhere between 8.0 and 9.0, as sync works properly in 8.0
but fails in 9.0.