Bhyve - Capsicum integration

[ionut.mihalache1506_gmail.com]

Summary

I did the necessary changes in order to activate capability mode for bhyve. Capsicum was deactivated in order to use for the snapshot processes to work so I changed the system calls that were problematic during the snapshot process such as open(I replaced it with openat).

Test Plan

Start the guest vm with: bhyve -c 1 -m 256M -H -A -P -s 0:0,hostbridge -s 1:0,lpc -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -s 2:0,virtio-net,tap0 -s 3:0,ahci-hd,main.img -l com1,stdio freebsd_guest

For testing the functionality open two processes at the same time on the guest vm:

• one copy process of a 500MB file
• one ping process

While the processes mentioned above are running:

• suspend the guest with the command: bhyvectl --suspend file.ckp --vm=freebsd_guestand
• restore its state with the command: bhyve -c 1 -m 256M -H -A -P -s 0:0,hostbridge -s 1:0,lpc -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -s 2:0,virtio-net,tap0 -s 3:0,ahci-hd,main.img -l com1,stdio -r file.ckp freebsd_guest.

After the restore command, both processes should finish successfully and the guest can be used without errors.

[emaste]

Makefile.inc1

2756

?

usr.sbin/bhyve/Makefile

123–124

This should just be deleted

usr.sbin/bhyve/bhyverun.c

1128

This should be handled separately (i.e., a separate change to improve the diagnostics independent of the capsicum change)

1264

extra whitespace

usr.sbin/bhyve/snapshot.c

1665

Perhaps simplify the #ifdef logic by just using cdir_fd always? WITHOUT_CAPSICUM should be an unusual special case and there's no need to e.g. optimize away the use of an extra fd in that case.

1686

extra whitespace

[ionut.mihalache1506_gmail.com]

I replaced ucl_parser_add_file with ucl_parser_add_fd in order to provide all the restrictions for the metadata file used during the restore process. I also a part of the changes suggested by emaste.

[rew]

usr.sbin/bhyve/bhyverun.c

1333

What if I'm booting from an md device such as -s 3, virtio-blk,/dev/md0, wouldn't ckp_path == '/dev'?

usr.sbin/bhyve/snapshot.c

1384–1412

Why is this a macro?

1635

I'm not sure it's necessary to drag ckp_path and chn through this layer since it doesn't appear to be used by anything but vm_checkpoint().

1663

As mentioned above, if I'm booting from an md device - this will open '/dev', which is probably not correct. What if I want to have my suspend/checkpoint file written to /home/rew/vm-checkpoint.ckp instead?

[ionut.mihalache1506_gmail.com]

usr.sbin/bhyve/snapshot.c

1384–1412

The macro is created in order to use casper such that 'system.sysctl' can be used in capability mode. When I tried to do something similar in vmmapi it doesn't work even if I build with -DWITH_CASPER so I used this workaround instead.

1663

When the a snapshot is created, capability mode is already active so a simple open call would not work, even for the checkpoint file's parent directory (/home/rew in this case) so I chose to use a path for the parent directory that is known before a snapshot is requested(the path of the first disk in this case), such that I can use openat when the snapshot is created. This solution is not ideal, another one would be to know the path of the checkpoint file from the start but it would still create a limitation regarding the path where the snapshot file is written.