mtree: own bhyve run dir by 'vmm' group

[novel]

Summary

vmm(4) provides write access to the /dev/vmmctl device file
for the members of the 'vmm' group. Thus, non-root members of this
group can create VMs. However, bhyve(8) fails in this case
because it has no permission to create a socket
file in /var/run/bhyve as it's owned by root:wheel.

Fix that by changing this directory's group to 'vmm' and making
it writable by that group.

[ivy]

wouldn't this allow non-root users, who are in the vmm group, to interfere with the VMs of other users (e.g., root) by pre-creating files/sockets in that directory? i wonder if it's better to say that non-root users should put their control sockets in a different directory.

[markj]

in this case

You might further specify which case this is?

In D57372#1314420, @ivy wrote:

wouldn't this allow non-root users, who are in the vmm group, to interfere with the VMs of other users (e.g., root) by pre-creating files/sockets in that directory? i wonder if it's better to say that non-root users should put their control sockets in a different directory.

I'm not sure offhand which files go in this directory, but in general bhyve should be unlinking preexisting files owned by the current VM. It might be sufficient to give each VM a subdirectory under /var/run/bhyve, with mode 0744. We might further split root-owned files into a separate /var/run/bhyve-priv.

[novel]

In D57372#1314650, @markj wrote:

in this case

You might further specify which case this is?

In case of running bhyve as a non-root user.

Specifically:

$ sudo -u bhyve -g vmm \
       bhyve -M -c 2 -m 4096 -u -H -P -s 0:0,hostbridge -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
                                                               -s 2:0,xhci,tablet -s 1:0,lpc \
                                                               -s 3:0,ahci,cd:/data/isos/alpine-standard-3.23.4-x86_64.iso -s 6:0,fbuf,tcp=127.0.0.1:5901 \
                                                               alpine
fbuf frame buffer base: 0x327758a00000 [sz 33554432]
Failed to bind socket "/var/run/bhyve/alpine": Permission denied

bhyve: Failed to start checkpoint thread
$

[markj]

In D57372#1314709, @novel wrote:

In D57372#1314650, @markj wrote:

in this case

You might further specify which case this is?

In case of running bhyve as a non-root user.

I run bhyve as a non-root user many times every day, and I don't see this.

Specifically:

$ sudo -u bhyve -g vmm \
       bhyve -M -c 2 -m 4096 -u -H -P -s 0:0,hostbridge -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
                                                               -s 2:0,xhci,tablet -s 1:0,lpc \
                                                               -s 3:0,ahci,cd:/data/isos/alpine-standard-3.23.4-x86_64.iso -s 6:0,fbuf,tcp=127.0.0.1:5901 \
                                                               alpine
fbuf frame buffer base: 0x327758a00000 [sz 33554432]
Failed to bind socket "/var/run/bhyve/alpine": Permission denied

bhyve: Failed to start checkpoint thread
$

Ah, this is with snapshot support enabled I guess?

[novel]

In D57372#1314712, @markj wrote:

In D57372#1314709, @novel wrote:

In D57372#1314650, @markj wrote:

in this case

You might further specify which case this is?

In case of running bhyve as a non-root user.

I run bhyve as a non-root user many times every day, and I don't see this.

Specifically:

$ sudo -u bhyve -g vmm \
       bhyve -M -c 2 -m 4096 -u -H -P -s 0:0,hostbridge -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
                                                               -s 2:0,xhci,tablet -s 1:0,lpc \
                                                               -s 3:0,ahci,cd:/data/isos/alpine-standard-3.23.4-x86_64.iso -s 6:0,fbuf,tcp=127.0.0.1:5901 \
                                                               alpine
fbuf frame buffer base: 0x327758a00000 [sz 33554432]
Failed to bind socket "/var/run/bhyve/alpine": Permission denied

bhyve: Failed to start checkpoint thread
$

Ah, this is with snapshot support enabled I guess?

Yes, it's with snapshot enabled, sorry, should have mentioned that.

[ivy]

I'm not sure offhand which files go in this directory, but in general bhyve should be unlinking preexisting files owned by the current VM.

well, perhaps that's so; my main concern here is that if i give a user access to run bhyve VMs, that doesn't imply i want them to have access to previously-restricted directories used by root. those are two separate things that should be configured separately, imo.

if this is specific to snapshots, perhaps a new bhyve argument could be used to specify the socket directory it uses for that.

It might be sufficient to give each VM a subdirectory under /var/run/bhyve, with mode 0744.

how would those directories be created?

[markj]

In D57372#1314941, @ivy wrote:

I'm not sure offhand which files go in this directory, but in general bhyve should be unlinking preexisting files owned by the current VM.

well, perhaps that's so; my main concern here is that if i give a user access to run bhyve VMs, that doesn't imply i want them to have access to previously-restricted directories used by root. those are two separate things that should be configured separately, imo.

if this is specific to snapshots, perhaps a new bhyve argument could be used to specify the socket directory it uses for that.

Yes, that seems better.

It might be sufficient to give each VM a subdirectory under /var/run/bhyve, with mode 0744.

how would those directories be created?

/var/run/bhyve would be writeable by the vmm group, so bhyve can just create them.

[novel]

In D57372#1315046, @markj wrote:

In D57372#1314941, @ivy wrote:

if this is specific to snapshots, perhaps a new bhyve argument could be used to specify the socket directory it uses for that.

Yes, that seems better.

Drafted a DR for this approach: https://reviews.freebsd.org/D57494