Add rc(8) script and bhyve_config(5) examples to start and stop bhyve(8) on FreeBSD via service(8)

[hs.freebsd_id.omnilan.de]

Summary

The attached patch contains two bhyve_config(5) examples due to the design of the actual extension,
which is a /etc/rc.d/bhyve script to start and stop VMs utilizing service(8) (and at boot/shutdown).

bhyve_config(5) is a powerful contribution to bhyve(8) which makes it easy to configure VMs to be run on FreeBSD.
Unfortunately I had to start them via 'rc.local' hack which is very inconvenient, especially compared to how we can handle jails with 'service jail'.
Initial goal was to have 'service bhyve' as user-friendly as jail is - it simply does what user expects for the start and stop commands.

rc.conf(5) variables are mostly optional:

• User has to enable auto-start with bhyve_enable="YES".
• bhyve_cfgdir is /etc/bhyve_config.d by default.
• bhyve_startvms is optional and defaults to AUTO, which means that every VM defined by a config file in $bhyve_cfgdir will be started and stopped at boot/shutdown. Start order in AUTO mode is determined from lexographically sorting file names, stop order by parsing (and altering) the lauchstate file (in /var/run/bhyve, which is in mtree/BSD.var.dist). If defined, bhyve_startvms limits starting/stopping VMs to the listed names and strictly defines the order of starting up / shutting down.
• there are currently three cross-referencing variables supported: bhyve_vmdisk_default_pool bhyve_vmdisk_default_dataset and bhyve_vmdisk_default_imagedir They can be referenced in bhyve_config(5) files without the bhyve_ prefix (as %(bhyve_vmdisk_default_imagedir) e.g.)
• bhyve_stop_any is present for people who want a 'killall' feature.
• bhyve_unreversed_stop is superfluous in my opinion, but users might expect such a switch which they know from jail.
• bhyve_start_timeout and bhyve_stop_timeout can be adjusted on demand

Only sequential starting/stopping is currently supported.

If bhyve_startvms is not AUTO and 'autostart' was disabled for a listed VM, a console message is printed. Likewise if user tries to start a already started VM via CLI provided VM name e.g. In default AUTO mode, no informal messages are printed.

bhyveload(8) is supported as well as booting with lpc.bootrom defined (UEFI).

rc.conf(5) man page update is missing currently, I'm not used to mdoc(7), but I'm happy to provide a diff too in case this diff is considered to be a realistic commit candidate.

I'm unsure how the non-bhyve_cfg(5) key/variable 'autostart' is to be documented. For cold-standby VMs e.g. it is important that we can exclude configured machines from being started automatically. Since bhyve_cfg(5) explicitly allows using private variables, I made use of it. But I think mentioning it in bhyve_cfg(5) is wrong.
Same applies to the 'bhyveload_hba' script-private variable.

[evfbsd_yandex.ru]

Using syslog as a journal is more suitable.

Maybe meant "bhyve_conf" or "bhyve_confdir" instead "bhyve_cfgdir"?
"conf" commonly used abbreviation for configuration.

[hs.freebsd_id.omnilan.de]

Thanks, ev, I was unsure about the uncommon "bhyve_cfgdir" but my reason to keep it uncommon was that the config file itself is in 'uncommon' format too in the context of parsing a config file from an rc(8) script.
But I'm fine with renaming if this kind of hint is more confusing than helpful of course.

About using syslog instead of a private journal:

The journal is more than a log, it is consulted (and altered) for two events:

• For informal purpose only when user stops a VM which has been shutdown from inside the VM e.g. In that case the info reads 'not running anymore' instead of 'was not started'.
• When user requests to 'stop' in AUTO mode (no bhyve_startvms defined in rc.conf and none defined on CLI). I determine the shut down order from the journal for several reasons. Determining the order the same way as for startup might not reflect reality: The VM can have autostart disabled but user started it via naming on CLI. And I haven't implemented a priority handling (yet). The whole part would have to be re-rwitten if once added. Parsing the logfile would be possible, but if user wants to stop her VMs after 3 weeks for example, newsyslog(8) would have rotated all default logs, maybe even overwritten archived logs. I could configure a private logfile via syslog.conf.d and local facility of course, but I need the option to remove stopped VM names from the journal too. Otherwise I'd need to parse/track for stopped VMs too, which is even significantly more overhead.

Since /var/run/bhyve is specified in /etc/mtree/BSD.var.dist, I took that cheap route.
Bad decision?

[evfbsd_yandex.ru]

But I'm fine with renaming if this kind of hint is more confusing than helpful of course.

Yes, imho more confusing.
For example:
cat /etc/defaults/rc.conf | grep -ci cfg
0
cat /etc/defaults/rc.conf | grep -ci conf
67

About using syslog instead of a private journal:

Journal and current-status are different concepts:
It is definitely necessary to place information about events in the log (syslog) - events may be needed redirected to a centralized logging system.

bhyve_startupjournal="/var/run/bhyve/startupjournal"
can be renamed, for sample
bhyve_startupstatus="/var/run/bhyve/startupstatus"

[hs.freebsd_id.omnilan.de]

Got it, you are right 'startupjournal' is wrong. Since it's only very loos state tracking, I I hesitated using naything with »state»...
I'm about to do some more production tests and will come up with a new diff these days - will change # REQUIRE: to DAEMON - even in jail motsly everybody will wan netwiork running before VM starts.|
Also the examples were not well generalized - will replace too.
Question for $bhyve_conf:
I guess /etc/bhyve.conf.d is better than /etc/bhyve.d.
jail(8) has jail.conf(5) and e/etc/jail.conf.d.
bhuve(8) has bhyve_config(5) - shall there be a relation to .d/, like /etc/bhyve_config.d ?

[dch]

add bhyve group for more visibility

[hs.freebsd_id.omnilan.de]

• Rename rc.conf(5) variable names "bhyve_cfg" to "bhyve_conf" and "bhyve_startupjournal" to "bhyve_launchstate" along with its filename.
• Extended __setvar_vmbootdiskimage() to allow booting FreeBSD from CD with the help of the new script-private "bhyveload_hba" variable in config file.
• Fixes and improvments in do_vm_spawn() - sould respect lpc.bootrom and bootrom.
• rcorder(8) changed from pre-jail to post DAEMON
• Both example config files reworked