bhyve: add support for QEMU's fwcfg

[corvink]

Summary

Bhyve's fwctl only passes the number of vcpus to the guest. That could be done easily by QEMU's fwcfg. Additionally, QEMU's fwcfg has much more features. Bhyve will support all those feature too, when it supports fwcfg.

Add -l bootrom,<path/to/your/UEFI.fd>,fwcfg=qemu to your bhyve cmdline to enable QEMU's fwcfg.

This revision is based on D33767. Additionally, it requires D33766 to work properly on UEFI VMs.

This revision is split into smaller commits at https://github.com/Beckhoff/freebsd-src/commits/phab/corvink/fwcfg

[grehan]

Bhyve's fwctl only passes the number of vcpus to the guest. That could be done easily by QEMU's fwcfg.

There's zero reason to change for this alone, and it requires a flag day for both bhyve and UEFI firmware which has thus far been avoided.

Additionally, QEMU's fwcfg has much more features.

Those features are dictated by Qemu development, and that makes it difficult or impossible to add custom features for bhyve.

Any required features could be added to bhyve fwctl, and in either case require bhyve-side development.

fwctl is an interface that is more sysctl-like so allows dynamic registration of OIDs, and also a presence-detect so that firmware and bhyve can be out-of-sync.

Sadly, bhyve's fwctl uses the same ports as QEMU's fwcfg. For that reason, it's not possible to keep them both.

There is a signature check so in theory a flag could be provided to switch between one or the other.

But, I would prefer that fwctl remain.

[corvink]

In D31578#711899, @grehan wrote:

Bhyve's fwctl only passes the number of vcpus to the guest. That could be done easily by QEMU's fwcfg.

There's zero reason to change for this alone, and it requires a flag day for both bhyve and UEFI firmware which has thus far been avoided.

As next step bhyve could implement more fwcfg items from QEMU to use QEMU's features without developing an own firmware implementation.

Additionally, QEMU's fwcfg has much more features.

Those features are dictated by Qemu development, and that makes it difficult or impossible to add custom features for bhyve.

Any required features could be added to bhyve fwctl, and in either case require bhyve-side development.

Bhyve could just use it's own namespace. QEMU specification allows users to define own namespaces (https://github.com/qemu/qemu/blob/master/docs/specs/fw_cfg.txt):

Item names beginning with "opt/" are reserved for users.  QEMU will
never create entries with such names unless explicitly ordered by the
user.

To avoid clashes among different users, it is strongly recommended
that you use names beginning with opt/RFQDN/, where RFQDN is a reverse
fully qualified domain name you control.  For instance, if SeaBIOS
wanted to define additional names, the prefix "opt/org.seabios/" would
be appropriate.

fwctl is an interface that is more sysctl-like so allows dynamic registration of OIDs, and also a presence-detect so that firmware and bhyve can be out-of-sync.

Maybe not intended by QEMU, but should be possible with it's fwcfg too.

Sadly, bhyve's fwctl uses the same ports as QEMU's fwcfg. For that reason, it's not possible to keep them both.

There is a signature check so in theory a flag could be provided to switch between one or the other.

Correct me if I'm wrong. Only the first four reads of the data port will return the signature. Since bhyve uses the same ports as QEMU, it's neccessary to check the signature before QEMU reads from that port. I don't know a good solution how to met this requirement.

But, I would prefer that fwctl remain.

[grehan]

There's zero reason to change for this alone, and it requires a flag day for both bhyve and UEFI firmware which has thus far been avoided.

As next step bhyve could implement more fwcfg items from QEMU to use QEMU's features without developing an own firmware implementation.

What are the features that you're looking to get from Qemu fwcfg ?

Bhyve could just use it's own namespace. QEMU specification allows users to define own namespaces (https://github.com/qemu/qemu/blob/master/docs/specs/fw_cfg.txt):

Ok, that seems a possibility.

There is a signature check so in theory a flag could be provided to switch between one or the other.

Correct me if I'm wrong. Only the first four reads of the data port will return the signature. Since bhyve uses the same ports as QEMU, it's neccessary to check the signature before QEMU reads from that port. I don't know a good solution how to met this requirement.

I meant an option flag for bhyve to present either one interface or the other, so guest firmware would only see a single signature.

[corvink]

In D31578#711977, @grehan wrote:

There's zero reason to change for this alone, and it requires a flag day for both bhyve and UEFI firmware which has thus far been avoided.

As next step bhyve could implement more fwcfg items from QEMU to use QEMU's features without developing an own firmware implementation.

What are the features that you're looking to get from Qemu fwcfg ?

• An E820 table. It could help D26209.

• ACPI/SMBIOS tables. Don't know if it's benefitial, yet.

• Specifying a bootorder.

I'm not a QEMU expert. Maybe I'm overlooking some interesting features.

I don't know if bhyve's fwctl can pass large structs like ACPI tables to the guest. Even if it's possible, why should bhyve create a new firmware implementation for that instead of using QEMU's one. If you don't like QEMU's implementation, it'll be still possible to create an own implementation by adding a new item to your own namespace.

There is a signature check so in theory a flag could be provided to switch between one or the other.

Correct me if I'm wrong. Only the first four reads of the data port will return the signature. Since bhyve uses the same ports as QEMU, it's neccessary to check the signature before QEMU reads from that port. I don't know a good solution how to met this requirement.

I meant an option flag for bhyve to present either one interface or the other, so guest firmware would only see a single signature.

Ok. Understood.

[corvink]

Add option -l fwcfg,<name> to select either QEMU or bhyve fwcfg.

[grehan]

What are the features that you're looking to get from Qemu fwcfg ?

• An E820 table. It could help D26209.

...

• ACPI/SMBIOS tables. Don't know if it's benefitial, yet.

SMBIOS is already done in bhyve - UEFI searches for the signature in memory and re-uses that.

For ACPI, I concede that building the DSDT in bhyve is dynamic and can be driven from configuration, as opposed to the static DSDT in bhyve/UEFI. There are already features in bhyve, such as multiple PCI segments, that aren't available in the UEFI version.

• Specifying a bootorder.

...

I'm not a QEMU expert. Maybe I'm overlooking some interesting features.

I don't know if bhyve's fwctl can pass large structs like ACPI tables to the guest.

It can.

Even if it's possible, why should bhyve create a new firmware implementation for that instead of
using QEMU's one. If you don't like QEMU's implementation, it'll be still possible to create an own
implementation by adding a new item to your own namespace.

Sure.

There is a bigger issue and that is forwards/backwards compatibility.

There *cannot* be a flag day for bhyve and firmware - the firmware is used across all supported FreeBSD versions, so has to support older bhyve. The current firmware patch will force older bhyve configurations back to a single vCPU. There needs to be concurrent support for bhyvectl in bhyve/UEFI until all versions without Qemu fwcg are EOL'd.

It may turn out that having to support that is greater than the advantage of cutting over to Qemu fwcg.

[grehan]

In D31578#712408, @c.koehne_beckhoff.com wrote:

Add option -l fwcfg,<name> to select either QEMU or bhyve fwcfg.

The -l option is for ISA devices. This flag should be an additional option to the existing -l bootrom

[corvink]

Update firmware to support both: BhyveFwCtl and QemuFwCfg

[corvink]

• Rebase on 14.0-CURRENT

[corvink]

• Rebase on 14.0-CURRENT
• Add new cmdline option to add fwcfg files specified by the user

[editor_callfortesting.org]

This comment was removed by editor_callfortesting.org.

[corvink]

Split into multiple patches @ D38331, D38332, D38333, D38334, D38335, D38336, D38337 and D38338