bhyve: add helper for adding fwcfg files

[corvink]

Summary

Fwcfg items without a fixed index are reported by the file_dir. They
have an index of 0x20 and above. This helper simplifies the addition of
such fwcfg items. It selects a new free index, assigns it to the fwcfg
items and creates an proper entry in the file_dir.

[corvink]

• rebase

[markj]

usr.sbin/bhyve/qemu_fwcfg.c

307

334

What synchronization (e.g., a mutex) prevents the guest from reading a directory entry while this function is running? Are directory entries ever added dynamically, while the guest is running?

[corvink]

usr.sbin/bhyve/qemu_fwcfg.c

334

All directory entries will be created at boot.

A dynamic file content could be interesting. However, I can't think of a use case for dynamic file creation.

[corvink]

• fix typo in comment

[markj]

usr.sbin/bhyve/qemu_fwcfg.c

281

Is there any constraint on the characters that can be used in a file name?

Why does it have to be sorted? I'm a little nervous about using strcmp() to enforce some ordering.

[corvink]

usr.sbin/bhyve/qemu_fwcfg.c

281

I'm not aware of any constraints.

QEMU sorts by strcmp too on non legacy vms:
https://github.com/qemu/qemu/blob/b86307ecef9222c335ebd0ed4da2b243e86f779e/hw/nvram/fw_cfg.c#L842-L855
https://github.com/qemu/qemu/blob/b86307ecef9222c335ebd0ed4da2b243e86f779e/hw/nvram/fw_cfg.c#L957-L960

[theraven]

Do you have plans to add the guest part of this device? Currently, Podman uses this mechanism to pass configuration information (specifically, accounts to create and ssh keys) to VMs that it creates to run containers. It would be possible to add an alternative mechanism for FreeBSD (though it’s not clear what that mechanism should be) but if there’s an existing device to grab this config and provide it to userspace that someone is working on then that would be much easier to use.

[corvink]

In D38336#922311, @theraven wrote:

Do you have plans to add the guest part of this device? Currently, Podman uses this mechanism to pass configuration information (specifically, accounts to create and ssh keys) to VMs that it creates to run containers. It would be possible to add an alternative mechanism for FreeBSD (though it’s not clear what that mechanism should be) but if there’s an existing device to grab this config and provide it to userspace that someone is working on then that would be much easier to use.

Not sure if I understand your question correctly.

Bhyve already reports the fwcfg device by it's ACPI tables. One week ago, EDKII merged a commit (https://github.com/tianocore/edk2/commit/4bd0849d81b48233196092868081315ec4d9233d) to pick up and install the ACPI tables provided by bhyve. So, a linux guest should automatically detect this device when using the latest EDKII version.

I'm not sure if we can upstream that early. Otherwise, we have to wait for the next EDKII release version which will be released end of august (https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Release-Planning). After that, we have to update our edk2 port at https://github.com/freebsd/freebsd-ports/blob/f213d11b20afd3d6b333b5a889a051f62afaf92b/sysutils/edk2/Makefile#L2.

[markj]

In D38336#922315, @corvink wrote:

In D38336#922311, @theraven wrote:

Do you have plans to add the guest part of this device? Currently, Podman uses this mechanism to pass configuration information (specifically, accounts to create and ssh keys) to VMs that it creates to run containers. It would be possible to add an alternative mechanism for FreeBSD (though it’s not clear what that mechanism should be) but if there’s an existing device to grab this config and provide it to userspace that someone is working on then that would be much easier to use.

Not sure if I understand your question correctly.

I'm not sure either, but I think David is asking about some mechanism in a FreeBSD guest to export fwcfg info configured by the host, something like a filesystem mounted at /dev/fwcfg in the guest.

(I'm not aware of any such driver, but it seems fairly straightforward to implement...?)

Bhyve already reports the fwcfg device by it's ACPI tables. One week ago, EDKII merged a commit (https://github.com/tianocore/edk2/commit/4bd0849d81b48233196092868081315ec4d9233d) to pick up and install the ACPI tables provided by bhyve. So, a linux guest should automatically detect this device when using the latest EDKII version.

I'm not sure if we can upstream that early. Otherwise, we have to wait for the next EDKII release version which will be released end of august (https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Release-Planning). After that, we have to update our edk2 port at https://github.com/freebsd/freebsd-ports/blob/f213d11b20afd3d6b333b5a889a051f62afaf92b/sysutils/edk2/Makefile#L2.

[theraven]

Yes, exactly. It looks as if NetBSD has such a driver (https://man.netbsd.org/qemufwcfg.4), along with a FUSE FS the shared files into the filesystem. If anyone is working on such a thing for FreeBSD, it would be helpful, otherwise I will have a go next time I have some spare time.

[corvink]

In D38336#922447, @markj wrote:

In D38336#922315, @corvink wrote:

In D38336#922311, @theraven wrote:

Do you have plans to add the guest part of this device? Currently, Podman uses this mechanism to pass configuration information (specifically, accounts to create and ssh keys) to VMs that it creates to run containers. It would be possible to add an alternative mechanism for FreeBSD (though it’s not clear what that mechanism should be) but if there’s an existing device to grab this config and provide it to userspace that someone is working on then that would be much easier to use.

Not sure if I understand your question correctly.

I'm not sure either, but I think David is asking about some mechanism in a FreeBSD guest to export fwcfg info configured by the host, something like a filesystem mounted at /dev/fwcfg in the guest.

Got it. No, I'm currently not working on such a driver.

(I'm not aware of any such driver, but it seems fairly straightforward to implement...?)

I'm not aware either. The fwcfg interface is very simple. You can find some information here https://www.qemu.org/docs/master/specs/fw_cfg.html.
PS: bhyve only implements the io port interface yet.

Bhyve already reports the fwcfg device by it's ACPI tables. One week ago, EDKII merged a commit (https://github.com/tianocore/edk2/commit/4bd0849d81b48233196092868081315ec4d9233d) to pick up and install the ACPI tables provided by bhyve. So, a linux guest should automatically detect this device when using the latest EDKII version.

I'm not sure if we can upstream that early. Otherwise, we have to wait for the next EDKII release version which will be released end of august (https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Release-Planning). After that, we have to update our edk2 port at https://github.com/freebsd/freebsd-ports/blob/f213d11b20afd3d6b333b5a889a051f62afaf92b/sysutils/edk2/Makefile#L2.

While the fwcfg has a fixed io port address, the OS shouldn't depend on it. Instead, it should parse the ACPI entry or use another mechanism like https://github.com/torvalds/linux/blob/master/drivers/firmware/qemu_fw_cfg.c#L9-L15. So, edk2 needs to be updated. Nevertheless, you can start by using a fixed io port address.

In D38336#922455, @theraven wrote:

Yes, exactly. It looks as if NetBSD has such a driver (https://man.netbsd.org/qemufwcfg.4), along with a FUSE FS the shared files into the filesystem. If anyone is working on such a thing for FreeBSD, it would be helpful, otherwise I will have a go next time I have some spare time.

I'm not familiar with fuse/sysfs etc. Additional, I'm not sure which way to go. Linux uses a fixed sysfs path at /sys/firmware/qemu_fw_cfg/ while NetBSD allows you to mount it where you want.

[theraven]

Thanks. I've implemented a NetBSD-compatible kernel interface and tested it with the NetBSD FUSE filesystem:

https://github.com/davidchisnall/qemufwcfg

So far, I've tested it only on QEMU/AArch64, more testing (and bug reports) would be welcome. In particular, it *should* support the I/O port API, but I have not tested this yet (Arm uses the MMIO interface, and it's also a bit nicer since it lets us transfer 8 bytes per VM exit rather than 1, which reduces the need for the DMA mode).

[corvink]

In D38336#927214, @theraven wrote:

Thanks. I've implemented a NetBSD-compatible kernel interface and tested it with the NetBSD FUSE filesystem:

https://github.com/davidchisnall/qemufwcfg

So far, I've tested it only on QEMU/AArch64, more testing (and bug reports) would be welcome. In particular, it *should* support the I/O port API, but I have not tested this yet (Arm uses the MMIO interface, and it's also a bit nicer since it lets us transfer 8 bytes per VM exit rather than 1, which reduces the need for the DMA mode).

Thanks. You can enable fwcfg by -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,fwcfg=qemu. You can also add additional item with the -f option. It has the same syntax like the -fwcfg option of qemu (https://www.qemu.org/docs/master/specs/fw_cfg.html#externally-provided-items).

[theraven]

Thanks. You can enable fwcfg by -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,fwcfg=qemu. You can also add additional item with the -f option. It has the same syntax like the -fwcfg option of qemu (https://www.qemu.org/docs/master/specs/fw_cfg.html#externally-provided-items).

Thanks. The only system where I have bhyve set up is 13.2, unfortunately, so I'll probably have to wait a bit for this to be MFC'd.

[corvink]

In D38336#927310, @theraven wrote:

Thanks. You can enable fwcfg by -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,fwcfg=qemu. You can also add additional item with the -f option. It has the same syntax like the -fwcfg option of qemu (https://www.qemu.org/docs/master/specs/fw_cfg.html#externally-provided-items).

Thanks. The only system where I have bhyve set up is 13.2, unfortunately, so I'll probably have to wait a bit for this to be MFC'd.

fwcfg is already MFC'd (https://github.com/freebsd/freebsd-src/commit/d468d5dd195f58d9898f505289364ac98cfcbf05). The -f option is missing in the stable/13 branch.

However, this option will never make it into 13.2. So, you have to wait for 13.3, 14.0 or build bhyve on your own.

[andy_omniosce.org]

usr.sbin/bhyve/qemu_fwcfg.c

310

While porting this to illumos I saw a buffer overrun here:

Status:         ready and active
Concurrency:    16
Logs:           transaction=256k (inactive)
Message buffer:
umem allocator: redzone violation: write past end of buffer
buffer=13e6040  bufctl=13de540  cache: umem_alloc_768
previous transaction on buffer 13e6040:
thread=1  time=T-0.000057448  slab=13b2bf0  cache: umem_alloc_768
libumem.so.1'calloc+0x30
bhyve'qemu_fwcfg_add_file+0x19d
bhyve'basl_finish_install_guest_tables+0x45
bhyve'basl_finish+0x64
bhyve'acpi_build+0x1e8
bhyve'main+0x699
bhyve'_start_crt+0x87
bhyve'_start+0x18
umem: heap corruption detected
stack trace:
libumem.so.1'umem_malloc_free+0x1d
bhyve'qemu_fwcfg_add_file+0x1ff
bhyve'qemu_loader_finish+0x93
bhyve'basl_finish+0x12c
bhyve'acpi_build+0x1e8
bhyve'main+0x699
bhyve'_start_crt+0x87
bhyve'_start+0x18

[andy_omniosce.org]

usr.sbin/bhyve/qemu_fwcfg.c

310

I did add a fix too, as a suggested change but I don't see it showing up now. This is what I am now testing with:

diff
                    &fwcfg_sc.directory->files[file_index],
-                    (count - file_index) * sizeof(struct qemu_fwcfg_file));
+                    (count - file_index - 1) * sizeof(struct qemu_fwcfg_file));

                /* free old directory */