bhyve.8: mention MAC and MTU config for ngd and slirp

[novel]

Summary

Both 'ngd' and 'slirp' network backends support configuring the MAC
address and MTU, just like the other backends, so mention that in the
manual page.

[markj]

Actually, the slirp backend has a fixed MTU. We should either update it to handle a configured size, or make it an error to specify the MTU.

[novel]

In D54133#1237001, @markj wrote:

Actually, the slirp backend has a fixed MTU. We should either update it to handle a configured size, or make it an error to specify the MTU.

Thanks, I'll try to see if I could add MTU configuration for SLIRP.

[markj]

In D54133#1237024, @novel wrote:

In D54133#1237001, @markj wrote:

Actually, the slirp backend has a fixed MTU. We should either update it to handle a configured size, or make it an error to specify the MTU.

Thanks, I'll try to see if I could add MTU configuration for SLIRP.

Cool! Search for uses of SLIRP_MTU, they all need to be updated.

[novel]

Update the SLIRP backend to respect MTU configuration.

[novel]

In D54133#1237026, @markj wrote:

In D54133#1237024, @novel wrote:

In D54133#1237001, @markj wrote:

Actually, the slirp backend has a fixed MTU. We should either update it to handle a configured size, or make it an error to specify the MTU.

Thanks, I'll try to see if I could add MTU configuration for SLIRP.

Cool! Search for uses of SLIRP_MTU, they all need to be updated.

I have updated this revision with these changes. It works fine for me with MTU values from 1476 to 4096 (didn't test larger), but for lower values it fails with:

Assertion failed: ((size_t)n <= priv->mtu), function slirp_recv, file /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/net_backend_slirp.c, line 238.

I'll debug it further, but now I'm not sure: currently I do not validate the MTU value range in the SLIRP backend assuming it's already validated on the device side (e.g. in pci_virtio_net.c). Now I wonder if the backend might have stricter requirements for MTU than the device, should the backend implement its own validation on top of the device side validation?

[markj]

In D54133#1237097, @novel wrote:

In D54133#1237026, @markj wrote:

In D54133#1237024, @novel wrote:

In D54133#1237001, @markj wrote:

Actually, the slirp backend has a fixed MTU. We should either update it to handle a configured size, or make it an error to specify the MTU.

Thanks, I'll try to see if I could add MTU configuration for SLIRP.

Cool! Search for uses of SLIRP_MTU, they all need to be updated.

I have updated this revision with these changes. It works fine for me with MTU values from 1476 to 4096 (didn't test larger), but for lower values it fails with:

Assertion failed: ((size_t)n <= priv->mtu), function slirp_recv, file /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/net_backend_slirp.c, line 238.

I'll debug it further, but now I'm not sure: currently I do not validate the MTU value range in the SLIRP backend assuming it's already validated on the device side (e.g. in pci_virtio_net.c). Now I wonder if the backend might have stricter requirements for MTU than the device, should the backend implement its own validation on top of the device side validation?

Do you have commit 69f61cee2efb1eec0640ca7de9b2d51599569a5d commit applied to the host kernel?

usr.sbin/bhyve/net_backend_slirp.c

210

Can we please allocate this buffer in the slirp_priv structure rather than calling malloc() once for every packet?

218

We can just get rid of this assertion, it's a bit silly.

usr.sbin/bhyve/slirp/slirp-helper.c

109

This assertion should be updated too.

[quentin.thebault_defenso.fr]

An ng_device(4) is not an interface, so it does not have a MAC address. The way I see it, it is just a wire. If you think about a tap(4), it's like an ng_eiface(4) linked to a ng_device. But the MAC address belongs to the ng_eiface end, not the ng_device one.

For the MTU it might be a bit different because the ng_device could maybe impose a limit on to the size it could transport, but in any case there is currently no code allowing the user to configure it. The ioctls are not implemented.

[novel]

• bhyve.8: Drop ngdN related changes
• net_backend_slirp, slirp-helper: move buf to struct slirp_priv
• net_backend_slirp: remove assert from slirp_peek_recvlen()
• slirp-helper: update assert in slirp_cb_send_packet()

[novel]

In D54133#1237246, @quentin.thebault_defenso.fr wrote:

An ng_device(4) is not an interface, so it does not have a MAC address. The way I see it, it is just a wire. If you think about a tap(4), it's like an ng_eiface(4) linked to a ng_device. But the MAC address belongs to the ng_eiface end, not the ng_device one.

For the MTU it might be a bit different because the ng_device could maybe impose a limit on to the size it could transport, but in any case there is currently no code allowing the user to configure it. The ioctls are not implemented.

Thanks for clarification. I've removed this bit from the manual page, I think it's better to focus on SLIRP for now. However, bhyve still allows to set mac= and mtu=, probably it should print error for this backend.

[novel]

In D54133#1237195, @markj wrote:

In D54133#1237097, @novel wrote:

In D54133#1237026, @markj wrote:

In D54133#1237024, @novel wrote:

In D54133#1237001, @markj wrote:

Actually, the slirp backend has a fixed MTU. We should either update it to handle a configured size, or make it an error to specify the MTU.

Thanks, I'll try to see if I could add MTU configuration for SLIRP.

Cool! Search for uses of SLIRP_MTU, they all need to be updated.

I have updated this revision with these changes. It works fine for me with MTU values from 1476 to 4096 (didn't test larger), but for lower values it fails with:

Assertion failed: ((size_t)n <= priv->mtu), function slirp_recv, file /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/net_backend_slirp.c, line 238.

I'll debug it further, but now I'm not sure: currently I do not validate the MTU value range in the SLIRP backend assuming it's already validated on the device side (e.g. in pci_virtio_net.c). Now I wonder if the backend might have stricter requirements for MTU than the device, should the backend implement its own validation on top of the device side validation?

Do you have commit 69f61cee2efb1eec0640ca7de9b2d51599569a5d commit applied to the host kernel?

Yes, I have commit 69f61cee2efb1eec0640ca7de9b2d51599569a5d (and also commit 82d8a5029a80a77166dca098b8fedb10d84e4e38 which appears to be a follow up of the former) included in the host kernel.

In the meantime, I was giving it some more testing and noticed another issues:

While running wget https://download.freebsd.org/releases/ISO-IMAGES/15.0/FreeBSD-15.0-RELEASE-arm64-aarch64-dvd1.iso -O /dev/null, somewhere half way it fails with:

`#1 0x000000080114c934 in raise (s=s@entry=6) at /usr/home/novel/code/freebsd-src/lib/libc/gen/raise.c:48
#2 0x00000008011fe7a9 in abort () at /usr/home/novel/code/freebsd-src/lib/libc/stdlib/abort.c:61
#3 0x000000080112f521 in assert (func=<optimized out>, file=<optimized out>, line=line@entry=115, failedexpr=<optimized out>) at /usr/home/novel/code/freebsd-src/lib/libc/gen/assert.c:47
#4 0x000000000102500b in slirp_cb_send_packet (buf=<optimized out>, len=1514, param=<optimized out>) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:115
#5 0x000000080141ad02 in ?? () from /usr/local/lib/libslirp.so.0
#6 0x0000000801412f7d in ?? () from /usr/local/lib/libslirp.so.0
#7 0x0000000801416366 in ?? () from /usr/local/lib/libslirp.so.0
#8 0x0000000801421197 in ?? () from /usr/local/lib/libslirp.so.0
#9 0x000000080141ef1a in ?? () from /usr/local/lib/libslirp.so.0
#10 0x0000000001024edd in slirp_pollfd_loop (priv=0x7fffffffe7e0) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:303
#11 main (argc=<optimized out>, argv=<optimized out>) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:580
(gdb) fr 4
#4 0x000000000102500b in slirp_cb_send_packet (buf=<optimized out>, len=1514, param=<optimized out>) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:115
115 assert((size_t)n == len);
(gdb) p n
$1 = 0
(gdb) `

That's with a fairly large mtu=4092. So it's probably a bug in my code, not a kernel issue.

[novel]

In D54133#1237392, @novel wrote:

In D54133#1237195, @markj wrote:

In D54133#1237097, @novel wrote:

In D54133#1237026, @markj wrote:

In D54133#1237024, @novel wrote:

In D54133#1237001, @markj wrote:

Actually, the slirp backend has a fixed MTU. We should either update it to handle a configured size, or make it an error to specify the MTU.

Thanks, I'll try to see if I could add MTU configuration for SLIRP.

Cool! Search for uses of SLIRP_MTU, they all need to be updated.

I have updated this revision with these changes. It works fine for me with MTU values from 1476 to 4096 (didn't test larger), but for lower values it fails with:

Assertion failed: ((size_t)n <= priv->mtu), function slirp_recv, file /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/net_backend_slirp.c, line 238.

I'll debug it further, but now I'm not sure: currently I do not validate the MTU value range in the SLIRP backend assuming it's already validated on the device side (e.g. in pci_virtio_net.c). Now I wonder if the backend might have stricter requirements for MTU than the device, should the backend implement its own validation on top of the device side validation?

Do you have commit 69f61cee2efb1eec0640ca7de9b2d51599569a5d commit applied to the host kernel?

Yes, I have commit 69f61cee2efb1eec0640ca7de9b2d51599569a5d (and also commit 82d8a5029a80a77166dca098b8fedb10d84e4e38 which appears to be a follow up of the former) included in the host kernel.

In the meantime, I was giving it some more testing and noticed another issues:

While running wget https://download.freebsd.org/releases/ISO-IMAGES/15.0/FreeBSD-15.0-RELEASE-arm64-aarch64-dvd1.iso -O /dev/null, somewhere half way it fails with:

`#1 0x000000080114c934 in raise (s=s@entry=6) at /usr/home/novel/code/freebsd-src/lib/libc/gen/raise.c:48
#2 0x00000008011fe7a9 in abort () at /usr/home/novel/code/freebsd-src/lib/libc/stdlib/abort.c:61
#3 0x000000080112f521 in assert (func=<optimized out>, file=<optimized out>, line=line@entry=115, failedexpr=<optimized out>) at /usr/home/novel/code/freebsd-src/lib/libc/gen/assert.c:47
#4 0x000000000102500b in slirp_cb_send_packet (buf=<optimized out>, len=1514, param=<optimized out>) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:115
#5 0x000000080141ad02 in ?? () from /usr/local/lib/libslirp.so.0
#6 0x0000000801412f7d in ?? () from /usr/local/lib/libslirp.so.0
#7 0x0000000801416366 in ?? () from /usr/local/lib/libslirp.so.0
#8 0x0000000801421197 in ?? () from /usr/local/lib/libslirp.so.0
#9 0x000000080141ef1a in ?? () from /usr/local/lib/libslirp.so.0
#10 0x0000000001024edd in slirp_pollfd_loop (priv=0x7fffffffe7e0) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:303
#11 main (argc=<optimized out>, argv=<optimized out>) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:580
(gdb) fr 4
#4 0x000000000102500b in slirp_cb_send_packet (buf=<optimized out>, len=1514, param=<optimized out>) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:115
115 assert((size_t)n == len);
(gdb) p n
$1 = 0
(gdb) `

That's with a fairly large mtu=4092. So it's probably a bug in my code, not a kernel issue.

I noticed that I can reproduce this issue without my changes, created a PR for that: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=291616

[novel]

Rebased on top of commit daef625cf884, works fine now.

Testing plan:

bhyve -c 2 -m 4096 -S -A -I -u -H -P -s 0:0,hostbridge -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -s 1:0,lpc \
   -s 7:0,virtio-blk,/data/img/ubuntu2510.img -l com1,stdio -s 8:0,virtio-net,slirp,mac=94:06:31:1a:a6:f7,mtu=4092,open \  # <-- set MTU here
   ubuntu2510

Inside the guest:

• iperf -c <iperf_server> -u -i 1 -P 8 -t 60 -b 500M
• while wget https://<your_freebsd_mirror>/releases/ISO-IMAGES/15.0/FreeBSD-15.0-RELEASE-arm64-aarch64-dvd1.iso -O /dev/null; do : ; done