Virtualise if_vxlan

[zlei]

Summary

This is the last driver which lacks VNET support. With this change it will be practical to do tests via test framework. This should also fix the long standing PR 255882.

Test Plan

1. 1. stress test with iperf3, with different vxlan interface in different jails which connected with epair(4).
2. test vmove, aka ifconfig vxlan0 vnet foo, ifconfig vxlan0 -vnet foo.
3. test vxlan hardware offload ( no hardware available right now )
4. create and config vxlan in vnet jail. Destroy the jail ( jail -R foo ).

[zlei]

Current know problem, the fourth test,

create and config vxlan in vnet jail. Destroy the jail ( jail -R foo ).

may panic, or block the jail from destroying, or the jail keep in dying state ( jls -d ).

[afedorov]

It won't work becaus, if_vxlan(4) exports various statistics and tables via sysctl. But for VNET we can only change the values, and not the tree itself.

The problem is somewhat broader.

[afedorov]

sys/net/if_vxlan.c

3583

We can’t move vxlan to the jail., because vxl_unit is the same.

[zlei]

In D48646#1109329, @afedorov wrote:

It won't work becaus, if_vxlan(4) exports various statistics and tables via sysctl. But for VNET we can only change the values, and not the tree itself.

The problem is somewhat broader.

Yeah, this is another known issue, I forgot to mention this. I'm proposing to have a global unique number for those dynamically added sysctl nodes, but that apparently may leak sensitive info , such as forwarding table entries. Or we should apply restrictions for that, but I have no idea how to restrict that.

[zlei]

sys/net/if_vxlan.c

3583

ifnet vmove is another known problem. For vxlan(4), it appears less useful to move an interface to another vnet, given the cloner has been already virtualized and you can attache to that jail / vnet to clone create a new one.

I once have the idea to introduce a cloner flag, say IFC_F_NO_VMOVE to forbid some clone create interfaces to be moved. Known good candidates are if_pflog, if_pfsync, and if_lo. Tunnel drivers such as if_gif / if_gre / if_vxlan may also have this restriction, so we no longer need if_reassign handler.

[zlei]

sys/net/if_vxlan.c

3583

We can’t move vxlan to the jail., because vxl_unit is the same.

See vxlan_reassign(), the interface to be vmoved was reset to factory default status, aka no vni, vxlan local or vxlan remote .

[zlei]

Fixed panic on destroying vxlan interface. Rebased on latest main branch.

[afedorov]

In D48646#1109461, @zlei wrote:

In D48646#1109329, @afedorov wrote:

It won't work becaus, if_vxlan(4) exports various statistics and tables via sysctl. But for VNET we can only change the values, and not the tree itself.

The problem is somewhat broader.

Yeah, this is another known issue, I forgot to mention this. I'm proposing to have a global unique number for those dynamically added sysctl nodes, but that apparently may leak sensitive info , such as forwarding table entries. Or we should apply restrictions for that, but I have no idea how to restrict that.

I think the only reliable method is to remove the creation of dynamic sysctl altogether and use ioctl(2) or netlink(4) to get statistics and "routing" tables. Or create them only in the host VNET. But this violates POLA.

As for this review, since it solves at least some of the problems that real users face, then of course it should be commited.