VNET has brought a huge advantage of easily creating independent networks stacks on a single machine. One of the pillars is the ability to assign certain interface to the particular VNET (and bring it back). This is useful when adding external-connectivity interfaces relying on physical interfaces (plans, bridges) or simply physical interfaces themselves
Internally this move (dubbed vmove) is implemented by shutting the interface down, removing all addresses, multicast groups, closing attached sockets, etc, and actually reassigning if_vnet the the new value.
Unfortunately, this approach leads to the abstraction leak - every network-related subsystem have to account for the random VNET change of an ifp.
Given the fact that decent number of variables are VNET-local, it manifests added complexity. For example, all traffic-queuing parts of the kernel have to handle this VNET changes. Nexthops logic have to account for that as well. It is not always easy to actually address. IMO the existing approach is too fragile and provokes hard-to-find “bugs” the neighbouring subsystems when migrating interfaces between VNETs.
This review proposes an alternative approach to the problem for the cloned interfaces. The base idea is to leverage existing cloner interfaces and create a new ifp in the target VNET, copying some properties from the original ifp and then destroying it. To be more specific, cloner framework adds a new vmove_f method that allows “complex” interfaces such as vlan to copy interface-specific properties such as tag or parent interface. It also provide helpers (ifc_save_vparams(), ifc_apply_vparams()) to copy basic properties like mtu or interface description. Not all interfaces need to implement vmove_f callback - many “simple” ones like lo can leverage default if_vmove_simple(), which creates an interface with the same name in the target VNET and destroys original interface upon success.
Note that this approach allows gradual migration - interfaces that are not converted yet can explicitly mark their cloners with IFC_F_NOMOVE, which will trigger fallback to the existing if_vmove() logic.
The implementation provides an implementation of vmove_f handler for if_vlan and if_epair. Some works on the implementation:
- if_vlan does interface destruction first to propagate eventhandler event to the driver, so it can update the ifp afterwards
- if_epair is more complex. oifp pointer is now not static, so some tweaks are required to safely access sc->oifp