VNET has brought a huge advantage of easily creating independent networks stacks on a single machine. One of the pillars is the ability to assign certain interface to the particular VNET (and bring it back). This is useful when adding external-connectivity interfaces relying on physical interfaces (plans, bridges) or simply physical interfaces themselves
Internally this move (dubbed //vmove//) is implemented by shutting the interface down, removing all addresses, multicast groups, closing attached sockets, etc, and actually reassigning `if_vnet` the the new value.
Unfortunately, this approach leads to the abstraction leak - every network-related subsystem have to account for the random VNET change of an ifp.
Given the fact that decent number of variables are VNET-local, it manifests added complexity. For example, all traffic-queuing parts of the kernel have to handle this VNET changes. Nexthops logic have to account for that as well. It is not always easy to actually address. IMO the existing approach is too fragile and provokes hard-to-find “bugs” the neighbouring subsystems when migrating interfaces between VNETs.
This review proposes an alternative approach to the problem for the cloned interfaces. The base idea is to leverage existing cloner interfaces and create a new `ifp` in the target VNET, copying some properties from the original `ifp` and then destroying it. To be more specific, cloner framework adds a new `vmove_f` method that allows “complex” interfaces such as vlan to copy interface-specific properties such as tag or parent interface. It also provide helpers (`ifc_save_vparams()`, `ifc_apply_vparams()`) to copy basic properties like mtu or interface description. Not all interfaces need to implement `vmove_f` callback - many “simple” ones like `lo` can leverage default `if_vmove_simple()`, which creates an interface with the same name in the target VNET and destroys original interface upon success.
Note that this approach allows gradual migration - interfaces that are not converted yet can explicitly mark their cloners with `IFC_F_NOMOVE`, which will trigger fallback to the existing `if_vmove()` logic.
The implementation provides an implementation of `vmove_f` handler for `if_vlan` and `if_epair`. Some works on the implementation:
* if_vlan does interface destruction first to propagate eventhandler event to the driver, so it can update the ifp afterwards
* if_epair is more complex. oifp pointer is now not static, so some tweaks are required to safely access `sc->oifp`. As the new approach allows to destruction of just one interface, separate index counters are required to support accounting for A and B interface. if_clone() KPI has been extended to support that.