Complete phase two of 3da1cf1e88f8.
In 3da1cf1e88f8, the meaning of the CTLFLAG_TUN flag is extended to automatically check if there is a kernel environment variable which shall initialize the SYSCTL during early boot. It works for all SYSCTL types both statically and dynamically created ones, except for the SYSCTLs which belong to VNETs.
This change extends the meaning of the CTLFLAG_TUN flag further, to allow it works for the SYSCTLs which belong to VNETs. A typical usage is `CTLFLAG_VNET | CTLFLAG_RWTUN`. `CTLFLAG_VNET | CTLFLAG_RDTUN` is also supported but that is not much useful, as per VNET readonly tunables can be shared with a global one.
Note that the values of SYSCTLs belong to VNETs will not reflect the change of kernel environment variable after the kernel modules been initialized. This behavior is currently the same with that of SYSCTLs not belong to VNETs.
The following SYSCTLs belong to VNETs have CTLFLAG_TUN flag, and will be fixed by this change:
net.add_addr_allfibs
net.bpf.optimize_writers
net.inet.tcp.fastopen.ccache_buckets
net.key.spdcache.maxentries
net.key.spdcache.threshold
net.link.bridge.pfil_onlyip
net.link.bridge.pfil_bridge
net.link.bridge.pfil_ipfw_arp
net.link.bridge.pfil_member
net.link.bridge.pfil_local_phys
net.link.bridge.log_stp
net.link.bridge.bridge_inherit_mac
net.link.lagg.default_use_flowid
net.link.lagg.default_use_numa
net.link.lagg.default_flowid_shift
net.link.lagg.lacp.debug
net.link.lagg.lacp.default_strict_mode
The values of these SYSCTLs belong to VNETs are loaded by TUNABLE_XXX_FETCH, thus are not affected by this change:
net.inet.ip.reass_hashsize
net.inet.tcp.hostcache.cachelimit
net.inet.tcp.hostcache.hashsize
net.inet.tcp.hostcache.bucketlimit
net.inet.tcp.syncache.bucketlimit
net.inet.tcp.syncache.cachelimit
net.inet.tcp.syncache.hashsize
Given TUNABLE_XXX_FETCH has maximum flexibility, those SYSCTLs in newly created vnets will have new values when corresponding kernel environment variables change. Ideally the CTLFLAG_TUN flag of those SYSCTLs can be removed, or we step further, automatically update the 'master' copy of the vnet SYSCTLs and newly created vnets can have new values, then we can eliminate TUNABLE_XXX_FETCH as much as possible.
Discussed with: hselasky, glebius
Fixes: 3da1cf1e88f8 Extend the meaning of the CTLFLAG_TUN flag ...
MFC after: 3 weeks