vimage: support kmods determining VIMAGE status at runtime

[kevans]

Summary

This is specifically for non-tied kernel modules -- tied kmods will use the options from the kernel build and shouldn't change.

Currently, we must hack VIMAGE into independent kmod builds if the result is to be
used on a VIMAGE kernel and a VIMAGE-enabled kmod cannot run on a non-VIMAGE kernel. This causes some amount of pain in iterating on a new kmod if you're oblivious to the fact that you've created an incompatible kmod because you didn't match the VIMAGE status of the kernel you're planning to run it on.

Push non-tied kernel modules into determining at runtime how we're operating unless they've specifically been compiled with VIMAGE set. Major changes:

• Non-VIMAGE VNET_{DECLARE,DEFINE,DEFINE_STATIC} macros now name the global with the VNET_NAME from the VIMAGE case, so that we can consistently reference them from kmods

• The kernel now includes a vimage_configured symbol to indicate whether it's been configured for VIMAGE or not

• vnet_support.c has been added with vimage_configured and some sysinit shims that will proxy the sysinit/sysuninit bits to the proper underlying function based on how the kernel's configured

VNET_{DEFINE,DEFINE_STATIC} still uses linker sets in kernel modules, but should DTRT (from what I've managed to grok of the kernel linker) and get allocated in global space in non-VIMAGE kernels. This is then reflected in VNET_PTR/VNET in the VIMAGE_RUNTIME case.

This patch is likely incomplete... it does work in some of the more primitive cases, though.

Test Plan

It admittedly has not received great testing -- I've compiled versions of if_bridge/if_tuntap/if_vether (from net/vether-kmod in ports) with the new vnet.h header and lightly tested these interfaces on both VIMAGE/non-VIMAGE kernels with this patch applied. The usual panics/misbehavior/failures from VIMAGE-mismatch were not present.

[bz]

Hi,

I just only scrolled through this and there's a few things. I am not going into detail at the moment as I'd love to give this a full review once my brain capacity for yet-another-module-thing-complication is there. It'll be helpful to later decompose this into "whitespace change" and functional change (as a lot of renaming, etc. makes this hard to read).

We still have architectures where we cannot get in-src-tree modules built along kernel non-panicing in all cases. We've been dealing and finding edge-cases over the last 18 months. While I appreciate the effort of trying to solve more kernel module problems and easing pain, can you please hold this off at least until the more severe problems are fixed?

Also, should this go ahead, the same needs to be done for DPCPU as that uses exactly the same mechanisms and that's where the VNET logic came from.

Thanks,
/bz

[kevans]

In D20278#441710, @bz wrote:

Hi,

I just only scrolled through this and there's a few things. I am not going into detail at the moment as I'd love to give this a full review once my brain capacity for yet-another-module-thing-complication is there. It'll be helpful to later decompose this into "whitespace change" and functional change (as a lot of renaming, etc. makes this hard to read).

We still have architectures where we cannot get in-src-tree modules built along kernel non-panicing in all cases. We've been dealing and finding edge-cases over the last 18 months. While I appreciate the effort of trying to solve more kernel module problems and easing pain, can you please hold this off at least until the more severe problems are fixed?

Can you point me to any open reports of such problems? I'm very much interested in getting this stuff working well.

Also, should this go ahead, the same needs to be done for DPCPU as that uses exactly the same mechanisms and that's where the VNET logic came from.

AFAICT DPCPU doesn't have the same problem. DPCPU stuff isn't optionally compiled into the kernel, so it doesn't need shims and the DPCPU* macros are always correct for untied modules. The problem only comes in with VIMAGE because there's some extra infrastructure that rides in with VIMAGE. That said, the shims here are wrong because they're compiled into the kernel and thus don't need to check vimage_configured to know if the VIMAGE SYSINIT infrastructure should be used.