Fix keep_state test.

remove 'extern name[]'

Fix build w/o Netlink.

.

Also reported in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=269075

In D40317#918251, @melifaro wrote:

In other words: this patch cannot go in as-is (or at least not until the IFF_UP issues are addressed).

I ran into that when using netlink to configure interfaces. I guess we need to issue an eventhandler contification if some of interface flags were changed. I’ll take a look at that

I've raised D40332 to address lack of notifications to carp and the rest of the system.

This one got committed.

In D39865#909198, @np wrote:

Almost there. Now the new ifnet is created successfully but there is an extra 0 in its name. The ifnet should have been t6nex0 and not t6nex00.

Are you sure it's t6nex0 without this change? Could you share what's the if_dunit value of this interface?
I'm struggling to understand the code path here :-(

# ifconfig t6nex0 create                                                             
t6nex00 
# ifconfig t6nex00
t6nex00: flags=840<RUNNING,SIMPLEX> metric 0 mtu 1500
       ether 00:00:00:00:00:00
       groups: tXnex
       media: Ethernet none <full-duplex>
       status: active
       nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

In D39614#907626, @kp wrote:

In D39614#907531, @melifaro wrote:

In D39614#902547, @kp wrote:

The fix to this particular issue depends on the answers of the above (or can be a bandaid as in diff if we want to kick the can down the road).

Those are all good questions, but I think in this case our issue is that the ifnet goes away between the task being enqueued and executed. There are a number of other places where we see things like that, and that's the 'general problem' I was referring to.

Ack! I was looking into it from a different angle. The mental model of "safe detach" to me is a) set the demarcation point that signals "no more data accepted", b) ensure no data is indeed accepted and c) clear the queued data. I'm a bit unsure if we can or should generalise the implementations for the different datapaths and control plane. The current code mostly have everything implemented - IFF_DYING flag is set, marking the end of the era and the taskq is cleaned from the link tasks matching this interface. The remaining part is rejecting new changes.

We can indeed do an MPASS() check in the enqueue code to reject such code patterns if that's what we agree with.

What do you folks think?

So that seems like a sane general approach, but I'm not clear on how it'd work here. We'd have to make sure the ifp sticks around until the task queue no longer has any if_linktasks remaining, and I don't think we have a mechanism for that currently.

I think we have something in if.c:if_detach_internal() for that ( https://cgit.freebsd.org/src/tree/sys/net/if.c#n1145 ):

Address comments.

I love the part of this diff that creates the modular encap KPI, that's really awesome and I love to see this in base.
What I'm not sure about is the creation of compile-in options. I'm not sure I completely understand the rationale here. Flexibility is good, but I guess people compile out certain functionality either to save space or to address security concerns (or both). It would be nice to add more datapoints in that regard (for example, "disabling feature X gives Y kilobytes on amd64 kernel" or "feature X can be seen as unsecure b/c .." and there is no good/simple way to disabling it in the runtime other that compiling out). It also comes up with the maintenance cost attached - ifdefs, build system conditionals and more code variation to test.
On the implementation side, number of people use custom kernel configs. Landing these changes will make them automatically build kernels without IPv6 extensions and other encaps. This silent feature disappearing worries me, actually.

Thank you for working on this! Conceptually it LGTM, please see a comment on the implementation part.

D40356 was committed instead.

In D40348#918719, @markj wrote:

In D40348#918710, @melifaro wrote:

I just experienced a similar scenario and complained to melifaro that netlink stopped working in that scenario. The ideal situation would be that netlink would just malloc memory, or use its own dedicated zone rather than mbufs.

Netlink by itself is opaque to underlying memory type (and uses plain malloc for linux app buffers / cases when the requested netlink message size is > 2k). I'd also prefer not to use mbufs at all, but that's the current way of interacting with socket buffers. To switch from that, one needs to implement routines like soreceive_generic, which is gigantic 500-line function. I'm looking into the alternative approaches, but for now let's try to get maximum from the current implementation

It is possible to provide an alternate allocator for mbufs. m_free() will invoke a custom destructor for M_EXT mbufs, so you can bypass the normal UMA zones used for network packets.

Ack. Thank you for the hint, I've created D40356, implementing this approach. It looks better than the current fix, so I'd prefer to land the new diff instead.

In D40348#918709, @adrian wrote:

I think having a separate dedicated allocator for netlink messages is the better call. Don't make your control plane reliant on your data plane's memory management.

I just experienced a similar scenario and complained to melifaro that netlink stopped working in that scenario. The ideal situation would be that netlink would just malloc memory, or use its own dedicated zone rather than mbufs.

Netlink by itself is opaque to underlying memory type (and uses plain malloc for linux app buffers / cases when the requested netlink message size is > 2k). I'd also prefer not to use mbufs at all, but that's the current way of interacting with socket buffers. To switch from that, one needs to implement routines like soreceive_generic, which is gigantic 500-line function. I'm looking into the alternative approaches, but for now let's try to get maximum from the current implementation

No concerns from my side.

In other words: this patch cannot go in as-is (or at least not until the IFF_UP issues are addressed).

I ran into that when using netlink to configure interfaces. I guess we need to issue an eventhandler contification if some of interface flags were changed. I’ll take a look at that

I've raised D40332 to address lack of notifications to carp and the rest of the system.

Wow, that's nice!
LGTM, though I'd suggest trying to add structured output from day one. I managed to convert ndp(8) ( D35677 ), so if you could consider looking into that option, that would be awesome!

In D40317#917957, @kp wrote:

There's a rather annoying bug left in this. If we add a carp address on an interface that's down (i.e. IFF_UP is not set) the interface will come up during this, but remain in SUPPRESS state.

That's because the SIOCSIFADDR ioctl is passed to ether_ioctl(), which sets IFF_UP in ifp->if_flags directly, without any event handling. Ordinarily we'd call if_up(), which gives carp a chance to react to the interface coming up. That's not the case here, so it stays in SUPPRESS.

Initial attempts to fix this, by replacing ifp->if_flags |= IFF_UP in ether_ioctl() with if_up(ifp) resulted in ifconfig: ioctl (SIOCAIFADDR): File exists when trying to set an address on the interface. There's clearly an order of operations problem around this, but I don't fully understand it.

In other words: this patch cannot go in as-is (or at least not until the IFF_UP issues are addressed).

I ran into that when using netlink to configure interfaces. I guess we need to issue an eventhandler contification if some of interface flags were changed. I’ll take a look at that

I'm afraid I won't be able to review this in the foreseeable future. I think I completely forgot pre-nexthop routing logic.

Ping :-)
I'd love to get rid of if_clone_advanced() before 14 and cxgbe is the only remaining user :-)

I would like to try to have another iteration of the discussion :-)