Add a bit more accessors.
Move NETLINK_ROUTE specific helpers to a separate file.
Update the manpage.

In D37784#860073, @jhibbits wrote:

In D37784#860049, @melifaro wrote:

Q: What blockers prevent moving the accessors outside of if_var.h?

Nothing, really. Just doing this piecemeal. I noticed in some of my conversions that keeping the if_t typedef in if_var.h necessitated pulling it in where it wasn't needed beyond getting if_t, so pulled it out. The current users of struct ifnet beyond a forward declaration already include if_var.h so there's no gain or loss there.

Got it, make sense. Do you have any tentative timeline in mind for the stage when these accessors can be moved outside if_var.h ?

I'd actually spend some time removing if_var.h header from the inside the stack as well :-)

Maybe later. Some people may have concerns about performance in the stack by using the accessors inside the stack, so save that for a later time.

SURE. I'm sorry for not being detailed enough.
Currently, if_var.h is used in ~270 files in the base (grep -RI if_var.h sys | grep -v sys/dev | wc -l). The performance argument is an absolutely valid one, but it affects ~10-15 files out of these 270.
What I meant was cleaning the remaining 200+ files. I wasn't suggesting you should do it - sorry if it read that way. That's something I have had on my list for quite some time.

In D37783#860067, @emaste wrote:

Netlinks is a communication protocol ...

Netlink

currently used in Linux kernel to modify, read and subscribe for nearly all networking state.

Maybe we can specify RFC 3549 first? Sure it originated in Linux, but the fact that it has an RFC gives additional credence to implementing it, and enabling it by default, IMO.

I updated the description, thank you! I hope it reads better now.

Netlink support was added in D36002.

When committing this ought to be the git hash instead of original review

Sure!

Q: What blockers prevent moving the accessors outside of if_var.h?
I'd actually spend some time removing if_var.h header from the inside the stack as well :-)

• extend struct route with the custom "update_cache" callback pointer

How does this callback work?

• the callback task is to update the cached data in struct route in the "safe" fashion

I'm interested in the safe portion.

• struct route_cache can start with struct route and is followed by the mutex, which can be used as the sync mechanism
• PCB caching is generalised to this mechanism, so ip[6]_output() and ether_output() should call the callback instead of invalidating cache directly

So PCB caching employ struct route_cache rather than struct route[_in6] ?

What do you think?

First of all, I'd like to thank you for working on this. Caching routes/nexthops for the tunnels is a thing we should have in base and I'd really love to have it landed.
I like the part of the diff where you establish a clear KPI to register/unregister/use cached routes.

Update to use the snl(3) api.

I apologise for the belated response.
Thank you for addressing the comments!
The only remaining thing I'd prefer to discuss a bit more is the setcaps2 thing. I'd rather prefer to move these bits to a dedicated review. It would be easier to iron out KPIs around that (I can come up with the one if you're okay with it).
The rest I'm totally fine with - and I'd love to see that landed.

Also: thank you for the review & feedback! It helps greatly.

• Address comments.
• Update snl(3) library.

LGTM!
Do you think it would be possible to convert PR_IP to be static inline function, so that kind of erros are spotted by the compiler?

>> Some questions -

1. When some action happens (like, a user sets the MTU), then (a) driver-specific internal structures and HW state are updated, (b) actual ifp datastructed gets updated, and (c) network-stack-specific staff is triggered (eventhandler etc).

In the current KPI most of the if_set() functions do just (b), but there are examples of functions doing all 3 items (if_setlladdr()). Any thoughts on differentiating between these two classes?

I'm mixed on this. I think it requires further thinking of how drivers and the stack interact, and how other things interact with the stack. if_setlladdr() for instance looks like it needs to be hardware and stack oriented, to keep everything in sync. Unless we make the KPI a "set" and "commit" style. Thoughts on that direction?

It certanly requires thinking. Also, there are also interaction inside the current stack that should be considered w.r.t KPI changes. Specifically, I mean the change notifications that needs to be propagated to the other parts of kernel and/or userland programs.
Set&commit may bring some benefits. I don't see any way that wouldn' add significant complexity both to the underlying implementation and the KPI users. I don't think it's worth the effort.

2. Any thoughts on locking for setters?

Maybe for the next iteration of things? Right now just doing the naive approach to get the KPI in place and get drivers ported over.

Sure, I don't expect it to be the scope of this change - more curious abouth the thoughts on the general approach.

3. Any plans to update ifnet(9)? :-)

Definitely need to do that. Thanks for reminding me.

That's great!

In D37708#857530, @ngie wrote:

Thank you for the review!

Hot take: the PEP8 tools used by phabricator should be updated to ignore inline type hints. It might also be a good idea to update the config to work with black’s defaults (<=88 columns).

Would be really nice, though I'm not sure where do we store the configs.

address comments

In D37588#856868, @olivier wrote:

I could add this patch to the upcoming new 2.0.11 bird port. This version already includes the merged sysdep/linux/netlink.c and sysdep/cf/bsd-netlink.h (only missing sysdep/bsd-netlink/Makefile).

That would be awesome!

First of all, I fully support the ifnet KPI approach and love to see this (and the other relevant) reviews landed. Thank you for doing this!
Some questions -

1. When some action happens (like, a user sets the MTU), then (a) driver-specific internal structures and HW state are updated, (b) actual ifp datastructed gets updated, and (c) network-stack-specific staff is triggered (eventhandler etc).

In the current KPI most of the if_set() functions do just (b), but there are examples of functions doing all 3 items (if_setlladdr()). Any thoughts on differentiating between these two classes?

2. Any thoughts on locking for setters?

In D37574#854409, @bapt wrote:

In D37574#854398, @melifaro wrote:

Thank you for addressing the comments!
Added a couple of nitpicks that can be addressed at the commit time.

Q: what do you think of providing these notifications to each VNET? This can be addressed later, especially given the support should be implemented in the core netlink code. For now, I'm more curious about the concept.

I don't know, so far I don't see any message going through devctl_notify which should be vnet aware, but let's see how this get used in the future

Oh, sorry, let me be a bit more specific.
I'm talking about the case when a user wants to put some notification consumer to jail (for example, as a part of jail-service-by-default) with VNET. Currently, such a service won't be able to receive any notifications as it's not in the default VNET anymore.
So the questions is that if this behaviour is desired or not. If not, how should we proceed (have per-jail tunable to allow events passing etc) ?

Thank you for addressing the comments!
Added a couple of nitpicks that can be addressed at the commit time.

Generally LGTM!
Please see some minor comments inline.

.

In D37566#853879, @kp wrote:

Does this abstract enough to be worth it? Where is it intended to be used?

I have a larger diff that allows netlink to update interface properties, such as description. Given we already have if_freedescr(), I moved towards the if_allocdescr() instead of exposing M_IFDESCR.

Address (most of) the comments.

add manpage update.

Address the comments.

@dchagin: want to commit it, or should I do do it?

Thank you for providing the updated version w/ rib_walk!
Looks good to me, please see some minor comments inline. Once addressed, it should be good to land.

In D35881#817255, @olivier wrote:

yes the sleep is racy, but I have no idea to fix it.
Without it, the test will always fails: There is a delay between the jail destruction and the interface being visible back from the host.

So I've tested a loop like this one:

while jls -dj jifdestroy >/dev/null 2>&1; do
        sleep 1
done

with a dummy script on 13.1-release like this one:

#!/bin/sh
#set -eu
ifconfig lo888 create
jail -c name=bug persist vnet vnet.interface=lo888
jexec bug ifconfig lo888 up
jail -R bug
while jls -dj bug >/dev/null 2>&1; do
        echo "wait"
        sleep 1
done
ifconfig lo888 destroy && echo "success" || echo "fails"

And it fails about 1 time on 5 runs:

 # sh -x ./yo.sh
+ ifconfig lo888 create
+ jail -c 'name=bug' persist vnet 'vnet.interface=lo888'
+ jexec bug ifconfig lo888 up
+ jail -R bug
+ jls -dj bug
+ ifconfig lo888 destroy
ifconfig: interface lo888 does not exist
+ echo fails
fails

So the only stable solution seems the racy sleep here.

Checking that vnet is not dying is the right thing - once vnet is destroyed you should get the interface back. I’d prefer to have this logic in the test instead of a random sleep.
If this fails (on current head) I can take a look in a ~week time