Page MenuHomeFreeBSD
Differential D58294

netlink: Add RTA_PREFSRC support
ClosedPublic
Actions

Authored by pouria on Fri, Jul 17, 12:14 PM.
Tags
None
Referenced Files
F164168445: D58294.id182737.diff
Wed, Jul 29, 7:49 AM
Unknown Object (File)
Mon, Jul 27, 8:23 AM
Unknown Object (File)
Sun, Jul 26, 10:23 PM
Unknown Object (File)
Sun, Jul 26, 12:17 PM
Unknown Object (File)
Sun, Jul 26, 11:30 AM
Unknown Object (File)
Sun, Jul 26, 11:10 AM
Unknown Object (File)
Sat, Jul 25, 10:58 PM
Unknown Object (File)
Sat, Jul 25, 10:07 AM
View All 60 Files
Subscribers
ae
imp
markj
zarychtam_plan-b.pwste.edu.pl
ziaee
zlei

Details

Reviewers
glebius
melifaro
ivy
bz
allanjude
ziaee
Group Reviewers
network
manpages
Commits
rGbe5994cdfbe2: netlink: Add RTA_PREFSRC support
rGdd235f097af4: route(8): Add prefsrc option in netlink
Summary

Add the ability to select source ip address of outgoing packets
even when the source ip address is configured on another interface.
Also add this new rtnetlink attribute to manual.

PR: 285422
Relnotes: yes

route(8): Add prefsrc option in netlink

Add prefsrc option that is frequently used on unnumbered interfaces
or L3 multi-homed network hosts.
This option uses RTA_PREFSRC.
Now you can add a static route by specifying the prefsrc option
with the loopback IP.

PR: 285422

Test Plan

This is an example for IPv4 over IPv6.

ifconfig lo0 inet 172.23.1.101/32 alias
route -n4 add 172.23.1.0/24 -inet6 -gateway fe80::5a9c:fcff:fe10:b89%vtnet0 -inet -prefsrc 172.23.1.101

tcpdump:

172.23.1.101 > 172.23.1.1: ICMP echo request, id 18696, seq 0, length 64

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 74875
Build 71758: arc lint + arc unit

Event Timeline

pouria created this revision.Fri, Jul 17, 12:14 PM
Herald added a subscriber: imp. · View Herald TranscriptFri, Jul 17, 12:14 PM
pouria requested review of this revision.Fri, Jul 17, 12:14 PM
Harbormaster completed remote builds in B74865: Diff 182110.Fri, Jul 17, 12:14 PM
ivy added a reviewer: allanjude.Fri, Jul 17, 12:14 PM
pouria added a comment.EditedFri, Jul 17, 12:15 PM

This is a WIP. I haven't run tests yet.
@ivy, how does it look?

pouria added inline comments.Fri, Jul 17, 12:19 PM
sbin/route/route.c
950

we can use ifa here too. but it may confuse people.

sys/netlink/route/route.h
148

We really don't support this:

% fgrep -r RTA_SRC sys
sys/netlink/route/route.h:      NL_RTA_SRC              = 2, /* not supported */
sys/netlink/route/route.h:#define RTA_SRC                       NL_RTA_SRC

That's why I change its comment.

203

This is needed for backward compatibility without breaking the rtsock.

pouria updated this revision to Diff 182111.Fri, Jul 17, 12:23 PM

Remove NL_DEBUG

Harbormaster completed remote builds in B74866: Diff 182111.Fri, Jul 17, 12:23 PM
pouria updated this revision to Diff 182112.Fri, Jul 17, 12:23 PM

Remove NL_DEBUG

Harbormaster completed remote builds in B74867: Diff 182112.Fri, Jul 17, 12:24 PM
ivy added a comment.Fri, Jul 17, 12:25 PM

i'll try to test this later today. is this compatible with the Linux netlink API? it would be nice if it worked out of the box with BIRD.

pouria updated this revision to Diff 182113.Fri, Jul 17, 12:25 PM

Oops, these days I'm trying to use git arc instead of arc and messed up :)

Harbormaster completed remote builds in B74868: Diff 182113.Fri, Jul 17, 12:25 PM
pouria updated this revision to Diff 182121.Fri, Jul 17, 1:39 PM

Fix bird by adding dump_prefsrc.
@ivy, it works now:

[root@ftsr1] [~] # fgrep -A4 -B4 prefsrc /usr/local/etc/bird.conf
protocol kernel kernel6 {
  ipv6 {
    import all;
    export filter {
      krt_prefsrc = fdb5:c59b:114e::a;
      accept;
    };
  };
}
--
protocol kernel kernel4 {
  ipv4 {
    import all;
    export filter {
      krt_prefsrc = 172.23.1.101;
      accept;
    };
  };
}
[root@ftsr1] [~] # ifconfig lo0 inet
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet 127.0.0.1 netmask 0xff000000
inet 172.23.1.101 netmask 0xffffffff
[root@ftsr1] [~] # ifconfig vtnet0 inet6 | grep fdb
inet6 fdb5:c59b:114e::a prefixlen 64
[root@ftsr1] [~] # birdc show ospf neigh
BIRD 3.2.0 ready.
ospf1:
Router ID       Pri          State      DTime   Interface  Router IP
100.64.10.101     1     Full/DR         30.313  epair0a    fe80::5a9c:fcff:fe10:1020

ospf2:
Router ID       Pri          State      DTime   Interface  Router IP
100.64.10.101     1     Full/PtP        30.314  epair0a    172.16.0.2
[root@ftsr1] [~] # netstat -rn4W | grep 192.168
192.168.0.0/24     172.16.0.2         UG          5   1500          epair0a      32
[root@ftsr1] [~] # netstat -on4W | grep 5
5            v4/gw 172.23.1.101       172.16.0.2         G             0   1500          epair0a                     2
[root@ftsr1] [~] # netstat -rn6W | grep 3fff
3fff:10::/64                      fe80::5a9c:fcff:fe10:1020%epair0a UG     11   1500          epair0a      32
[root@ftsr1] [~] # netstat -on6W | grep 11
11           v6/gw fdb5:c59b:114e::a             fe80::5a9c:fcff:fe10:1020%epair0a G         0   1500          epair0a                     2
Harbormaster completed remote builds in B74874: Diff 182121.Fri, Jul 17, 1:39 PM
pouria updated this revision to Diff 182122.Fri, Jul 17, 1:41 PM

Sorry for noise. I'm really used to arc.

Harbormaster completed remote builds in B74875: Diff 182122.Fri, Jul 17, 1:41 PM
pouria updated this revision to Diff 182123.Fri, Jul 17, 1:44 PM

Remove dump_prefsrc from mpnh.

Harbormaster completed remote builds in B74876: Diff 182123.Fri, Jul 17, 1:44 PM
zarychtam_plan-b.pwste.edu.pl added a subscriber: zarychtam_plan-b.pwste.edu.pl.Fri, Jul 17, 2:04 PM
pouria added a child revision: D58300: rtnetlink.4: Add RTA_PREFSRC.Fri, Jul 17, 2:11 PM
pouria mentioned this in D58300: rtnetlink.4: Add RTA_PREFSRC.Fri, Jul 17, 8:14 PM
pouria updated this revision to Diff 182158.Fri, Jul 17, 8:17 PM

Set prefsrc after setting the nh_ifp in non-multipath route.

Harbormaster completed remote builds in B74902: Diff 182158.Fri, Jul 17, 8:17 PM
pouria updated this revision to Diff 182161.Fri, Jul 17, 9:19 PM

Tests are passed.
Refactor by breaking prefsrc netlink logic into nl_set_nexthop_prefsrc() and setting the forgotten nh_aifp.
Now, I will wait for D58305, unless the consensus is to NOT limit the ability to set a prefsrc from another fib for VRF route leak.

Herald added a subscriber: ae. · View Herald TranscriptFri, Jul 17, 9:19 PM
Harbormaster completed remote builds in B74904: Diff 182161.Fri, Jul 17, 9:20 PM
ivy added a comment.Sat, Jul 18, 6:19 AM

i've done some basic testing with this. IPv4 seems to work with both route(8) and BIRD. IPv6 works with route(8) but not with BIRD:

protocol static {
        ipv6;

        route 2001:db8::/32 via 2a00:1098:6b:401::11 {
                krt_prefsrc = 2001:db8::10;
        };
}
Idx   Type         IFA                           Gateway                        Flags      Use Mtu               Netif           Addrif Refcnt Prepend
10           v6/gw fe80::5a9c:fcff:fe09:c9fd%vtnet0 2a00:1098:6b:401::11       G             0   1500           vtnet0                     2

setting krt_prefsrc in the kernel protocol doesn't make a difference. 2001:db8::10/128 is assigned to lo0 on the test system.

zarychtam_plan-b.pwste.edu.pl added a comment.Sat, Jul 18, 7:43 AM

I have been testing since yesterday. Now, together with D58305. There is nice progress. Thanks a lot for giving the userbase the opportunity to test this upcoming solution.

For routes assigned by hand, it works perfect when the "-prefsrc" address belongs to another interface, ie., loopback. If the "-prefsrc" address belongs to the same interface that the packet is originating from, and this address is an alias, it doesn't work - perhaps this solution is fine and fully acceptable. That's my opinion, though I am not reviewing this code.

BIRD indeed has a problem accepting IPv6 routes: "bird[9954]: nl_parse_attrs: Malformed attribute received".
Perhaps BIRD expects something else. IIRC, there were also problems with BIRD expecting an older API concerning FIB attribution in the past.

pouria added a comment.Sat, Jul 18, 8:12 AM
In D58294#1337455, @ivy wrote:

i've done some basic testing with this. IPv4 seems to work with both route(8) and BIRD. IPv6 works with route(8) but not with BIRD:

protocol static {
        ipv6;

        route 2001:db8::/32 via 2a00:1098:6b:401::11 {
                krt_prefsrc = 2001:db8::10;
        };
}
Idx   Type         IFA                           Gateway                        Flags      Use Mtu               Netif           Addrif Refcnt Prepend
10           v6/gw fe80::5a9c:fcff:fe09:c9fd%vtnet0 2a00:1098:6b:401::11       G             0   1500           vtnet0                     2

setting krt_prefsrc in the kernel protocol doesn't make a difference. 2001:db8::10/128 is assigned to lo0 on the test system.

Thank you for testing:

  1. Did you use the last version of this patch?
  2. Is there any " nl_parse_attrs: Malformed attribute received" in your bird logs? (only if you're using the last version of this patch)

I've verified that this patch works with bird and I sent its output here earlier.
But, if it doesn't work for you, let me know. (Make sure you're using the last version)

In D58294#1337457, @zarychtam_plan-b.pwste.edu.pl wrote:

I have been testing since yesterday. Now, together with D58305. There is nice progress. Thanks a lot for giving the userbase the opportunity to test this upcoming solution.

For routes assigned by hand, it works perfect when the "-prefsrc" address belongs to another interface, ie., loopback. If the "-prefsrc" address belongs to the same interface that the packet is originating from, and this address is an alias, it doesn't work - perhaps this solution is fine and fully acceptable. That's my opinion, though I am not reviewing this code.

It should accept prefsrc of another interface.
If that doesn't work, It's not intended and we should fix that.

BIRD indeed has a problem accepting IPv6 routes: "bird[9954]: nl_parse_attrs: Malformed attribute received".

I saw this error and fixed it last night.
Did you use the last version?

ivy added a comment.Sat, Jul 18, 9:03 AM
In D58294#1337460, @pouria wrote:
  1. Did you use the last version of this patch?

yes, i applied it this morning with git arc.

  1. Is there any " nl_parse_attrs: Malformed attribute received" in your bird logs? (only if you're using the last version of this patch)

no.

the full BIRD log:

2026-07-18 10:00:00.355 <TRACE> device1: Initializing
2026-07-18 10:00:00.355 <TRACE> direct1.ipv4: Connected to table master4
2026-07-18 10:00:00.355 <TRACE> direct1.ipv6: Connected to table master6
2026-07-18 10:00:00.355 <TRACE> direct1: Initializing [disabled]
2026-07-18 10:00:00.355 <TRACE> kernel1.ipv4: Connected to table master4
2026-07-18 10:00:00.355 <TRACE> kernel1: Initializing
2026-07-18 10:00:00.355 <TRACE> kernel2.ipv6: Connected to table master6
2026-07-18 10:00:00.355 <TRACE> kernel2: Initializing
2026-07-18 10:00:00.355 <TRACE> static1.ipv4: Connected to table master4
2026-07-18 10:00:00.355 <TRACE> static1: Initializing
2026-07-18 10:00:00.355 <TRACE> static2.ipv6: Connected to table master6
2026-07-18 10:00:00.355 <TRACE> static2: Initializing
2026-07-18 10:00:00.355 <TRACE> device1: Starting
2026-07-18 10:00:00.355 <TRACE> device1: Scanning interfaces
2026-07-18 10:00:00.355 <TRACE> device1: State changed to up
2026-07-18 10:00:00.355 <TRACE> kernel1: Starting
2026-07-18 10:00:00.355 <TRACE> kernel1.ipv4: State changed to UP/FEEDING
2026-07-18 10:00:00.355 <TRACE> kernel1: State changed to up
2026-07-18 10:00:00.355 <TRACE> kernel2: Starting
2026-07-18 10:00:00.355 <TRACE> kernel2.ipv6: State changed to UP/FEEDING
2026-07-18 10:00:00.355 <TRACE> kernel2: State changed to up
2026-07-18 10:00:00.355 <TRACE> static1: Starting
2026-07-18 10:00:00.355 <TRACE> static1.ipv4: State changed to UP
2026-07-18 10:00:00.355 <TRACE> static1: State changed to up
2026-07-18 10:00:00.355 <TRACE> static1.ipv4 > added [best] 10.0.0.0/24 0L 4G unicast
2026-07-18 10:00:00.355 <TRACE> kernel1.ipv4 < added 10.0.0.0/24 0L 4G unicast
2026-07-18 10:00:00.355 <TRACE> static2: Starting
2026-07-18 10:00:00.355 <TRACE> static2.ipv6: State changed to UP
2026-07-18 10:00:00.355 <TRACE> static2: State changed to up
2026-07-18 10:00:00.355 <TRACE> static2.ipv6 > added [best] 2001:db8::/32 0L 5G unicast
2026-07-18 10:00:00.355 <TRACE> kernel2.ipv6 < added 2001:db8::/32 0L 5G unicast
2026-07-18 10:00:00.355 <INFO> Started
2026-07-18 10:00:00.356 <TRACE> kernel1 < interface vtnet0 goes up
2026-07-18 10:00:00.356 <TRACE> kernel1 < interface lo0 goes up
2026-07-18 10:00:00.356 <TRACE> kernel1 < interface bridge0 goes up
2026-07-18 10:00:00.356 <TRACE> kernel1 < interface epair0a created
2026-07-18 10:00:00.356 <TRACE> kernel1 < interface epair0b created
2026-07-18 10:00:00.356 <TRACE> kernel2 < interface vtnet0 goes up
2026-07-18 10:00:00.356 <TRACE> kernel2 < interface lo0 goes up
2026-07-18 10:00:00.356 <TRACE> kernel2 < interface bridge0 goes up
2026-07-18 10:00:00.356 <TRACE> kernel2 < interface epair0a created
2026-07-18 10:00:00.356 <TRACE> kernel2 < interface epair0b created
2026-07-18 10:00:00.356 <TRACE> kernel1.ipv4 < replaced 10.0.0.0/24 0L 4G unicast
2026-07-18 10:00:00.356 <TRACE> kernel1.ipv4: State changed to UP/READY
2026-07-18 10:00:00.356 <TRACE> kernel2.ipv6 < replaced 2001:db8::/32 0L 5G unicast
2026-07-18 10:00:00.356 <TRACE> kernel2.ipv6: State changed to UP/READY
2026-07-18 10:00:00.356 <TRACE> kernel2: Scanning routing table
2026-07-18 10:00:00.356 <TRACE> kernel2: ::/96: [alien] ignored
2026-07-18 10:00:00.356 <TRACE> kernel2: ::/0: [alien] ignored
2026-07-18 10:00:00.356 <TRACE> kernel2: 2001:db8::10/128: [alien] ignored
2026-07-18 10:00:00.356 <TRACE> kernel2: 2a00:1098:6b:401::/64: ignored
2026-07-18 10:00:00.356 <TRACE> kernel2: 2a00:1098:6b:401::16/128: [alien] ignored
2026-07-18 10:00:00.356 <TRACE> kernel2: Pruning table master6
2026-07-18 10:00:00.356 <TRACE> kernel2: 2001:db8::/32: installing
2026-07-18 10:00:00.356 <TRACE> kernel1: Scanning routing table
2026-07-18 10:00:00.356 <TRACE> kernel1: 10.0.0.0/24: updating
2026-07-18 10:00:00.356 <TRACE> kernel1: 10.1.1.1/32: ignored
2026-07-18 10:00:00.356 <TRACE> kernel1: Pruning table master4

in case it matters, the interface i'm testing with has an IPv6 address assigned:

vtnet0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=880028<VLAN_MTU,JUMBO_MTU,LINKSTATE,HWSTATS>
        ether 58:9c:fc:09:c9:fd
        inet6 fe80::5a9c:fcff:fe09:c9fd%vtnet0 prefixlen 64 scopeid 0x1
        inet6 2a00:1098:6b:401::16 prefixlen 64
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=821<PERFORMNUD,AUTO_LINKLOCAL,STABLEADDR>
pouria updated this revision to Diff 182171.Sat, Jul 18, 12:14 PM

In contrast to ifa_ifwithnet the ifaof_ifpforaddr returns an ifa even if it's not appropriate for prefsrc.
Fix when the interface is specified (which is usually done by ospf) by removing ifaof_ifpforaddr and use ifa_ifwithaddr_fib instead.
Parent: D58305

Harbormaster completed remote builds in B74911: Diff 182171.Sat, Jul 18, 12:14 PM
pouria added a parent revision: D58305: net/if.c: Add fib-aware ifa_ifwithaddr().Sat, Jul 18, 12:15 PM

This should fix it.
Could you please test it again? @ivy @zarychtam_plan-b.pwste.edu.pl

ivy added a comment.Sat, Jul 18, 1:10 PM

with D58305 + the new version of D58294, everything seems to be working fine, including with BIRD. (i only did cursory testing, though.)

pouria added a comment.Sat, Jul 18, 1:39 PM
In D58294#1337514, @ivy wrote:

with D58305 + the new version of D58294, everything seems to be working fine, including with BIRD. (i only did cursory testing, though.)

Let's also wait for @zarychtam_plan-b.pwste.edu.pl results and it should be ready to review.
I'll ping reviewers again after the results.
I also should write a test for it. Will let you know.

ivy added a comment.Sat, Jul 18, 2:04 PM

by the way, there seems to be a typo in the review title :-) (RTA_PERFSRC)

pouria retitled this revision from netlink: Add RTA_PERFSRC support to netlink: Add RTA_PREFSRC support.Sat, Jul 18, 2:05 PM
pouria added a child revision: D58326: routing: Add tests for prefsrc route attribute.Sat, Jul 18, 2:48 PM
zarychtam_plan-b.pwste.edu.pl added a comment.Sat, Jul 18, 5:53 PM

I can confirm that the configuration below works flawlessly in BIRD:

protocol ospf v3 testing_ospf {
    ipv6 {
        import filter {
            krt_prefsrc = 2001:db8:abc::1;
            ...

Dear reviewers, please consider this enhancement to be a bug fix, as it aligns BIRD's behaviour with Linux with respect to the krt_prefsrc directive.

Excellent work, and thank you for providing the opportunity to test this functionality!

pouria added a subscriber: markj.Sat, Jul 18, 6:18 PM

This revision is ready to review.
kind ping :D
@glebius @markj @melifaro

zlei added a subscriber: zlei.Mon, Jul 20, 2:59 AM

Nice feature!

pouria added a comment.Thu, Jul 23, 8:53 AM

Even a C-wise review would help!

ivy added inline comments.Thu, Jul 23, 9:12 AM
sys/netlink/route/rt.c
170–173
ivy added a comment.Thu, Jul 23, 9:16 AM

i can try backporting this to 15.0 (or maybe 15.1) and running it on a couple of BIRD routers here to see how that goes. that will take a few days though.

pouria updated this revision to Diff 182495.Thu, Jul 23, 9:21 AM
pouria marked an inline comment as done.

Address @ivy comment.
Thank you!

Harbormaster completed remote builds in B75033: Diff 182495.Thu, Jul 23, 9:21 AM
pouria updated this revision to Diff 182737.Sun, Jul 26, 7:58 PM

Despite limited size of nh_flags, we really need to know if prefsrc is static or not.
Add NHF_PREFSRC for nexthop with static ifa.
Dump netlink RTA_PREFSRC attr and override nh_aifp only if NHF_PREFSRC exists.

Tests are passed.

Harbormaster completed remote builds in B75153: Diff 182737.Sun, Jul 26, 7:58 PM
glebius accepted this revision as: glebius.Tue, Jul 28, 3:13 AM

Thanks a lot for this project!

This revision is now accepted and ready to land.Tue, Jul 28, 3:13 AM
pouria updated this revision to Diff 182917.Tue, Jul 28, 8:25 AM

Address @glebius comment on merging D58300.

This revision now requires review to proceed.Tue, Jul 28, 8:25 AM
Herald added a subscriber: ziaee. · View Herald TranscriptTue, Jul 28, 8:25 AM
Harbormaster completed remote builds in B75228: Diff 182917.Tue, Jul 28, 8:25 AM
pouria edited the summary of this revision. (Show Details)Tue, Jul 28, 8:36 AM
ziaee accepted this revision.Tue, Jul 28, 8:15 PM
ziaee added a reviewer: manpages.

Manpage LGTM, and I like that you folded it into the revision, no comment on the anything else.

This revision is now accepted and ready to land.Tue, Jul 28, 8:16 PM
Closed by commit rGdd235f097af4: route(8): Add prefsrc option in netlink (authored by pouria). · Explain WhyTue, Jul 28, 8:58 PM
This revision was automatically updated to reflect the committed changes.
pouria added a commit: rGdd235f097af4: route(8): Add prefsrc option in netlink.
pouria added a commit: rGbe5994cdfbe2: netlink: Add RTA_PREFSRC support.

Revision Contents
Changeset List

PathSize
sbin/
route/
1 line
1 line
2 lines
sys/
netlink/
2 lines
route/
5 lines
51 lines

Diff 182122

View Options

sbin/route/keywords

Loading...
View Options

sbin/route/route.c

Loading...
View Options

sbin/route/route_netlink.c

Loading...
View Options

sys/netlink/netlink_snl_route_parsers.h

Loading...
View Options

sys/netlink/route/route.h

Loading...
View Options

sys/netlink/route/rt.c

Loading...