What the current status of this review ? Wasn't it supposed to be committed at the end of August ?

switching a variable to const.

Tested with net/frr7 compiled with MULTIPATH option, and it is working great:

Tested on my ECMP lab (https://bsdrp.net/documentation/examples/ecmp#testing_load_balancing) and the flow-id load-balancing seems working great.
Would you like test using bird and FRR too ?

Look like the review contains a patch to file usr.bin/netstat/nhgrp.c that doesn't exist on -head

My jail/vnet burn test is still generating a panic:

In D26137#585268, @mjg wrote:

@olivier Thanks for the analysis. Could you share some pmc profiles from unhalted core cycles?

Comparing wireguard userland vs kernel module on a small device: +300% (from 117Mb/s to 483Mb/s)

• PC Engines APU2C4 (quad core AMD GX-412T Processor 1 GHz)
• 3 Intel i210AT Gigabit Ethernet ports
• FreeBSD 13-head r365033
• Wireguard kernel: D26137
• Wireguard userland: 1.0.20200827
• 2000 flows of UDP packets
• 500Bytes UDP load => packet size: 528B => Ethernet frame size:542B

Updating the patch to latest -head.

Following gleb's advice, use getsockname avoiding the random port generation part
I've still binded the port (I didn't understand the improvement brings by not binding it).

First result on a 'small' server Xeon E5 2650 8 cores with a 10G Chelsio T540-CR (one port used as RX and the other as TX) :

x r362778: inet4 packets-per-second forwarded
+ r362778 with D25454: inet4 packets-per-second forwarded
+--------------------------------------------------------------------------+
|                                                                         +|
|                                                                         +|
|                                                                         +|
|     x                                                                   +|
|x   xx  x                                                                +|
|  |__A_|                                                                  |
|                                                                         A|
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5      11767747      11782957      11777032      11776267     5465.3431
+   5      11905318      11905348      11905340      11905335     11.436783
Difference at 95.0% confidence
        129068 +/- 5636.28
        1.096% +/- 0.0483859%
        (Student's t, pooled s = 3864.59)

Fixing style and almost all other remarks.

Tested on igb and excellent result!

Once this patch applied to a current, my build kernel failed with a

Moving module load later.

Here are my result:

x r359647 in bridge mode: inet 4 packets forwarded per second
+ r359647 with D24250 in bridge mode: inet 4 packets forwarded per second
+--------------------------------------------------------------------------+
|xx                                                                      + |
|xx                                                                 +  + ++|
|A|                                                                        |
|                                                                     |_AM||
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5       3314595       3582601       3450403       3448858     99435.922
+   5      22726995      24346441      23963008      23715416     617156.45
Difference at 95.0% confidence
        2.02666e+07 +/- 644666
        587.631% +/- 27.5422%
        (Student's t, pooled s = 442024)

Keept the same behaviour as previously by loading the if_epair module.

In D23876#525342, @kp wrote:

This does change the behaviour of the test: if the if_epair module isn't loaded yet it doesn't load it, it skips the test.

Arguably that's better, because that's what we do for other modules (e.g. pf). On the other hand, other tests load it implicitly (e.g. the pf tests) in the process of making their test setup.
I'm torn on the best way to deal with this. Perhaps the best fix is to keep the kldload(), but to skip the test if the load fails rather than failing the test.

If we do go with this approach we should change freebsd-ci so that it loads if_epair (just as it already loads pf) for the automated tests.

In D23886#525161, @bz wrote:

@olivier do you have a non-forwarding TCP test-bed these days?

Thanks to their maintainers, the RC scripts of net/quagga, net/openbgpd and net/openbgpd6 are now publishing the "dynamicrouting" keyword too.

So, if I've correctly understood: