routing: remove ROUTE_MPATH compile option

[pouria]

Summary

The ROUTE_MPATH compile option was introduced to
test the new multipath implementation.
Since compiling it has no overhead, and it's enabled
by default.
Remove it to reduce routing complexity.

Test Plan

Run all network related tests:

kyua test -k /usr/tests/Kyuafile sys/net
kyua test -k /usr/tests/Kyuafile sys/netlink
kyua test -k /usr/tests/Kyuafile sys/netinet
kyua test -k /usr/tests/Kyuafile sys/netinet6
kyua test -k /usr/tests/Kyuafile sys/netgraph

[pouria]

I strongly believe this option is useless now and it just increase complexity of the code.
multipath is part of routing, making it optional is just like making arp/ndp optional in ipv4/ipv6.
the original reason to have this option is no longer valid.
Also, as noted in the d5fe384b4d41a5c5adeaa6039d33d59d7bc33e76, ROUTE_MPATH does not consume memory unless actually used.

[bms]

+1 vote for just doing this, all the RIBs built on top of modern FreeBSD FIB need to change with it anyway.

[glebius]

sys/net/route.h

127–152

Glad this is cleanup up (modulo RSS). In general putting config(8) derived options into headers installed into /usr/include is a land mine.

[pouria]

sys/net/route.h

127–152

This part was my original motiviation :)
See PR293136

[zlei]

The ROUTE_MPATH still has subtle performance impact and the kernel built with this option has larger text size. I'd expect some people want to keep this option so they can disable it.

Another option is VIMAGE which has little performance impact. The feature is stable enough, but still there're cases people want to disable it entirely.

[zlei]

I'm for reduce routing complexity, if it is possible to keep that option.

[glebius]

In D55884#1278650, @zlei wrote:

The ROUTE_MPATH still has subtle performance impact and the kernel built with this option has larger text size. I'd expect some people want to keep this option so they can disable it.

What is the performance impact? AFAIU, it is just array of fibs that by default has only one element.

[zlei]

In D55884#1278698, @glebius wrote:

In D55884#1278650, @zlei wrote:

The ROUTE_MPATH still has subtle performance impact and the kernel built with this option has larger text size. I'd expect some people want to keep this option so they can disable it.

What is the performance impact? AFAIU, it is just array of fibs that by default has only one element.

I re-checked the fast path, fib4_lookup() and fib6_lookup(), I think you're right. The last time I do benchmark on the route lookup is years ago, I might misremember and messed up ROUTE_MPATH with other options.

[pouria]

remove opt_route.h from files below:
sys/net/route/nhgrp.c
sys/net/route/nhgrp_ctl.c
sys/net/route/nhop.c
sys/net/route/nhop_ctl.c
sys/net/route/nhop_utils.c
sys/net/route/route_rtentry.c
sys/net/route/route_subscription.c
sys/netgraph/netflow/netflow.c
sys/netgraph/netflow/netflow_v9.c
sys/netinet/in_pcb.c
sys/netinet/ip_input.c
sys/netinet/ip_output.c
sys/netinet/raw_ip.c
sys/netinet/udp_usrreq.c
sys/netinet6/in6_proto.c
sys/netinet6/ip6_input.c
sys/netinet6/ip6_output.c
sys/netinet6/raw_ip6.c
sys/netinet6/udp6_usrreq.c
sys/netlink/route/nexthop.c

Still some files exists with opt_route.h included:
sys/modules/netlink/Makefile:SRCS+= opt_inet.h opt_inet6.h opt_route.h
sys/modules/netgraph/netflow/Makefile:SRCS= opt_route.h
sys/netlink/route/rt.c:#include "opt_route.h"
sys/net/rtsock.c:#include "opt_route.h"
sys/net/route.c:#include "opt_route.h"
sys/net/route/route_tables.c:#include "opt_route.h"
sys/net/route/route_helpers.c:#include "opt_route.h"
sys/net/route/fib_algo.c:#include "opt_route.h"
sys/net/route/route_ctl.c:#include "opt_route.h"
sys/net/route/route_ifaddrs.c:#include "opt_route.h"
sys/netgraph/netflow/ng_netflow.c:#include "opt_route.h"
sys/netinet/in_fib.c:#include "opt_route.h"
sys/netinet6/in6_fib.c:#include "opt_route.h"
sys/netinet6/in6_pcb.c:#include "opt_route.h"

Those are probably required by two options below:
sys/conf/options:ROUTETABLES opt_route.h
sys/conf/options:FIB_ALGO opt_route.h

But I'm not sure about ROUTETABLES, so I didn't remove from them.
Why?
because compiling works, even by removing opt_route.h from all other files except
for the files below:
sys/net/route/fib_algo.c:#include "opt_route.h"
sys/netinet/in_fib.c:#include "opt_route.h"
sys/netinet6/in6_fib.c:#include "opt_route.h"

Not sure why!