With the new FIB_ALGO infrastructure, nearly all subsystems use fib[46]_lookup() function, which provide lockless lookups.
However, FIB_ALGO makes some tradeoffs, resulting in (relatively) prolonged periods of holding RIB_WLOCK. If the lock is held, and datapath competes for it, the RX ring may get blocked, ending in traffic delays and losses.
As currently arp processing is performed directly, RIB-backed lookups (rib_lookup_info()) require RIB_RLOCK, which triggers the problem described above when the amount of ARP replies is high.
To be more specific, prior to creating new ARP entry, the following check are executed:
- Routing lookup for the entry address in interface fib. If lookup return empty result, or the resulting prefix is non-directly-reachable, failure is returned. The only exception are host routes w/ gateway==address.
- If the routing lookup returns different interface and non-host route, we want to support the use case, when there are multiple interfaces with the same prefix. In fact, we just check if the current returned prefix covers our address (always true) and effectively allow allocating ARP entries for any directly-reachable prefix, regardless of its interface.
First check does not require knowledge of an actual prefix, nexthop flags
provide enough information to covert the logic as is. The second check is
more tricky. Following the intent, check per-interface addresses for the
matching prefix covering the target address.