pf: Do not hold PF_RULES_RLOCK while processing Ethernet rules

[kp]

Summary

Avoid the overhead of acquiring a (read) RULES lock when processing the
Ethernet rules.
We can get away with that because when rules are modified they're staged
in V_pf_keth_inactive. We take care to ensure the swap to V_pf_keth is
atomic, so that pf_test_eth_rule() always sees either the old rules, or
the new ruleset.

We need to take care not to delete the old ruleset until we're sure no
pf_test_eth_rule() is still running with those. We accomplish that by
using NET_EPOCH_CALL() to actually free the old rules.

Sponsored by: Rubicon Communications, LLC ("Netgate")

[kp]

Rebase

[kp]

Rebase & cleanup

[mjg]

Why the use of ck_* instead of atomic_*?

[kp]

In D31739#776252, @mjg wrote:

Why the use of ck_* instead of atomic_*?

That'd probably work too, but we typically use the ck_* functions in combination with epoch. For example, in ifnet_byindex() or prison_ip_check().