pf_source_used() puts a source in the table of its source limiter when
the source has more states than the high-water mark, and
pf_source_rele() takes it out again below the low-water mark. Both
changed the table in place: pf_source_used() from pf_create_state(),
with the rules lock held for reading, and pf_source_rele() from
pf_remove_state(), with no rules lock at all. A table may only be
changed with the rules lock held for writing, as packets look addresses
up in its radix tree at the same time. With INVARIANTS the first
insertion panics ("Lock pf rulesets not exclusively locked" in
pfr_route_kentry()); without, the tree can be corrupted.
Queue the changes instead, as pf_overload_task() does for overload
tables, and make them from a task with the rules lock held for writing.
An entry names the limiter by its id and remembers its table: the
limiter, or its table, may be gone or replaced by the time the task
runs, and then the entry is dropped. pfsr_intable changes only once
the change is queued, so a failed allocation is retried the next time
the source crosses a mark; a removal is also retried when the source is
purged. Drain the task when a vnet goes away, after its last state.
Add a regression test.
Fixes: 461648121230 ("pf: introduce source and state limiters")
Sponsored by: Rubicon Communications, LLC ("Netgate")