Make pf more VNET safe

[bz]

Summary

Update pf(4) and pflog(4) to survive basic VNET testing, which includes
proper virtualisation, teardown, avoiding use-after-free, race conditions,
no longer creating a thread per VNET (which could easily be a couple of
thousand threads), handling global events (e.g., eventhandlers) on teardown,
clearing various globally cached pointers and checking them before use.

[robak]

I can't discuss the code, I can only test it and see if it makes the difference in system stability. Any chance for this to land in 11.0?

[bz]

In D6924#145160, @robak wrote:

I can't discuss the code, I can only test it and see if it makes the difference in system stability. Any chance for this to land in 11.0?

Yes, that's the plan. Needs all the testing it can get and I am still looking at pfsync and a possible very old security issue (don't run with untrusted customers in VNET yet) which I don't know if FreeBSD is vulnerable to.

[kp]

sys/netpfil/pf/pf.c

302 ↗

(On Diff #17762)

There's a '#define V_pf_end_threads VNET(pf_end_threads)' in net/pfvar.h.
Presumably you'll want to get rid of that too.

sys/netpfil/pf/pf_if.c

911 ↗

(On Diff #17762)

Minor typo here and in the others like it. 'expensie'.

sys/netpfil/pf/pf_ioctl.c

3782 ↗

(On Diff #17762)

When is this ever not true?
As far as I can tell you removed the only pf_end_threads++; from pf_purge_thread()

[bz]

Should all be addressed in the updated diff to come.

sys/netpfil/pf/pf_ioctl.c

3782 ↗

(On Diff #17762)

Good catch; that would be an endless loop...

[bz]

Address comments from @kristof .
Especially get rid of an endless loooCp on unload hopefully.p