Well, this is how it was in ipfilter before. Of course this looks very much like a bug, but fixing ipfilter out of scope of this commit.

Again, this is how ipfilter does now. I'm all for improving it, but out of scope of this work.

The function converts from a memptr to mbuf, so it creates a fake a mbuf, so that's why the name. Naming it pfil_memptr, IMHO, will be misleading.

I'd prefer to keep single synopsis line if it is possible. Is my syntax legit?

No idea which is better :)

The proper form would be:
At least one of
.Fl i
or
.Fl o
is required.

The and is the wrong joiner, and "flags" is not needed. Could also be written as
Either
.Fl i
or
.Fl o
must be present.

I think, you planned to use PFIL_EPOCH macro here in epoch_enter_preempt() and epoch_exit_preempt() :)

maybe it would be better to use MAXMODNAME from <sys/module.h>?

Maybe we can reserve a bit more bytes for this name to be able store some meaning descriptions here?
Later it will be not so easy to extend the size.

s/ /\t/

It seems there is not any difference between DROPPED and CONSUMED. Also, note that the code that calls pfil_run_hooks() expects that the return value is an error from errno(2), where 1 corresponds to EPERM, but another values can be passed to userlevel and ENOENT=2, ESRCH=3 can confuse. If you want to change this protocol, you need to inspect the code around the pfil_run_hook().

I could be wrong, but it seems it is unsafe to just free() memory in pfil_head_unregister(). What if in the same time some packet will be handled by pfil_run_hooks()?

I think each interface can register own pfil head and on detach/destroy interface should unregister it (it looks like you missed this part in mlx5en). It looks quite possible, that on high pps trough interface pfil_head_unregister() can be invoked in the same time that pfil_run_hooks(). And pfil_run_hooks() can already get link pointer, so when you do free() to this pointer, dereferencing link in the pfil_run_hooks() will lead to page fault.

Doesn't really matter, since future plan is to assert net_epoch in pfil, not enter it.

No, it is responsibility of a subsystem to guarantee that pfil_head_unregister is called only when packet flow has completely stopped. This was the case before my changes and this remains as is.

P.S. Yes, mlx5en patch lacks unregistering.

Would require including sys/module.h into userland utility to manipulate pfil(9), which is ugly, since purpose of sys/module.h is completely different.

I created MAXPFILNAME equal to 64. This is longest name of a pf queue. In ipfw we don't have long string constants yet. 64 should be enough for anyone :)

We can't go too big, e.g. MAXPATHLEN, because pfilioc_link is used internally in the kernel.

Will double check that.

We can't change ip to ip6_hdr, at least not yet. ipf_check defines the second argument as ip. Later in ipf_check it casts ip to ip6_t if the version is 6. It's another thing I need to sift through.

This looks like unintended change. dir variable keeps internal ipfw's value here, and it has different value than PFIL_IN/OUT.

can be removed.

DIR_IN != PFIL_IN

This line is wrongly placed, the code is unreachable.

redundant line.

can be removed.

return (PFILL_CONSUMED);

ret = PFIL_CONSUMED;

ret != PFIL_PASS

Thanks a lot! A mismerge, when I rebased over your ipfw improvements.

Small nit, if you can either drop the All rights reserved (no longer needed) or if you want it could you fit it on the line with the copyright so that someone does not insert a copyright between yours and it. Ditto in other places, I just picked one.

Thanks. Should I keep the line where it existed before?

I think this is the place your talking about keeping it. Yes, you need to keep this as it looks like Matthew R Green put it there, and it would have to be him to remove it. You could fold it up onto the same line as his copyright, making it clear from here forward that it was him that asserted this, but there is not a need to.

By the way, your adding your copyright a good way, above the other so it doesnt seperate his copyright from his all rights reserved, thank you for this detail!