Sorry for the mess, my latest actions on this differential were unintentional, some problems with old Firefox.

CCing Roman Kurakin (ce(4)) and Serge Vakulenko (cp(4)), see later.

I'm going to commit this soon unless an objection is raised.

My goal is to improve performance by default not requiring any user actions while not breaking things same time. Both GNU binutils version of strip(1) and our version from contrib/elftoolchain/elfcopy support -o, so there would be no slowdown but improvement for all supported branches. However, install(1) could be used in some strange environment for cases other than "installworld" and I don't want it to break there making regression.

Thanks, I've fixed this.

Use: strip -o to_name -- from_name

Thanks!

Correct wording in the manual page.

I'm going to commit this soon unless an objection is raised.

In D25029#552845, @lutz_donnerhacke.de wrote:

If I understand correctly

ifconfig -a -g lagg -G lagg*1

will match all lagg interfaces besides those ending in 1.

But

ifconfig -a -g lagg -g vlan

will match vlan interfaces only.

Correct?

Add explicit note to the manual page that additional options -g override previous ones, same for -G.

Fix case when none of -g nor -G is specified that was broken with addition of second option.

Do not use prefix to negate condition. Instead, use new flag -G and allow user to specify both -g and -G same time.

In D25029#552131, @lutz_donnerhacke.de wrote:

"^" does look like an regex starting point and will cause confusion: "Why is '-g la.*2[345]' not working?"

In D25029#551705, @ae wrote:

You can just use another option name to specify excludes.

Compactify code a bit.

Fix misprint that somehow passed testing.

Kernel side asprintf() uses M_NOWAIT internally and may return -1 on memory allocation failure. It would need a check.

Please consider to decrease abuse of kernel stack. We already had problems with kernel-land double faults due to kernel thread stack overflows. Could you please aggregate these big but fixed-sized arrays to single struct and allocate it using one malloc() call at heap instead?

In D24416#537135, @kp wrote:

I can't create epair0 again even though it no longer exists in the host vnet.

In D24416#537151, @wollman wrote:

I haven't been paying attention to this, but IME the problem with epair (and also bridge) interfaces was always that the generator would come up with the same MAC address on different *machines*. We simply gave up and manually assigned LAA MAC addresses to all such interfaces.

You still miss the point. Please read PR 229957 there all epairs are created within host, so they have same UUID. But after epair is created and MAC generated, epair get moved to another VNET, so next one may be created within host with same properties and get same MAC address.

Sadly, ether_gen_addr() cannot be used for epair safely. Your change removes parts of code that was brought in for a reason. This change would re-introduce problems fixed by code removed. Please read PRs 184149 and 229957 for details.

Also, trim(8) calls candelete() for cosmetic reasons only and in verbose mode only (by default).
"trim -q" does not call candelete() and always reveals real error of system call.

This seems to be driver-dependent as it works as expected for md:

I'm not sure I understand the problem.

In D23577#519163, @aleksandr.fedorov_itglobal.com wrote:

divert sockets can be used with other software but present exactly same significant overhead.
Do you have an example when suggested change improves performance really?

I agree with you that divert sockets have a known overhead. And I don't have a suitable example, because I don't use them anywhere. On the other hand, I see no reason why the initial socket buffer size should be hardcoded. My experience with other types of sockets indicates that large buffers can increase throughput. For example, netgraph sockets.

So, if some people want to use divert sockets, why not add the ability to resize the buffer socket?

In D23577#520228, @lutz_donnerhacke.de wrote:

In D23577#520226, @eugen_grosbein.net wrote:

In D23577#519182, @lutz_donnerhacke.de wrote:

Can you please explain, what the issue is with the sysctl itself?

Sysctls are great tools and very handy, so our sysctl tree grows quick and already bloated and needs increasing amounts of memory. I don't think we should add new one just because it's easy and we can do it, without any practical use case.

So how implementing an (existing) socketopt to modify the desired behavior from the application side?

In D23577#519182, @lutz_donnerhacke.de wrote:

Can you please explain, what the issue is with the sysctl itself?

In D23577#519163, @aleksandr.fedorov_itglobal.com wrote:

divert sockets can be used with other software but present exactly same significant overhead.
Do you have an example when suggested change improves performance really?

I agree with you that divert sockets have a known overhead. And I don't have a suitable example, because I don't use them anywhere. On the other hand, I see no reason why the initial socket buffer size should be hardcoded. My experience with other types of sockets indicates that large buffers can increase throughput. For example, netgraph sockets.

So, if some people want to use divert sockets, why not add the ability to resize the buffer socket?

In D23577#518798, @aleksandr.fedorov_itglobal.com wrote:

divert(4) sockets can be used not only with natd(8), so the changes looks reasonable for me.

In D23577#518767, @neel_neelc.org wrote:

In D23577#518755, @eugen_grosbein.net wrote:

Please describe use case for this change. Also, defauls are not 65536 but (65536 + 100).

This can be used to reduce divert memory consumption on "small" devices (e.g. low power routers) or increase divert performance on "big" devices (e.g. middleboxes, IDS).

I updated the description.

Please describe use case for this change. Also, defauls are not 65536 but (65536 + 100).

In D23091#518544, @lutz_donnerhacke.de wrote:

@eugen_grosbein.net are your concerns handled?

We already have "nat tablearg" feature for this task:

In D23450#515674, @pi wrote:

What happens, if some port mapping is defined unexpected source IPs are sent from behind the CGN ? Will this packet/port end up 'somewhere' in the reserved range or will it map somewhere outside of all reserved ranges ?

In D23450#515673, @pi wrote:

The use case is a static mapping from behind-nat users to the outside world. If such a static mapping is possible, CGN-operators do not need to log every transaction.

Please describe why do you need to limit aliasing port rangle for NAT instance and what is wrong with current unlimited and mixed usage of ports for NAT instances?

In D23091#507459, @lutz_donnerhacke.de wrote:

In D23091#507447, @lutz_donnerhacke.de wrote:

Add an explanation to the man page.

Could you please supply use case? Why do we need another virtual ethernet interface? We already have many kinds of them.

In D21724#477488, @kaktus wrote:

Wouldn't /usr/libexec be a better place for it, like for many other such daemons (like fingerd etc…)?

Sometimes I suffer from being unable to bring the interface to "administratively shutdown" state as opposed to "operative shutdown". It would be nice if ifconfig(8) "down" was able to update status of the interface with "admindown" string or similar, and "up" was able to auto-remove such note.

In D21306#464409, @driesm.michiels_gmail.com wrote:

Thanks for the feedback Eugen, that will take a bit more time to restructure / add some of your comments.

I have one remark regarding 1), rc.firewall creates the NAT rule at number 50, although I think the example in the handbook and the ruleset rc.firewall creates are different. rc.firewall only has one NAT rule and does not have all stateful rules. So I don't feel too strong about renumbering the existing ruleset in the handbook just to match the NAT rule at number 50.

There is also one subtle difference between "ipfw divert" command used with natd and "ipfw nat" command.

Thank you very much for starting this work. We really need updates to the Handbook.

I like the idea.

In D18382#391348, @sobomax wrote:

o Due to popular request rename "erase" into "trim".

That makes sense, thanks.

In D18546#395642, @sobomax wrote:

In D18546#395596, @sobomax wrote:

In D18546#395445, @eugen_grosbein.net wrote:

Hmm, there was https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229432 and corresponding commit https://svnweb.freebsd.org/base?view=revision&revision=336195 fixing the problem.

Was it insufficient or your tree does not have that fix?

Yes, you are right. We were looking at the FreeBSD 11.2 code here, I have not noticed there is another change in a trunk to fix the same issue. :(

Hmm, there was https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229432 and corresponding commit https://svnweb.freebsd.org/base?view=revision&revision=336195 fixing the problem.

In D18535#395370, @sobomax wrote:

In D18535#395273, @eugen_grosbein.net wrote:

Yes, it requires corruption of private node memory. As owner of multiple routers mass servicing thousands of customers using multiple NETGRAPH nodes I can assure you that panic is not appropriatie action. Appropriate action is some form of block for traffic flow trough the node in question (with logging) leaving other nodes working.

Well, that's where I respectively disagree. As an owner of hundreds of FreeBSD systems servicing many millions of customers I think that rebooting the system immediately after any slight kernel heap/stack memory corruption is detected is not just appropriate but the only sane action available. Shutting down particular netgraph node and hope for the best would just leave the service down indefinitely with no hope for any sorts of automatic recovery.

Looks good with one exception: additional plain panic(). Can it be replaced with KASSERT?

IDK, there is no way this option to be set to anything but DLT_RAW or DLT_EN10MB in the course of normal operation of the node. So it would require some form of memory corruption to actually happen. IDK, panic(9) seems an appropriate action in that case. There are other panic(9) call in the code in similar situations.

Looks good with one exception: additional plain panic(). Can it be replaced with KASSERT?

In D18380#391441, @imp wrote:

I'm pissed this was committed. It wasn't ready and in total breach of protocol.
This matter is *NOT* settled and you're lucky I don't just remove it from the tree.

Closed prematurely.

In D18380#391343, @cem wrote:

In D18380#391248, @eugen_grosbein.net wrote:

First, DIOCGDELETE is not GEOM-specific ioctl()

I don't know why you say that. It was created specifically for GEOM (r169284). That's what the 'G' in the name represents, along with other similar ioctls (r92698).

I strongly disagree against GEOM'ifying or CAM'ifyng such simple code.

I'm fine with rest of changes.

The check for CHR and SBLK is to issue nice and correct error message in case of pilot error instead of generic and obscure error after ioctl() so I'd prefer to keep it.

The operation in question is dangerous and can easily result in loss of data if used by unexperienced root. This is why I want it to NOT defaults to -f but instead defaults to verbose dry-run mode. Hence the need for -f and -q.

Use .Cm instead of .Nm for keyword. Begin sentences with new line.