Would it be possible to provide details on testing these changes?

Thank you for submitting the patch!

Just for the logging purposes, the timeline looked like this:

In D15488#493981, @rgrimes wrote:

I have come to the state that if your running into this problem you probably should not be using netstat -rn to look at route tables but rather use the proper tool that talks with your routing daemon, adding a "bang on the kernel" repeat inside of netstat is probably a poor solution.

I have a generic question about the snd_tag functionality.

I read the problem description from the PR194485 and looked in the other (D15405, D15406) patches.

Update revision.

Looks like a really cool and logical thing to do! Several comments inline.

LGTM. The last one: are you planning to implement the same functionality in ip6_output()? :-)

Would it be possible to include firewalls in the test plan?
For example, ensure that ipfw still accepts the packet using something like 'allow ip6 from any to any via lo0' rule.

Hi Jason,

I was thinking of having something like the below one as a macro, moving the rest to the actual function.

Would you mind describe the proposed locking model somewhere in the file explicitly?
In particular, 1) what does _XLOCK or _SLOCK locks protect? 2) How does the sc_slowpath work? 3) What is "right" lock order?
It would be beneficial to describe some examples of problematic potential LORs as well.

In D14802#311336, @jason_eggnet.com wrote:

@melifaro I'm thinking about how to make GET_PKTOPT_VAR a function or perhaps... how to make it much smaller and call a function for the bulk of the logic.

That would be great. The current looks a bit hard to grasp.

Thank you!

In D14702#309683, @kib wrote:

In D14702#309660, @melifaro wrote:

In D14702#309602, @kib wrote:

The feature is disabled by default, so I do not see it as critical or even important that some stuff would break when vid 0 encapsulation is enabled. The feature is added for the cases where it works.

That's the topic we see differently. The functionality indeed is not used by default, however the actual code complicates ether_output().

Indeed, it complicates the function because it adds the new feature.

Furthermore, I'm afraid that after someone actually tries to use this, the code will get more complicated. I'd really appreciate if you could change the ether_output() part to call the (inlined) function doing all of the business logic for handling pcp.

Can you explain more explicitly what do you want to change ? The only interpretation for your words which I was able to construct is that you want ether_8021q_frame() to become inlined in ether_output_frame(). Is it correct ?
If yes, I do not see much sense in it, because ether_8021q_frame() is only called for non-default path, and it really makes sense to keep ether_output_frame() short to not pollute icache.

No, not exactly.

In D14702#309602, @kib wrote:

In D14702#309569, @melifaro wrote:

Several questions.

1. What was the driver of implementing interface-level pcp settings?

For example, currently pcp can be set on per-mbuf basis for the "real" vlans, which can give more granular control on the outgoing traffic.

This is supported in the patch as well. If the scheduled mbuf has 8021Q tag attached, the pcp value from the tag is inserted into the VLAN frame tag.
If you look at the code, you will see that I extracted the fragment from vlan_trasnmit() to reuse in both places.

Ah, my bad. I haven't noticed the 8021q tag override part. In that case it's more like setting default pcp.

Personally it is a bit hard for me to see the benefits of having the same pcp applied to all packets from the host.
Typically this kind of configuration can be easily configured on the access switch and host is supposed to be able to do more fine-granular control.

I think that this is complimentary, and one method of manipulating pcp does not exclude the validity, and apparent usefulness, of another.
The patch was written because there are users who need this feature (I cannot say more).

What comes into my mind is having something like dscp-to-pcp map, which allows user to benefit from all existing dscp manipulation framework.
What do you think?

Perhaps yes, but this is out of scope of the change. If pcp is already assigned, it will be honored. The assignment should be managed by the layer above the place where the framing and transmit are performed.

2. Have you performed interop testing for the non-routable protocols like lldp, stp, lacp, etc?

Having seen some issues related to the routers/switches control plane forgetting to remove similar "dummy" headers, I'd recommend to perform validation with multiple vendors before committing this.

No, no interop testing was done. As an anecdote, I can say that my home switch filters the vid 0 packets outright. More, Mellanox driver has the flow table programmed to drop such packets as well, right now (the fix is already worked out).

The feature is disabled by default, so I do not see it as critical or even important that some stuff would break when vid 0 encapsulation is enabled. The feature is added for the cases where it works.

That's the topic we see differently. The functionality indeed is not used by default, however the actual code complicates ether_output(). Furthermore, I'm afraid that after someone actually tries to use this, the code will get more complicated. I'd really appreciate if you could change the ether_output() part to call the (inlined) function doing all of the business logic for handling pcp.

Several questions.

The issue which this change is trying to solve is control plane issue. Dealing with it in the data path code seem to be a wrong approach. Additionally, it imposes significant performance penalties.
The better way of doing this is to have a "solver" function which is able to handle such cases. Calls to this function can be triggered by ifnet/lagg change events subscription.

In D14257#299409, @vangyzen wrote:

I would like to commit this change and leave improvements in rt_updatemtu() for another day.

Sure, it's totally not the scope of this CR.

In D14257#299355, @ae wrote:

Invoking of rt_updatemtu() looks right to me. But I don't like how rt_updatemtu() works.
Recently we found that in6_mtutimo() on systems with many IPv6 routes produces noticeable delay for packets processing.
It holds RIB_WLOCK while all routes are processed, and thus normal packets processing is blocked on RIB_RLOCK for this time.

There are other rt_foreach_fib() users as well. Probably we can change iteration logic the following way:

retry = 1
while (retry && retry_count < N) {
retry = 0
RADIX_RLOCK
generation = rnh_gen
foreach_route()
  if_match()
    if linked_list_add(route)
      refcount(route)
    else
     retry++; // allocation failure, retry
RADIX_RUNLOCK

Could you please explain why this should be done in kernel?

Great news, thanks for implementing that!

Please take a look on -HEAD ipfw version.
Along with some architectural changes, there are several performance oriented ones: rmlock for fast path, per-cpu rule counters, more compact rule structure and faster tables.
It could be merged to 10 (in fact, we even run in on 9/ in several places)

Sorry for taking that long.