Address feedback.

In D44274#1017742, @sanastasio_raptorengineering.com wrote:

Hi all, sorry for the ping. If anybody had time to review this, it would be greatly appreciated. We've deployed this patch internally on some of our production systems and in addition to the massive performance improvement, it has proven stable under various workloads.

Committed by @stevek in 3bbe8ed1a7 (D44535)

IfAPI looks fine.

In D44498#1014893, @kib wrote:

In D44498#1014892, @jhibbits wrote:

In D44498#1014883, @kib wrote:

In D44498#1014879, @emaste wrote:

Would it be excessive to just zero td_frame in MI code?

I am not sure that this is the correct state of the registers file for all arches.

Does it not go through exec_setregs() at some point through kern_execve()? That should set all the registers.

Please read the summary which specifically mentions %rflags.PSL_T as the cause of this patch.

In D44498#1014883, @kib wrote:

In D44498#1014879, @emaste wrote:

Would it be excessive to just zero td_frame in MI code?

I am not sure that this is the correct state of the registers file for all arches.

I'll MFC it in the next couple weeks, so it'll be in 14.1.

This means the function now uses more than 1kB stack space. We should be careful bumping it again in the future. It's called early enough that stack space shouldn't be an issue, but something to keep in mind, definitely.

Addressed by D43359

Does psim even still work?

Looks fine to me.

In D43432#991924, @imp wrote:

I wonder if the PS3 option might be past its prime and time to retire.
I couldn't figure out how to run it for my kboot work, for example.

I like the approach of the RF_LITTLEENDIAN, but fsl_sata hangs off simple-bus, not the lbc, so it's stuck with manually setting the bus tag. I can test the lbc changes later.

Looks fine. I have no way to test, though.

I don't disagree with the change, but I'm not sure it's needed. We could simply remove cryptodev from the Book-E kernels; I think it was added for the sec device in MPC85XX (MPC85XXSPE and QORIQ64 are copied from that), but I don't even know how complete the sec driver really is, and it's definitely *not* supported on QORIQ64 (DPAA-based SEC driver is missing, and I have no hardware with it to develop).

Looks right to me.

In D42988#982075, @glebius wrote:

In D42988#981874, @kp wrote:

In D42988#981761, @glebius wrote:

In D42988#980277, @ae wrote:

Do you plan to rework access to if_afdata? There are still several panics related to access to already freed if_afdata[AF_INET6].

Can you please assign those PRs to me? Or send links to information if there is no PR.

We see this issue on pfSense as well: https://redmine.pfsense.org/issues/14431 . There are a few different backtraces, but I believe they're all the same root problem.
The short version of the story is that there's outbound traffic, which does a route lookup (often to get an IPv6 hop limit), only to be returned a netxhop with a struct ifnet that has a NULL afdata[AF_INET6].
I believe it to correlated with said struct ifnet going away (in the case of pfSense users often because they're PPPoE users).

In addition to the pfSense bug report there's also a summary e-mail on net@: https://lists.freebsd.org/archives/freebsd-net/2023-October/004104.html

I still have both core dump and debug kernel for at least one occurrence of this issue (on a pfsense kernel, roughly equivalent to main from August).

There is definitely lots of place for improvement & simplification here. For example, dom_ifdetach should be retired in favor of eventhandler(9), just like I did for other original 4.3BSD domain/proto stuff.

Looks like we got a lot of evidence about the problem. Can we have a single place to track it? Be it pfsense redmine, FreeBSD bugzilla, phabricator?

I love the smell of deleted cruft.

In D42972#980139, @glebius wrote:

Got it. Looks like there is also an assumption here that the first address is always the link level address. Maybe we just need to provide KPI if_getlladdr? May I ask to wait for Alexander melifaro@ to reappear before we proceed forward with that?

P.S. I'm also waiting for him on my netlink reviews.

In D42972#980127, @glebius wrote:

I don't understand the change. The epoch protection is already right here. We can safely use CK_STAILQ_FIRST.

Address @kp's feedback.

In D42453#970257, @zlei wrote:

In D42453#968771, @jhibbits wrote:

There's no "netstack" module in the FreeBSD kernel. I think a DECLARE_MODULE needs added, probably to sys/net/if.c, before this works.

The "netstack" is built into kernel. Maybe the following one?

MODULE_DEPEND(ibcore, kernel, __FreeBSD_version, __FreeBSD_version, maxver);

There's no "netstack" module in the FreeBSD kernel. I think a DECLARE_MODULE needs added, probably to sys/net/if.c, before this works.

Committed as c81dd8e5f.

Address further comments from @zlei.

Address @zlei's feedback and make an IfAPI.

Closed by 2a3716432d

In D41540#953513, @sanastasio_raptorengineering.com wrote:

Also, thank you @jhibbits for approving the patch. I'm not super familiar with the FreeBSD contribution work flow yet, so excuse the question, but is there anything that I need to do at this point to have this patch committed? I presume the patch must now be merged by someone other than me with commit privileges.

As long as it doesn't require VSX (able to run on PowerMac G5), I think it's good.

Overall this looks good, just the one comment on SPE. Can you also test with powerpc64 BE to avoid surprises?

I'm not familiar enough to comment at all. My only role was mechanically converting to IfAPI.

Big "Oops", sorry.