You are right, inp_route6 would be better. I'm sure I copied and pasted, then didn't make as many changes as I should have. I will happily approve a review to make that change if you test that it compiles :-).

Looks good to me. I could go either way on the blank line.

Thanks for catching the typo.

I agree with the change as well; the regression seems to have happened when the code was refactored and the "add" and "change" code became joined.

Looks good.

Got it. I didn't realize it, but Phabricator lists the reviews in reverse order, so that's the order I looked at them.

I just realized that this is necessary when the L3 gateway changes. The L3 cache doesn't need to be invalidated, but L2 does. A comment to this effect would be helpful.

Could this use the new invalidate macro?

Was there a place where only L3 was invalidated?

Why is this necessary, or desirable? If a route that is cached is modified, that doesn't invalidate it. It should still be the best route if it was before.

Tweak man page

A list of ioctl names didn't seem very useful to me, so I extracted the information from the header file.

Update man page

IIRC, there are ten of these ioctls, although I didn't add any. I thought I'd correct the obvious out-of-date statement, but wasn't sure I could correctly document all of those added over time. Suggestions?

Rod, perhaps you have never needed this functionality; but have you ever had a system hang? Along the same lines, there are probably about 1000 bits of code that you haven't needed in the kernel; should we add options to remove each of them? This way lies madness. There are two hardclock routines; which are you using? Should we ifdef out the other?

Sorry, my last comment was responding to imp@; it crossed with the comment from rgrimes.

Thanks. About loadable hardware watchdogs: that should work if they attach before watchdogd starts.

Update date.

Fix macro usage, update text

Another attempt to clarify, flesh out example.

Update Dd date

I hadn't noticed this snippet of code before. It was definitely wrong.

Commited in https://reviews.freebsd.org/rS316065

I think the change is a step in the right direction. Certainly, "ifconfig xxN down" followed by an implicit UP should not cause any change to the routing table. Does anyone know why the "down" is removing the route? That seems wrong to me.

Seems fine to me; will let someone else approve.

btw, the second chunk (first after include) is new since original version.

This patch has been tested by Mike Andrews, who will continue monitoring. I've asked Peter if he can test.

I'm looking into the account issue.

I'm not sure my acceptance "counts" using mike-karels.net; if you add karels@, I can re-approve.

Looks good to me. This will only update one connection, but there is nothing wrong with that. We could receive a large number of redirects if there are a lot of connections using the same route. Another option would be to increment the route table revision if a redirect adds a new route, doing nothing if it changes a route. But this is quick and expedient. It should obviously be MFC'ed to 11.0.

Update to current HEAD; fix to work with flowtable

Hello? Is anyone looking at this?

I disagree; congestion is congestion, not "congestion for everyone but me". I'd prefer to leave the cwnd change until it is replaced by something more modern.

btw, I think the line to set the snd_cwnd should remain for now, until something replaces it. ENOBUFS signals local congestion.

Setting a retransmission timer on an ACK makes no sense; I don't think tcp_output will send an ACK on a retransmission timeout.

I believe that the original code is wrong, and the change is not sufficient
to fix it. The retransmit timer should be running if and only if we have
sent data into the receive window and are awaiting an ACK. The persist
timer should be running if and only if the retransmit timer is not running,
and we have data to send that do not reasonably fit in the window. If we
get an ENOBUFS when sending data, we will already be running the retransmit
timer. If we drop an ACK on ENOBUFS, either we will receive more data and
attempt another ACK, or the sender will time out and resend data. Either
will get the connection started again. I believe lines 1552-1554 should
simply be deleted.

This change does not affect forwarding, just TCP and UDP termination. It makes the most difference when TSO is not enabled.

Fix user-level include of in_pcb.h
rt_gen_t was in an ifdef _KERNEL

Argh, I see what I did. I ended up with the wrong version of route.h in /usr/include.

Update to match head, fix UDP/IPv4 locking
Move definitions to match movement in header files.

OK, I missed the malloc/copy in if_ethersubr.c. So there isn't a problem with pointers to freed memory, but the L2 code still has the issue as the routing code with copying the pointers to/from the route structure.

About L3 caching:

This revision does several things:

I haven't reviewed this as closely as I should. However, if I read this correctly, this is a new caching mechanism that:

Randall: see also D4306 for another approach.

Thanks, I think I understand your position better now. But you are objecting mostly to the current API and data structures, not against using them. What I mean is that the "struct route" *is* the existing mechanism to get and hold a reference to a rtentry. The rtentry is the existing entity that contains the exported information, and "private" management information, and includes a reference count and a bit (RTF_UP) to report invalidation of the route to the remaining references. (And the route structure did not change for radix; that the rtentry.) So, while you apparently want to replace the API and implementation, we should take full advantage of what's there now, unless/until something new arrives in head. The same points apply to the llentry.

Thanks for the pointer to your wiki page; I wasn't aware of that before now. It sounds like you have some interesting work in progress. However, I don't agree with your conclusion that caching the route and lle are "wrong" or "hacks". The interface to route and lle validation could be encapsulated, but ip*output need to look up routes and get the information, and it will always be faster to cache the value than to make a "fast" lookup.

In D4306#90594, @kmacy wrote:

I've had a patch to do this floating about for about 5 years (hence the otherwise vestigial rnh_gen). I hope you're successful in pushing in something that provides this functionality. I brought it up with melifaro@ about a year ago and he said that he had something better going in shortly.

Looking forward to continuing improvements.