Page MenuHomeFreeBSD

add soshutdown() calls to the server side krpc whenever a socket is no longer usable
ClosedPublic

Authored by rmacklem on Apr 1 2021, 2:41 AM.

Details

Summary

It has been reported that a Linux NFSv4.1 client mount to a
FreeBSD server can result in the FreeBSD end of the TCP
socket being stuck in CLOSE_WAIT.
I think this can happen because the soclose() for the socket
cannot happen until the back channel is re-assigned to another
TCP connection.

This patch adds soshutdown(..SHUT_WR) calls where the socket
is no longer usable.

The patch is simple, but the question is "does soshutdown(..SHUT_WR)
sound like the correct thing to do to get the TCP connection past
CLOSE_WAIT?

Test Plan

Tested by network partitioning a client from a server
for a few minutes to break the TCP connection.

For some tests, the soclose() calls were commented
out, so that soshutdown(..SHUT_WR) was all that
was being done and the TCP connections did go
away.

Diff Detail

Repository
R10 FreeBSD src repository
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

rmacklem created this revision.

The idea seems generally sane to me, but I would definitely wait for tuexen to pitch in. My understanding is that shutdown() will either progress us somewhat quickly to FIN_WAIT_1 or LAST_ACK depending on our prior state, and the tcp state machine will thereafter deliver the final blow modulo freeing local resources.

It is fine to call soshutdown(..., SHUT_WR), when you know that you don't want to write anymore to the socket, but you want to read from it. This will trigger a state transition to FINWAIT-1 and then FINWAIT-2, if you are in ESTABLISHED and to LAST-ACK and then to CLOSED when you are in CLOSE-WAIT.
Please note that I do not know anything about NFS. But you say you call soshutdown(..., SHUT_WR), when the socket is no longer usable. Based on that description, I'm wondering why you need to keep the socket around locally and can't close it.

In the server side kernel RPC, the socket (struct socket *) is in a
structure called SVCXPRT (normally pointed to by "xprt").
These structures a ref counted and the soclose() is done
when the ref. cnt goes to zero. My understanding is that
"struct socket *" is free'd by soclose() so this cannot be done
before the xprt ref. cnt goes to zero.

For NFSv4.1/4.2 there is something called a back channel
which means that a "xprt" is used for server->client RPCs,
although the TCP connection is established by the client
to the server.
--> This back channel holds a ref cnt on "xprt" until the

 client re-assigns it to a different TCP connection
 via an operation called BindConnectionToSession
 and the Linux client is not doing this soon enough,
it appears.

So, the soclose() is delayed, which is why I think the
TCP connection gets stuck in CLOSE_WAIT and that is
why I've added the soshutdown(..SHUT_WR) calls,
which can happen before the client gets around to
re-assigning the back channel.

This revision is now accepted and ready to land.Apr 27 2021, 8:01 PM