Hmm. Sounds like it should only be enabled if TLS is
not enabled?
For the TLS case, the receive code in clnt_vc.c does
upcalls to userland for non-application data records
and these are handled by the OpenSSL library using
a SSL_read() call.

One more "bigger picture" comment.
If setting TCP_USE_DDP is something
that the NFS client will always want to
try and do, then I would scrap the client
side changes in this patch and put the
so_setsockopt() right in clnt_vc_create().

One more comment. If this socket option must/should be
set right away (before any data travels on the TCP
connection), it should be left in clnt_reconnect_coonect(),
but should be moved to before the rpctls_connect() call block.
(Right after clnt_vc_create() at line#202.)

Oh, and to make it work on the client side, the
CLSET_TCP_DDP needs to be set in newnfs_connect().
(In the nmp != NULL section.)

As noted in the second inline comment, the socket
opt would normally be set in clnt_vc.c and clnt_rc.c
would just do a CLSET on the client to do that.

Looks ok to me.

I ran a test cycle mounting the client laptop locally
(which is all I can do at this time) and with another
experimental patch removed and...
-> I see no performance degradation and do not

go anywhere need the runningbuf limit (it never
exceeded 1Mbyte, from what I saw).

In D43249#987015, @imp wrote:

In D43249#987014, @rmacklem wrote:

Although I have not found any correctness issue with this patch,
I am seeing a little performance degradation.
For example, a kernel build over NFS takes about 5% longer
on the setup/hardware I have.

I do not know if 5% is worth worrying about?

Are you hitting the runningbufs limit?

vfs.hirunningspace: 16777216
vfs.lorunningspace: 11206656
vfs.runningbufspace: 0

The first two are control, and the last one is the currently used space...

Although I have not found any correctness issue with this patch,
I am seeing a little performance degradation.
For example, a kernel build over NFS takes about 5% longer
on the setup/hardware I have.

Yes, I think a FreeBSD NFS server exporting msdosfs
file systems will change file handles when a rename
occurs.

Looks ok to me, although I have limited knowledge w.r.t.
the buffer cache.

Added kib@'s lines to the calculation as an extra
safety belt (and to avoid any overflow to a negative
value).

This looks ok to me and testing with the patch has not
found any regression.

This looks fine to me. I did a little testing and did not
find any regression.

Changed the description for EINVAL for flags not 0,
as suggested by karels@, except I used "argument"
instead of "parameter", since .FA is defined as function argument.

Oh, now I see you are talking about the Title here.
I don't know how to change that and it won't affect
the commit.
(I only copy the Summary into the commit and then edit
it from there.)

Oh, ok, I now see retruned in the Summary here on
phabricator (not in the diff). That I can fix.

Something weird is going on...
I searched for rumge (both lower and upper case) and there
is no occurrence of it in the copy_file_range.2 I an diff'ng against.
Same goes for retrun.

Ok, I'm not seeing the typo in the man page name?
(Can you please point it out.)

Committed as 922e78f65fd9.

Changed imbedded to embedded as suggested by others.
I also changed a "for" to a "to", which was just a typo.

First, I have no idea when/who did the nfs_directio_enable
code. It was either in the old NFS client or, if not, it was in
the OpenBSD2.6 code I used as a base for doing NFSv4, 20+
years ago.

Unfortunately,, if you
sysctl vfs.nfs.nfs_directio_enable=1
and the run a program that opens a file O_DIRECT | O_RDONLY
and then reads the file, it will use the copyout().

I am not familiar with the stacking of vfs's, so I
can't really help with this.

This version of the patch only slocks allprison_lock,
as suggested by jamie@.

This version of the patch acquires a shared lock on
allprison_lock (which looks sufficient to ensure the
jail does not go from alive to dying.
It also acquires pr_mtx to check for prison_isalive(),
although I am not 100% sure it is needed?

Yuck. It looks like the only way to ensure that a jail
does not transition from alive to dying is to hold
the allprison_lock or pr_mtx.

In D42672#973753, @markj wrote:

Since vfs_exjail_delete() is called when there are no processes running in the jail, a caller of vfs_exjail_clone() cannot be running in that jail.

It's not clear to me that the first part is true. In particular, prison_deref() first calls prison_deref_kill(), which calls prison_cleanup() and thus vfs_exjail_delete(); later it kills processes within the jail.

Hmm, unless I'm missing something, this is at odds with the comment in vfs_exjail_delete(). Perhaps we need to check the prison state when setting mnt_exjail? i.e., refuse to set it if the jail is dying.

I realized that the previous version on the patch
would result in the weird case where:
A process running locally in the NFS server that
is not in the jail that has exported the fs could
automount the snapshot, but it would not be
exported.

In D42672#973723, @markj wrote:

Do we need to interlock somehow with vfs_exjail_delete()? In other words, what happens if ZFS tries to clone the reference to a mountpoint as vfs_exjail_delete() is purging references to the prison, and the target mountpoint has already been visited?

Patch has been updated to check for "same prison
as the thread is in before doing the cloning".
I think this is sufficient to deal with interaction
with vfs_exjail_delete().

This patch is the vfs_exjail_clone() subset of the
previous one, since this is all I can commit to FreeBSD.

And just to confirm (as Mike Karels suggested in an email),
by PR you mean a git pull request and not a Problem Report?

Ok, so am I supposed to get the ZFS part
reviewed over on OpenZFS's git via a pull
request?

This problem is now in FreeBSD's bugzilla as
PR#275200 and OpenZFS's github as #15546.