In D31844#718844, @eric_metricspace.net wrote:

Abandoning, due to the existence of the upstream module in cddl

In D45647#1041512, @zlei wrote:

I guess the change for sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vfsops.c go to upstream first. Will it ?

I guess the change for sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vfsops.c go to upstream first. Will it ?

I would subtly change the title :-)
Otherwise, looks good to me.

Why not backport 506fe78c48 instead?

reviewed-by: allanjude

I don't have any objection to the change, then.

I have now successfully run a FreeBSD regression test suite on a ZFS root with block cloning enabled

In D41991#958356, @mm wrote:

I have run the ZFS test suite with block cloning enabled. There are no tests with different results as when it is not enabled.

I have run the ZFS test suite with block cloning enabled. There are no tests with different results as when it is not enabled.

I have run the ZFS test suite

In D41991#957622, @mjg wrote:

for example there was panic under load when running poudriere, i don't know if that is fixed

for example there was panic under load when running poudriere, i don't know if that is fixed

@mjg Does any specific bug come to your mind?

For this to be an option this review has to provide a list of bugs reported against block cloning and commits which fix them, as is I'm not even sure it was all sorted out.

Have you tried running the test suite on a ZFS system with this change applied?

I'll added some inline comments, but they are only suggestions.
I think this minor semantics change for VOP_COPY_FILE_RANGE()
is ok, but we'll see if others disagree.

I have changed the code to the way rmacklem@ recommended. In fusefs we don't have to change anything as there is already a mountpoint check at the very beginning. In nfsclient I have added the mountpoint check. ZFS can handle this change without modifications.

Maybe I didn't make the NFS case clear.
For two different NFS mounts it can do the Copy
without reads/writes for some situations.
(Both mounts NFSv4.2 and either same server
or maybe different servers.)

As noted, I think a check for "same file system type" is needed
before a call to VOP_COPY_FILE_RANGE().

I have updated the diff to use a kernel mount flag.

Committed in https://cgit.freebsd.org/src/commit/?id=c98ecfceb36159e52c8d7e5cfc0d376393a6a60b

Also add to Makefile.inc1 and SUBDIR_DEPEND_*. Should always work now.

Alright, I incorporated these last feedback items. I'm going to commit this shortly.
Thanks to all reviewers, your feedback and suggestions helped a lot in improving the text!

As far as I'm concerned, I think you can go ahead and commit at your discretion after this round of (very minor) suggestions. Getting to diminishing returns here.

A buildworld fails after applying this patch to main, seems to be while linking libavl:

Change an instance of "alone" to "only" in the L2ARC description to not confuse people about adding multiple devices there.

Reply with confirmation about the L2ARC vdevs question.

I'll have another look at the whole thing, but it will likely be a few days before I can.

Update with some of my own replies.

Yet another update addressing some of @pauamma_gundo.com's comments.

Looks right to me.

In D32521#733703, @asomers wrote:

This looks good, but I'm surprised that there aren't more transitive dependencies that can be removed. For example, zfsd doesn't use libzfs_core directly, only through libzfs. Can libzfs_core be removed from its LIBADD?

Remove libzfs_core dep from zfsd

This looks good, but I'm surprised that there aren't more transitive dependencies that can be removed. For example, zfsd doesn't use libzfs_core directly, only through libzfs. Can libzfs_core be removed from its LIBADD?

Aaaaaaaand done! (Split glossary to 1 sentence per line, please.)

Mentioning it here to avoid repeating it a gazillion times: 1 sentence per line.

In D31725#720207, @rew wrote:

Sounds like a bug in the upstream module. I’d prefer to see that fixed rather than using autofs to work around the zfs_key pam module bug. Or see if it’s feasible to implement a pure autofs solution (i.e., autofs loads and unloads the key for a given dataset). If neither of the above options are possible, I’m more inclined to oppose this change until a more streamlined solution can be found.

Sounds like a bug in the upstream module. I’d prefer to see that fixed rather than using autofs to work around the zfs_key pam module bug. Or see if it’s feasible to implement a pure autofs solution (i.e., autofs loads and unloads the key for a given dataset). If neither of the above options are possible, I’m more inclined to oppose this change until a more streamlined solution can be found.

In D31725#719491, @rew wrote:

In D31725#719489, @eric_metricspace.net wrote:

As I explained in the description, this is intended to be used in conjunction with PAM to load a user's auth token as a key, allowing their encrypted home directory to be loaded when they log in, and unloaded once they've fully logged out. This in turn is a means to implement a common requirement on high-security systems.

According to the OpenZFS commit, the zfs_key pam module already unmounts the dataset and unloads the key when a session is closed.

For reference: https://github.com/openzfs/zfs/commit/221e67040fc47c15b3da2afb09bb48f1e9700fb9

In D31725#719489, @eric_metricspace.net wrote:

As I explained in the description, this is intended to be used in conjunction with PAM to load a user's auth token as a key, allowing their encrypted home directory to be loaded when they log in, and unloaded once they've fully logged out. This in turn is a means to implement a common requirement on high-security systems.

According to the OpenZFS commit, the zfs_key pam module already unmounts the dataset and unloads the key when a session is closed.

In D31725#719350, @rew wrote:

Does it make sense to have autofs unload zfs keys if it doesn't even know how to load them?

Does it make sense to have autofs unload zfs keys if it doesn't even know how to load them?

In D31725#718850, @delphij wrote:

In D31725#718616, @eric_metricspace.net wrote:

In D31725#718415, @delphij wrote:

In general I don't think it's a good idea to add an option to opt-in unload of crypto keys: it should always be done unless explicitly opted out (or can't be opted out), so I'd recommend either removing the option and default to be safe, or make it an opt-out option (when specified, do not unload crypto key).

I'm not sure it's a good idea to abruptly change behavior in this way. I think it would be a better idea to introduce the ability as opt-in, and then change it to opt-out after announcing in advance.

I'm not very convinced that this is a behavior change (please do correct me if I was wrong). Here is my thought: the change seems to affect ZFS case only, and for ZFS the regular usage is that they are mounted by zfs mount -a, and my understanding is that this change is intended to be used with the PAM module to provide more automation to the load/unload key process.

If the assumption above was right, since there isn't current working way of loading ZFS encrypted dataset with this new workflow, we are not changing any existing behavior.

In D31725#718616, @eric_metricspace.net wrote:

In D31725#718415, @delphij wrote:

In general I don't think it's a good idea to add an option to opt-in unload of crypto keys: it should always be done unless explicitly opted out (or can't be opted out), so I'd recommend either removing the option and default to be safe, or make it an opt-out option (when specified, do not unload crypto key).

I'm not sure it's a good idea to abruptly change behavior in this way. I think it would be a better idea to introduce the ability as opt-in, and then change it to opt-out after announcing in advance.

Abandoning, due to the existence of the upstream module in cddl

In D31725#718415, @delphij wrote:

In general I don't think it's a good idea to add an option to opt-in unload of crypto keys: it should always be done unless explicitly opted out (or can't be opted out), so I'd recommend either removing the option and default to be safe, or make it an opt-out option (when specified, do not unload crypto key).

In general I don't think it's a good idea to add an option to opt-in unload of crypto keys: it should always be done unless explicitly opted out (or can't be opted out), so I'd recommend either removing the option and default to be safe, or make it an opt-out option (when specified, do not unload crypto key).

This proposed module have a lot of behavior that is very different from the Oracle and the OpenZFS implementation and since we are naming it the same way I don't think it's a good idea to diverge from the other implementations, especially the most commonly used parameters like homes= and the lack of mounting of the datasets.

Update including the lastest suggestions to the lower last part of the chapter.

Another big update, including both @ygy's and @pauamma_gundo.com's comments.
I should have kept the diff much smaller in retrospect, sorry about that. We're getting close to the final piece now, thanks to your efforts!
The line breaks were caused by my editor trying to be smarter than it should be.

Taking a break before tackling the wall of text, but splitting lines into sentences first would make needed edits easier to spot. Pretty please?

Hoping to finish reviewing it tonight. 4 installments is pushing it.

I've given it another once-over, and am pretty happy with it, so unless anyone else has any interjections, I say it's good to go.

New version that includes suggestions by @pauamma_gundo.com

Phew, a lot of changes, most of which I concur with and changed. Thanks!
Updated patch follows.

Taking me longer than i thought (when doesn't it for anyone?) but getting there.

In D31707#715576, @bcr wrote:

That's right. My understanding is that everything is a dataset unless it was created by "zfs create -V ...", which for me is a volume (and used in this way). As you said, it may change or even blur some more with future changes. My idea is to use them consistently, for example mention datasets everywhere when it comes to ZFS features. When involving mountpoints, I'd refer to file systems, like this: "Mount the dataset as a file system into the directory tree." That way, we use it only when necessary, but keep the dataset syntax. Same when it involves volumes.