Background: ZFS filesets carry a hidden .zfs directory at their toplevel. Within that directory are the current snapsots of this fileset mounted as readonly trees. Therfore, after one accidentally deletes a file and a suitable snapshot exists, one might just restore the file from that snapshot: $ cp .../.zfs/snapshot/snapshot_X/someplace/somefile .../someplace/somefile Problem: Doing this on a local ZFS fileset works as expected. But doing it via NFS on a remote ZFS fileset copies 0 byte (the empty file is created) and then reports the error "Read-only file system". Tests by forum members have shown only NFSv4 minor 2 is affected, the other versions, and NFSv3 apparently work correct. And apparently, besides 14.4, 15.0 and 16-current is also affected. On forums popular demand this should become a bug report (sorry, Rick): https://forums.freebsd.org/threads/nfs-read-only-does-not-allow-reading.102080/post-750701
(In reply to Peter Much from comment #0) Yes, using the mount option minorversion=1 will work around this, since only NFSv4.2 has a Copy operation. I'll take a look at zfs_copy_file_range(). I recall that it can work on files on different file systems within a pool, but I suspect that a snapshot is a special case. (Or that the read-only check is applied to the wrong file system, so that it sees read-only for the output file when it is no problem for the input file.) --> For most/all other file systems, copy_file_range() only works within one file system, but ZFS is, as usual, different.
Created attachment 269060 [details] zfs_vnops_os.c: Add copy_file_range fallback for EROFS This patch adds EROFS to the list of error returns for zfs_clone_range() where zfs_copy_file_range() returns ENOSYS so that vn_generic_copy_file_range() will be used. zfs_clone_range() appears to check for a read-only file system for the output fs, but maybe ZFS doesn't recognize a snapshot as a separate file system from the output fs? Maybe the reporter can try this patch and see if it fixes the problem?
(In reply to Rick Macklem from comment #2) On a quick test the patch does not seem to have the desired effect. But I am on travel now and cannot further test...
Created attachment 269090 [details] nfsd: Fix handling of Copy and Clone across file systems This patch fixes a problem on the NFSv4.2 server when doing a Copy or Clone operation across server file systems. ZFS can copy across server file systems within the same pool. This patch allows the Copy/Clone attempt to let the VOP_COPY_FILE_RANGE() code decide if it can be done. With this patch, the cp(1) from a snapshot works if the mount is done with "noatime". The EROFS error occurs when the Copy/Clone attempts to do a Setattr of atime on the input file, which will be attempted unless the NFSv4.2 mount specifies "noatime". (See commits cc760de and 57ce37f.)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=b65e7b4944cc2f594c9d9e6abc9b8618d3d62ff8 commit b65e7b4944cc2f594c9d9e6abc9b8618d3d62ff8 Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-03-28 19:39:10 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-03-28 19:39:10 +0000 nfs_nfsdsocket.c: All Copy and Clone across file systems For some server file system types, such as ZFS, a Copy/Clone operation can be done across file systems of the same file system type. As such, this patch allows the Copy/Clone to be attempted when the file handles are for files on different file systems. This fixes a problem for exported ZFS file systems when a copy_files on file_range(2) between file systems in the same NFSv4 mount is attempted. PR: 294010 MFC after: 2 weeks sys/fs/nfsserver/nfs_nfsdsocket.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Created attachment 269191 [details] nfs_nfsdserv.c: Make NFSv4.2 Setattr of atime pretend to succeed for ro mnt This patch makes the NFSv4.2 server pretend that atime was set for a Setattr of only atime when the server file system is read-only. This makes an NFSv.4 Copy/Clone behave the same way as it does locally, for a ZFS snapshot or a copy from a read-only mounted file system. With this patch and attachment #269090 [details], the copy from a snapshot works for NFSv4.2 mounts.
Attachment #269090 [details] has been committed to main and will be MFC'd.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=b5815ee99a015c6ac118d7e9646d0c95b72e9f2d commit b5815ee99a015c6ac118d7e9646d0c95b72e9f2d Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-03-29 21:41:36 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-03-29 21:41:36 +0000 nfs_nfsdsocket.c: Allow Copy/Clone from a read-only fs For some server file system types, such as ZFS, a Copy/Clone operation can be done across file systems of the same file system type. However, without this patch, the Copy/Clone will fail with EROFS if the input file is on a read-only mounted file system. This happens because Copy/Clone will try to do a VOP_SETATTR() of atime to set the atime. This patch pretends the VOP_SETATTR() of atime worked for read-only file systems. It fixes a problem when copying files from a ZFS snapshot. PR: 294010 MFC after: 2 weeks sys/fs/nfsserver/nfs_nfsdserv.c | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=6dddd2503fc71b6e192b198209cdcd81b1b04fa8 commit 6dddd2503fc71b6e192b198209cdcd81b1b04fa8 Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-03-28 19:39:10 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-04-11 00:57:08 +0000 nfs_nfsdsocket.c: All Copy and Clone across file systems For some server file system types, such as ZFS, a Copy/Clone operation can be done across file systems of the same file system type. As such, this patch allows the Copy/Clone to be attempted when the file handles are for files on different file systems. This fixes a problem for exported ZFS file systems when a copy_files on file_range(2) between file systems in the same NFSv4 mount is attempted. PR: 294010 (cherry picked from commit b65e7b4944cc2f594c9d9e6abc9b8618d3d62ff8) sys/fs/nfsserver/nfs_nfsdsocket.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=1d4e9d2b5ef5c8055740c949ad4446aed9a4d308 commit 1d4e9d2b5ef5c8055740c949ad4446aed9a4d308 Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-03-28 19:39:10 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-04-11 03:33:19 +0000 nfs_nfsdsocket.c: Allow Copy across file systems For some server file system types, such as ZFS, a Copy/Clone operation can be done across file systems of the same file system type. As such, this patch allows the Copy/Clone to be attempted when the file handles are for files on different file systems. This fixes a problem for exported ZFS file systems when a copy_files on file_range(2) between file systems in the same NFSv4 mount is attempted. PR: 294010 (cherry picked from commit b65e7b4944cc2f594c9d9e6abc9b8618d3d62ff8) sys/fs/nfsserver/nfs_nfsdsocket.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=4beef6a4c5676249c6e87de5354938d5b15dbd88 commit 4beef6a4c5676249c6e87de5354938d5b15dbd88 Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-03-29 21:41:36 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-04-12 00:36:51 +0000 nfs_nfsdsocket.c: Allow Copy/Clone from a read-only fs For some server file system types, such as ZFS, a Copy/Clone operation can be done across file systems of the same file system type. However, without this patch, the Copy/Clone will fail with EROFS if the input file is on a read-only mounted file system. This happens because Copy/Clone will try to do a VOP_SETATTR() of atime to set the atime. This patch pretends the VOP_SETATTR() of atime worked for read-only file systems. It fixes a problem when copying files from a ZFS snapshot. PR: 294010 (cherry picked from commit b5815ee99a015c6ac118d7e9646d0c95b72e9f2d) sys/fs/nfsserver/nfs_nfsdserv.c | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=fcec95bca38fb9bd3bd30b94efc4a0fe2bde3034 commit fcec95bca38fb9bd3bd30b94efc4a0fe2bde3034 Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-03-29 21:41:36 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-04-12 00:45:40 +0000 nfs_nfsdsocket.c: Allow Copy/Clone from a read-only fs For some server file system types, such as ZFS, a Copy/Clone operation can be done across file systems of the same file system type. However, without this patch, the Copy/Clone will fail with EROFS if the input file is on a read-only mounted file system. This happens because Copy/Clone will try to do a VOP_SETATTR() of atime to set the atime. This patch pretends the VOP_SETATTR() of atime worked for read-only file systems. It fixes a problem when copying files from a ZFS snapshot. PR: 294010 (cherry picked from commit b5815ee99a015c6ac118d7e9646d0c95b72e9f2d) sys/fs/nfsserver/nfs_nfsdserv.c | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)
The patches have been committed and MFC'd.