Created attachment 271811 [details] Patch to fix the two issues I've got a macOS 26.5 client mounting a NFSv4 share from a FreeBSD 15.1-RELEASE server. The exported filesystem is on a ZFS pool. I noticed that on the Mac, if I try to create a new file with vi, I get the following error: >>> E325: ATTENTION Found a swap file by the name "/Volumes/zraid/.foobar6.swp" dated: Fri Jun 12 14:50:49 2026 [cannot be opened] While opening file "/Volumes/zraid/foobar6" CANNOT BE FOUND (1) Another program may be editing the same file. If this is the case, be careful not to end up with two different instances of the same file when making changes. Quit, or continue with caution. (2) An edit session for this file crashed. If this is the case, use ":recover" or "vim -r /Volumes/zraid/foobar6" to recover the changes (see ":help recovery"). If you did this already, delete the swap file "/Volumes/zraid/.foobar6.swp" to avoid this message. Swap file "/Volumes/zraid/.foobar6.swp" already exists! [O]pen Read-Only, (E)dit anyway, (R)ecover, (D)elete it, (Q)uit, (A)bort: <<< If I inspect the swap file, I see it has mode 0 permissions: $ ls -l /Volumes/zraid/.foobar6.swp ---------- 1 1001 1001 0 Jun 12 14:50 /Volumes/zraid/.foobar6.swp $ I am not versed in the NFS protocol, nor in the FreeBSD kernel or C, so I used AI to help me debug some packet captures and trace the issue in the kernel. It ultimately identified two separate bugs I was running into, and gave me a patch to fix both of them, which did indeed resolve the issues I was having. I generally take AI's conclusions with a grain of salt, but given that it was correct with a prior FreeBSD bug, and that its patch did fix this issue, I figure I'd bring it to you all the experts and see if there's anything there. I've attached the patch for your reference rather than as a proposed fix, since I imagine you all can do a better one. I'll also include its summary here in the hope that it is more illuminating: >>> Title: nfsrvd_setattr: SETATTR of time_create and BSD file flags fails non-fatally, leaving swap files with mode 0 on NFSv4 exports Component: sys/fs/nfsserver/nfs_nfsdserv.c Affected versions: FreeBSD 15.1-RELEASE (and likely earlier) # Summary: When a macOS NFSv4.0 client creates a file using OPEN4_CREATE with EXCLUSIVE4 createmode and then issues a follow-up SETATTR containing mode, time_access_set, time_modify_set, time_create, archive, and hidden in a single compound, the FreeBSD NFSv4 server fails the entire SETATTR and reports a non-OK status to the client. This causes vim (and other applications using the same open-create pattern) to leave swap files with mode 0 (----------), triggering spurious E325 ATTENTION warnings on every subsequent open of the file. Root cause — two bugs in nfsrvd_setattr(): ## Bug 1: NFSATTRBIT_TIMECREATE in SETATTR nfsrvd_setattr() attempts to set birthtime (va_btime) via nfsvno_setattr() → VOP_SETATTR() when the client includes time_create in a SETATTR request. On FreeBSD, birthtime is not settable via VOP_SETATTR — ZFS returns EINVAL (because AT_XVATTR is set but z_use_fuids is false, or simply because XAT_CREATETIME is a read-only attribute). This sets nd->nd_repstat before the mode attribute group is processed, so the mode is never applied and the file remains at mode 0. The attrsset bitmap in the reply shows only time_access_set and time_modify_set (the groups that ran before time_create), and the overall status is NFS4ERR_INVAL. RFC 7530 does not require servers to support setting time_create via SETATTR. The correct behavior is to silently treat it as a no-op and report it as successfully set, consistent with the behavior of Linux nfsd and other NFSv4 server implementations. ## Bug 2: NFSATTRBIT_ARCHIVE/NFSATTRBIT_HIDDEN/NFSATTRBIT_SYSTEM in SETATTR After Bug 1 is fixed, a second failure surfaces: nfsrvd_setattr() attempts to set BSD va_flags (via nfsvno_setattr()) when the client includes archive or hidden in the SETATTR request. On ZFS filesystems that do not support BSD file flags (including any pool where z_use_fuids is false, such as ZFS pools at version 1), this call fails and the error is returned as the overall SETATTR status — even though mode, time_access_set, and time_modify_set have all been successfully applied. The result is NFS4ERR_IO on the wire, which causes vim to treat the swap file as unreadable even though its mode is now correct. These attributes are Windows/HFS+ compatibility metadata with no semantic equivalent on such filesystems. Failure to set them should be treated as non-fatal. # Fix: Two changes to nfsrvd_setattr() in sys/fs/nfsserver/nfs_nfsdserv.c: Replace the NFSATTRBIT_TIMECREATE block's nfsvno_setattr() call with an unconditional success — skip the VOP call entirely and just set the bit in retbits. After the NFSATTRBIT_ARCHIVE/HIDDEN/SYSTEM block's nfsvno_setattr() call, clear nd->nd_repstat if it is non-zero, treating the failure as non-fatal. # Reproduction: FreeBSD 15.1-RELEASE server exporting a ZFS filesystem via NFSv4. macOS client mounting the export via NFSv4.0. Run vim <file> on the mount. Observe E325 ATTENTION and mode 0 on the .swp file. Packet capture confirms the follow-up SETATTR after EXCLUSIVE4 open-create includes time_create, archive, and hidden, and the server returns a non-OK status. <<<
Could you please capture packets when this failure occurs and attach them here (like you did for the other bugzilla PR). The ai isn't quite correct, in that a FreeBSD VOP_SETATTR() can set birthtime. (I suspect the problem is that the birthtime is specified for the Open/Create and needs to be treated like hidden/system/archive, in that it cannot be set for VOP_CREATE(), but can be set for VOP_SETATTR() after the VOP_CREATE().) Thanks, rick
Created attachment 271842 [details] tshark packet capture Sure, here's the output of a tshark capture while I try to vi a new file and get the error I described. Let me know if there's anything else I can provide!
I should clarify that packet capture only depicts the first bug (NFSATTRBIT_TIMECREATE in SETATTR). Here's the AI's summary of how the frames correlate with that bug, in case it's helpful: >>> Frame 10/13 (XID 0x0d37c2cb) — OPEN with EXCLUSIVE4 create. The client sends PUTFH, SAVEFH, OPEN, GETATTR, RESTOREFH, GETATTR, with OPEN4_CREATE / Create Mode: EXCLUSIVE4 and a verifier. The reply is NFS4_OK; the embedded post-create GETATTR already shows mode: 0 for the new file — this is expected at this stage, since exclusive-create semantics leave attributes to be set by a follow-up SETATTR (the client deliberately didn't pass fattr4 on OPEN). Frame 16 (XID 0x0d37c2cd) — the follow-up SETATTR. This is the operation that triggers bug #1. The compound is PUTFH, SETATTR, GETATTR against the new file's filehandle (0x480c818d). The SETATTR attribute mask spans two words: Archive, Hidden (mask[0]) and Mode, Time_Access_Set, Time_Create, Time_Modify_Set (mask[1]) — confirming the macOS client bundles mode: 0600, time_access_set, time_create, time_modify_set, plus archive/hidden, all in one SETATTR, exactly as the bug summary describes. Frame 17 — the failing reply. Status: NFS4ERR_INVAL, and critically the attrsset bitmap returned is only Time_Access_Set, Time_Modify_Set — meaning the server processed those two groups successfully, then hit Time_Create next and aborted with EINVAL before ever reaching Mode or Archive/Hidden. This is the direct on-wire evidence for Bug 1: time_create failing non-fatally before the mode group runs, so the file is left at mode 0. Note the per-spec NFSv4 attribute processing order (numerically by bit position) puts Time_Create (bit 50) ahead of Mode (bit 33) in mask word 1 only because Wireshark lists them by attribute number, not transmission order — but the attrsset result confirms Mode simply never got applied. <<< If I apply the first part of the patch, I am able to encounter bug #2, which I'm attaching a second packet capture for. The AI's summary of that is as follows: >>> Frame 7/10 (XID 0x0d37fb5c) — OPEN with EXCLUSIVE4 create. Same pattern as the prior capture: OPEN4_CREATE/EXCLUSIVE4 against a new file, with the embedded post-create GETATTR showing mode: 0 as expected (exclusive-create leaves attributes to the follow-up SETATTR). Frame 14 (XID 0x0d37fb5e) — the follow-up SETATTR. Same client-side attribute mask as before: Archive, Hidden plus Mode, Time_Access_Set, Time_Create, Time_Modify_Set in one compound. This confirms the test scenario is identical to the bug #1 capture — same client behavior, same attribute set. Frame 15 — the reply, and the key diagnostic. Status: NFS4ERR_IO, but the returned attrsset bitmap is now Mode, Time_Access_Set, Time_Create, Time_Modify_Set — all four attributes from mask word 1, including Mode. This is the direct evidence that the bug #1 patch worked: time_create no longer aborts the compound, and the server proceeds far enough to actually apply mode: 0600 along with the timestamps. The failure has moved downstream to the Archive/Hidden group (mask word 0), which is conspicuously absent from attrsset and is the only remaining unset group — consistent with the ZFS pool-version-1 va_flags/EOPNOTSUPP failure described in Bug 2, with that error code surfacing on the wire as NFS4ERR_IO. <<<
Created attachment 271844 [details] tshark packet capture of bug #2
Created attachment 271872 [details] Patch to fix the two issues (in git diff format) ^Triage: convert to git diff format.
Hmm. I cannot reproduce this. For me, setting of birthtime works fine for ZFS. The only thing I can see in the sources is that ZFS must be at least version 3 (I've got version 5) or something called spa_version must be at least 9. # zfs get version <file-system> shows you the ZFS version. I don't know how to get the spa_version. I did find this in the ZFS sources.. /* * Versioning wasn't explicitly added to the label until later, so if * it's not present treat it as the initial version. */ if (nvlist_lookup_uint64(config, ZPOOL_CONFIG_VERSION, &spa->spa_ubsync.ub_version) != 0) spa->spa_ubsync.ub_version = SPA_VERSION_INITIAL; So, all I can suggest is that, maybe, the version or spa_version of your ZFS is too old to support birthtime. The patch the ai created will fix it for you, but I don't think it is appropriate to disable it for everyone when it works for me?
Created attachment 271885 [details] Allow setting of brith_time to fail if atime or mtime was also set Please try this little patch to see if it fixes the setattr case. If it does, I'll make a similar patch for Open. (As I noted, I cannot reproduce the problem. For me, ZFS allows the Time_create to be set.)
Yes, that appears to fix bug #1. When I try to open a new file with vi, I get a slightly different error ("cannot be read" instead of "cannot be opened"), the one that indicates bug #2: >>> E325: ATTENTION Found a swap file by the name "/Volumes/storage/.foobar938hewag.swp" owned by: will dated: Wed Jun 17 19:35:54 2026 [cannot be read] While opening file "/Volumes/storage/foobar938hewag" CANNOT BE FOUND (1) Another program may be editing the same file. If this is the case, be careful not to end up with two different instances of the same file when making changes. Quit, or continue with caution. (2) An edit session for this file crashed. If this is the case, use ":recover" or "vim -r /Volumes/storage/foobar938hewag" to recover the changes (see ":help recovery"). If you did this already, delete the swap file "/Volumes/storage/.foobar938hewag.swp" to avoid this message. Swap file "/Volumes/storage/.foobar938hewag.swp" already exists! [O]pen Read-Only, (E)dit anyway, (R)ecover, (D)elete it, (Q)uit, (A)bort: <<< And indeed the file is no longer created with mode 0: $ ls -l "/Volumes/storage/.foobar938hewag.swp" -rw------- 1 will wheel 0 Jun 17 19:35 /Volumes/storage/.foobar938hewag.swp It does appear my ZFS filesystem is version 1: $ zfs get version zraid/storage NAME PROPERTY VALUE SOURCE zraid/storage version 1 -
(In reply to Will from comment #8) Can you capture packets for the new error and put them up here? Thanks, rick
Created attachment 271892 [details] 2nd capture of bug #2 Sure, here's another capture of me trying again after applying your last patch. I think this will show the same thing as the previously-attached capture of bug #2.
Created attachment 271893 [details] Fix the ZFS detection of support for archive/hidden/system I think this patch might fix the 2nd bug. It fixes ZFS so that it doesn't report that old ZFS versions support archive/hidden/system. Please test this patch and let us know if it fixes the problem for you?
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=b1af05406b5117d76f567056fba0a023a6374465 commit b1af05406b5117d76f567056fba0a023a6374465 Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-06-18 15:45:27 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-06-18 15:45:27 +0000 nfs_nfsdserv.c: Fix setting of birthtime for some ZFS pools Some ZFS pools do not support va_birthtime and will return EINVAL when a VOP_SETATTR() of it is attempted. The MacOS NFSv4 client sets va_birthtime (TimeCreate) in the same Setattr with ctime/mtime and other attributes after a new file is created. The EINVAL failure leaves these new files messed up (mode == 0). This patch pretends the setting of TimeCreate succeeded if ctime/mtime were also set in the same Setattr RPC, which resolves the problem for the MacOS client. If this fix is not sufficient, a new pathconf name to detect if a file system supports birthtime may be needed. PR: 296066 Tested by: Will <freebsd.geography231@slmails.com> MFC after: 2 weeks sys/fs/nfsserver/nfs_nfsdserv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
The patch for the first bug has been committed and will be MFC'd. I'll wait to see if the ZFS patch resolves the 2nd bug.
That fixed it! I'm able to open a new file with vi on the NFS export from the Mac client and there are no longer any errors. Thank you very much for your help! What does the release of these patches look like? 15.2-RELEASE? Regardless, I can manually apply these patches to future releases of the kernel. You've presumably already got it staged, but just to be super specific, the final diff I'm running that fixed it is: diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c index 6e52d90e0940..a31b4c4c3732 100644 --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c @@ -5770,7 +5770,8 @@ zfs_freebsd_pathconf(struct vop_pathconf_args *ap) { ulong_t val; int error; -#if defined(_PC_CLONE_BLKSIZE) || defined(_PC_CASE_INSENSITIVE) +#if defined(_PC_CLONE_BLKSIZE) || defined(_PC_CASE_INSENSITIVE) || \ + defined(_PC_HAS_HIDDENSYSTEM) zfsvfs_t *zfsvfs; #endif @@ -5816,7 +5817,11 @@ zfs_freebsd_pathconf(struct vop_pathconf_args *ap) #endif #ifdef _PC_HAS_HIDDENSYSTEM case _PC_HAS_HIDDENSYSTEM: - *ap->a_retval = 1; + zfsvfs = (zfsvfs_t *)ap->a_vp->v_mount->mnt_data; + if (zfsvfs->z_use_fuids == B_TRUE) + *ap->a_retval = 1; + else + *ap->a_retval = 0; return (0); #endif #ifdef _PC_CLONE_BLKSIZE diff --git a/sys/fs/nfsserver/nfs_nfsdserv.c b/sys/fs/nfsserver/nfs_nfsdserv.c index 55bf56c4daf1..bbbe35c3c14e 100644 --- a/sys/fs/nfsserver/nfs_nfsdserv.c +++ b/sys/fs/nfsserver/nfs_nfsdserv.c @@ -579,6 +579,10 @@ nfsrvd_setattr(struct nfsrv_descript *nd, __unused int isdgram, NFSVNO_SETATTRVAL(&nva2, btime, nva.na_btime); nd->nd_repstat = nfsvno_setattr(vp, &nva2, nd->nd_cred, p, exp); + if (nd->nd_repstat == EINVAL && + (NFSISSET_ATTRBIT(&retbits, NFSATTRBIT_TIMEACCESSSET) || + NFSISSET_ATTRBIT(&retbits, NFSATTRBIT_TIMEMODIFYSET))) + nd->nd_repstat = 0; if (!nd->nd_repstat) NFSSETBIT_ATTRBIT(&retbits, NFSATTRBIT_TIMECREATE); }
(In reply to Will from comment #14) Yes, FreeBSD-15.2 should have these. The first is already in main and should be in stable/15 in 2 weeks. The second needs to be set up as a Pull Request for OpenZFS and then, once pulled into OpenZFS, it needs to downstream into FreeBSD. But that should happen in time for 15.2.
That great. Should I open the ticket in the OpenZFS GitHub, relaying issue #2 here?
(In reply to Will from comment #16) I don't think that is necessary. I have already done a Pull Request. If you do so, reference Pull Request #18688.
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=76badd25811e9cbea84818c23004ff0d13157a4f commit 76badd25811e9cbea84818c23004ff0d13157a4f Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-06-18 15:45:27 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-07-02 00:13:49 +0000 nfs_nfsdserv.c: Fix setting of birthtime for some ZFS pools Some ZFS pools do not support va_birthtime and will return EINVAL when a VOP_SETATTR() of it is attempted. The MacOS NFSv4 client sets va_birthtime (TimeCreate) in the same Setattr with ctime/mtime and other attributes after a new file is created. The EINVAL failure leaves these new files messed up (mode == 0). This patch pretends the setting of TimeCreate succeeded if ctime/mtime were also set in the same Setattr RPC, which resolves the problem for the MacOS client. If this fix is not sufficient, a new pathconf name to detect if a file system supports birthtime may be needed. PR: 296066 (cherry picked from commit b1af05406b5117d76f567056fba0a023a6374465) sys/fs/nfsserver/nfs_nfsdserv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=0b9f4c6b0a03df1522bba415b67d875a1e73f811 commit 0b9f4c6b0a03df1522bba415b67d875a1e73f811 Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-06-18 15:45:27 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-07-02 00:19:24 +0000 nfs_nfsdserv.c: Fix setting of birthtime for some ZFS pools Some ZFS pools do not support va_birthtime and will return EINVAL when a VOP_SETATTR() of it is attempted. The MacOS NFSv4 client sets va_birthtime (TimeCreate) in the same Setattr with ctime/mtime and other attributes after a new file is created. The EINVAL failure leaves these new files messed up (mode == 0). This patch pretends the setting of TimeCreate succeeded if ctime/mtime were also set in the same Setattr RPC, which resolves the problem for the MacOS client. If this fix is not sufficient, a new pathconf name to detect if a file system supports birthtime may be needed. PR: 296066 (cherry picked from commit b1af05406b5117d76f567056fba0a023a6374465) sys/fs/nfsserver/nfs_nfsdserv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
The birthtime patch has been MFC'd. I'll leave this Open until the ZFS patch is in the stable branches.