Created attachment 272066 [details] crashinfo output Tried to do a "make -j12 buildworld" over NFS, system panicked after a few minutes while still in bootstrap-tools: Memory modified after free 0xfffff80e29883c00 (384, malloc-384, 0xffffffff81aae4b0) + 88 = deadc0dedeadc0dd panic: Memory modified after free 0xfffff80e29883c00 (384, malloc-384, NFSCL deleg) + 88 = deadc0dedeadc0dd cpuid = 0 time = 1782200437 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x36/frame 0xfffffe01f7803280 vpanic() at vpanic+0x149/frame 0xfffffe01f78033b0 panic() at panic+0x43/frame 0xfffffe01f7803410 mtrash_ctor() at mtrash_ctor+0x147/frame 0xfffffe01f7803450 item_ctor() at item_ctor+0x117/frame 0xfffffe01f78034a0 malloc() at malloc+0x7d/frame 0xfffffe01f78034e0 nfsrpc_create() at nfsrpc_create+0x10ce/frame 0xfffffe01f7803740 nfs_create() at nfs_create+0x2c1/frame 0xfffffe01f7803a70 vop_sigdefer() at vop_sigdefer+0x30/frame 0xfffffe01f7803aa0 VOP_CREATE_APV() at VOP_CREATE_APV+0x57/frame 0xfffffe01f7803ac0 vn_open_cred() at vn_open_cred+0x46a/frame 0xfffffe01f7803c50 openatfp() at openatfp+0x2ed/frame 0xfffffe01f7803dc0 sys_openat() at sys_openat+0x3d/frame 0xfffffe01f7803df0 amd64_syscall() at amd64_syscall+0x179/frame 0xfffffe01f7803f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01f7803f30 --- syscall (499, FreeBSD ELF64, openat), rip = 0xe9dfaca528a, rsp = 0xe9de648fd18, rbp = 0xe9de648fd50 --- KDB: enter: panic Uptime: 18m57s
(In reply to Hans Rosenfeld from comment #0) Could you please give us a little information related to your NFS mount? Run # nfsstat -m and # ps ax | fgrep nfscbd on the client and briefly explain what your NFS server is. If it is a FreeBSD NFS server, please # sysctl vfs.nfsd.issue_delegations and post the results of the above commands here. Thanks, rick
Created attachment 272128 [details] nfscl_state.c: Add a check for delegation busy This patch might fix the problem. Please test this patch and let us know of any "atomic upgrade.." entries in the console log (/var/log/messages). In particular, if you get any of the above log messages where any of the values are non-zero.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=fe6677e7f440d1aa52de036639efc55047ab9a2b commit fe6677e7f440d1aa52de036639efc55047ab9a2b Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-07-04 22:00:02 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-07-04 22:00:02 +0000 nfs_clstate.c: Fix handling of delegation upgrades Commit 016570c4463d modified the client to handle the upgrade of a read delegation to a write delegation, where the server provides the same delegation stateid to the client. However, it failed to check if the delegation structure was currently in use. Without this patch, if the structure was in use, a use after free could occur. This patch handles the "in use" case by copying the necessary fields into the current/old structure and free's the new one instead of the old one that is "in use". PR: 296224 MFC after: 2 weeks sys/fs/nfs/nfsclstate.h | 6 ++++-- sys/fs/nfsclient/nfs_clstate.c | 45 +++++++++++++++++++++++++++++++++--------- 2 files changed, 40 insertions(+), 11 deletions(-)
Although the reporter never responded to indicate if the patch fixed their problem, I have now committed the patch to main and will MFC it.
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=299bbb762db7c6971ad68de412830dee8c0bae0a commit 299bbb762db7c6971ad68de412830dee8c0bae0a Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-07-04 22:00:02 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-07-18 01:15:17 +0000 nfs_clstate.c: Fix handling of delegation upgrades Commit 016570c4463d modified the client to handle the upgrade of a read delegation to a write delegation, where the server provides the same delegation stateid to the client. However, it failed to check if the delegation structure was currently in use. Without this patch, if the structure was in use, a use after free could occur. This patch handles the "in use" case by copying the necessary fields into the current/old structure and free's the new one instead of the old one that is "in use". PR: 296224 (cherry picked from commit fe6677e7f440d1aa52de036639efc55047ab9a2b) sys/fs/nfs/nfsclstate.h | 6 ++++-- sys/fs/nfsclient/nfs_clstate.c | 45 +++++++++++++++++++++++++++++++++--------- 2 files changed, 40 insertions(+), 11 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=a51fdce1b28aba5e857f0d4a27d2e2e59dd0ef7e commit a51fdce1b28aba5e857f0d4a27d2e2e59dd0ef7e Author: Rick Macklem <rmacklem@FreeBSD.org> AuthorDate: 2026-07-04 22:00:02 +0000 Commit: Rick Macklem <rmacklem@FreeBSD.org> CommitDate: 2026-07-18 01:36:06 +0000 nfs_clstate.c: Fix handling of delegation upgrades Commit 016570c4463d modified the client to handle the upgrade of a read delegation to a write delegation, where the server provides the same delegation stateid to the client. However, it failed to check if the delegation structure was currently in use. Without this patch, if the structure was in use, a use after free could occur. This patch handles the "in use" case by copying the necessary fields into the current/old structure and free's the new one instead of the old one that is "in use". PR: 296224 (cherry picked from commit fe6677e7f440d1aa52de036639efc55047ab9a2b) sys/fs/nfs/nfsclstate.h | 6 ++++-- sys/fs/nfsclient/nfs_clstate.c | 45 +++++++++++++++++++++++++++++++++--------- 2 files changed, 40 insertions(+), 11 deletions(-)
The patch has been MFC'd. If this crash re-occurs with this patch, it can be re-opened.