Bug 255436 - nfs client panic with non-sleepable locks on 14.0-CURRENT main-n246340-01bad87a76d
Summary: nfs client panic with non-sleepable locks on 14.0-CURRENT main-n246340-01bad...
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Rick Macklem
URL:
Keywords: crash
Depends on:
Blocks:
 
Reported: 2021-04-27 04:27 UTC by Masachika ISHIZUKA
Modified: 2022-10-12 00:49 UTC (History)
1 user (show)

See Also:


Attachments
core.txt (145.76 KB, text/plain)
2021-04-27 04:27 UTC, Masachika ISHIZUKA
no flags Details
add check for NULL and forced dismounts for nfscl_delegreturnvp() (1.37 KB, patch)
2021-04-27 22:02 UTC, Rick Macklem
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Masachika ISHIZUKA 2021-04-27 04:27:46 UTC
Created attachment 224466 [details]
core.txt

On 14.0-CURRENT main-n246340-01bad87a76d
 
Kernel page fault with the following non-sleepable locks held:
exclusive sleep mutex nfs_clstate_mutex (nfs_clstate_mutex) r = 0 (0xffffffff81e7e640) locked @ /usr/src/sys/fs/nfsclient/nfs_clstate.c:3303

# cat info.last
Dump header from device: /dev/gpt/fbswap
  Architecture: amd64
  Architecture Version: 2
  Dump Length: 609054720
  Blocksize: 512
  Compression: none
  Dumptime: 2021-04-27 12:42:40 +0900
  Hostname: carrot.ish.org
  Magic: FreeBSD Kernel Dump
  Version String: FreeBSD 14.0-CURRENT #31 main-n246340-01bad87a76d-dirty: Tue Apr 27 11:51:50 JST 2021
    ishizuka@okra.ish.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
  Panic String: page fault
  Dump Parity: 732936796
  Bounds: 3
  Dump Status: good

vmcore.3 was saved on https://www.ish.org/files/vmcore.3-n246340.xz
Comment 1 Rick Macklem freebsd_committer freebsd_triage 2021-04-27 19:56:06 UTC
A couple of questions?
- Did this happen during normal operation or during
  a umount?
- Are delegations enabled on your server and nfscbd(8)
  running in the client?
Comment 2 Rick Macklem freebsd_committer freebsd_triage 2021-04-27 20:51:31 UTC
Ok. I looked at the attachment and, yes, I
can see it was an unmount.

I'll put a patch here soon that I think
will fix it.
Comment 3 Rick Macklem freebsd_committer freebsd_triage 2021-04-27 22:02:47 UTC
Created attachment 224479 [details]
add check for NULL and forced dismounts for nfscl_delegreturnvp()

A recent commit to main added a function called
nfscl_delegreturnvp() to return delegations during
VOP_RECLAIM().
The function erroneously assumed that nm_clp would
be non-NULL. It will be NULL for NFSV4.0 mounts until
a regular file is opened. It will also be NULL during
vflush() in the NFS VFS_UNMOUNT() for a forced dismount.

This patch adds a check for clp == NULL to fix this.

Also, since it makes no sense to call nfscl_delegreturnvp()
during a forced dismount, the patch adds a check for that
case.
Comment 4 Rick Macklem freebsd_committer freebsd_triage 2021-04-27 22:04:02 UTC
The patch in the attachment will be committed soon.
Hopefully the reporter can confirm it fixes the
problem for them.
Comment 5 Masachika ISHIZUKA 2021-04-28 00:28:52 UTC
Thank you for patch.
This patch seems to solve my problem.
Applying this path to main-n246364-db8c27f4991, autounmountd can successfully unmount after 10 minutes.
Comment 6 commit-hook freebsd_committer freebsd_triage 2021-04-28 00:34:14 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=f6fec55fe30088bbefd3efe70b62565399a7b9b8

commit f6fec55fe30088bbefd3efe70b62565399a7b9b8
Author:     Rick Macklem <rmacklem@FreeBSD.org>
AuthorDate: 2021-04-28 00:30:16 +0000
Commit:     Rick Macklem <rmacklem@FreeBSD.org>
CommitDate: 2021-04-28 00:30:16 +0000

    nfscl: add check for NULL clp and forced dismounts to nfscl_delegreturnvp()

    Commit aad780464fad added a function called nfscl_delegreturnvp()
    to return delegations during the NFS VOP_RECLAIM().
    The function erroneously assumed that nm_clp would
    be non-NULL. It will be NULL for NFSV4.0 mounts until
    a regular file is opened. It will also be NULL during
    vflush() in nfs_unmount() for a forced dismount.

    This patch adds a check for clp == NULL to fix this.

    Also, since it makes no sense to call nfscl_delegreturnvp()
    during a forced dismount, the patch adds a check for that
    case and does not do the call during forced dismounts.

    PR:     255436
    Reported by:    ish@amail.plala.or.jp
    MFC after:      2 weeks

 sys/fs/nfsclient/nfs_clnode.c  | 9 ++++++++-
 sys/fs/nfsclient/nfs_clstate.c | 6 ++++--
 2 files changed, 12 insertions(+), 3 deletions(-)
Comment 7 Rick Macklem freebsd_committer freebsd_triage 2021-05-01 00:00:06 UTC
Since I believe the patch committed to main
should have fixed the problem, I will close this PR.
Comment 8 commit-hook freebsd_committer freebsd_triage 2021-05-12 02:22:47 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=f6bc043b7a16529fa475b1759011a1bdf83f9652

commit f6bc043b7a16529fa475b1759011a1bdf83f9652
Author:     Rick Macklem <rmacklem@FreeBSD.org>
AuthorDate: 2021-04-28 00:30:16 +0000
Commit:     Rick Macklem <rmacklem@FreeBSD.org>
CommitDate: 2021-05-12 02:18:17 +0000

    nfscl: add check for NULL clp and forced dismounts to nfscl_delegreturnvp()

    Commit aad780464fad added a function called nfscl_delegreturnvp()
    to return delegations during the NFS VOP_RECLAIM().
    The function erroneously assumed that nm_clp would
    be non-NULL. It will be NULL for NFSV4.0 mounts until
    a regular file is opened. It will also be NULL during
    vflush() in nfs_unmount() for a forced dismount.

    This patch adds a check for clp == NULL to fix this.

    Also, since it makes no sense to call nfscl_delegreturnvp()
    during a forced dismount, the patch adds a check for that
    case and does not do the call during forced dismounts.

    PR:     255436
    (cherry picked from commit f6fec55fe30088bbefd3efe70b62565399a7b9b8)

 sys/fs/nfsclient/nfs_clnode.c  | 9 ++++++++-
 sys/fs/nfsclient/nfs_clstate.c | 6 ++++--
 2 files changed, 12 insertions(+), 3 deletions(-)
Comment 9 commit-hook freebsd_committer freebsd_triage 2021-05-12 02:43:52 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=8686e00fdfac505cded6d8f4a4bcf8e51fc10349

commit 8686e00fdfac505cded6d8f4a4bcf8e51fc10349
Author:     Rick Macklem <rmacklem@FreeBSD.org>
AuthorDate: 2021-04-28 00:30:16 +0000
Commit:     Rick Macklem <rmacklem@FreeBSD.org>
CommitDate: 2021-05-12 02:38:42 +0000

    nfscl: add check for NULL clp and forced dismounts to nfscl_delegreturnvp()

    Commit aad780464fad added a function called nfscl_delegreturnvp()
    to return delegations during the NFS VOP_RECLAIM().
    The function erroneously assumed that nm_clp would
    be non-NULL. It will be NULL for NFSV4.0 mounts until
    a regular file is opened. It will also be NULL during
    vflush() in nfs_unmount() for a forced dismount.

    This patch adds a check for clp == NULL to fix this.

    Also, since it makes no sense to call nfscl_delegreturnvp()
    during a forced dismount, the patch adds a check for that
    case and does not do the call during forced dismounts.

    PR:     255436
    (cherry picked from commit f6fec55fe30088bbefd3efe70b62565399a7b9b8)

 sys/fs/nfsclient/nfs_clnode.c  | 10 +++++++++-
 sys/fs/nfsclient/nfs_clstate.c |  6 ++++--
 2 files changed, 13 insertions(+), 3 deletions(-)