Bug 168947

Summary: [nfs] [zfs] .zfs/snapshot directory is messed up when viewed by a Linux client, and ls -l can hang it
Product: Base System Reporter: Peter Maloney <peter.maloney>
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Open ---    
Severity: Affects Only Me CC: grahamperrin
Priority: Normal    
Version: 8.3-STABLE   
Hardware: Any   
OS: Any   

Description Peter Maloney 2012-06-11 11:20:11 UTC
I set this to serious/medium since it can cause hangs, and not critical since it is only one server and I can work around it, and I'm not sure if it still hangs with the latest FreeBSD I'm running.

The messed up .zfs snapshot directory 
(non-critical, but likely related): 

FreeBSD local (FreeBSD nfs client looks the same):
root@bcnas1bak:/tank/.zfs/snapshot# ls -ld daily*
drwxr-xr-x  29 root  wheel  29 Apr 11 13:47 daily-2012-04-13T00:00:00
drwxr-xr-x  29 root  wheel  29 Apr 11 13:47 daily-2012-04-14T00:00:01
drwxr-xr-x  29 root  wheel  29 Apr 11 13:47 daily-2012-04-15T00:00:00
drwxr-xr-x  29 root  wheel  29 Apr 11 13:47 daily-2012-04-16T00:00:00
drwxr-xr-x  29 root  wheel  29 Apr 11 13:47 daily-2012-04-17T00:00:00
drwxr-xr-x  29 root  wheel  29 Apr 11 13:47 daily-2012-04-18T00:00:00

Linux nfs client:
# mount -t nfs -o nolock bcnas1bak:/tank/bcnasvm1 tank
# cd tank/.zfs/snapshot
# ls -ld daily*

(this line looks fine)
drwxr-xr-x 29 root  root         29 Apr 11 13:47 daily-2012-04-13T00:00:00

(this line is a file, not a directory, and has wrong uid and gid)
-r-xr-xr-x  1 peter vboxusers 50384 Jan 18  2010 daily-2012-06-10T00:18:00

(this line is a link, not a directory)
lrwxrwxrwx  1 root  root         21 May 17  2011 daily-2012-05-20T00:18:00 -> package-x-generic.png


The hang (last seen 8.2-STABLE 

But unfortunately, I have failed every attempt to create a 2nd server that will hang. Only one production server hangs this way. The replicated backup server with all the same snapshots (shown above) does not hang.

The hang is likely not related to using up all memory mounting the snapshots. I can also hang it just as easily mounting a small number of them. After running "ls -l" (after a fresh reboot due to previous PR I submitted) on the Linux nfs client, arc is only using 667.29 MiB, and the system has 44 GB free.

When it hangs, only the one dataset is affected. All nfs services are frozen. "ls" in the dataset directories hangs. If I remember correctly, "zpool" commands work, and "zfs list" will hang, but "zfs list tank/someunrelateddataset" will not.

Also note I have not hung my production server this way yet with 8.3. The last hang was with 8.2-STABLE Sept2011.

more info...

Sometimes I would see messages like this:
Oct 31 14:55:39 bcnas1 mountd[47733]: can't delete exports for /tank/.zfs/snapshot/daily-2011-10-06T09:27:52: Invalid argument

But lately I don't, possibly because I hid the .zfs directory with "zfs set snapdir=hidden tank". Or possibly because of a newer FreeBSD.

How-To-Repeat: To repeat the messed up looking snapshot directory problem, just take a snapshot every day (or maybe just make lots all at once), (and then maybe restart nfs or reboot,) and then connect with a Linux client, and use ls -l in the .zfs/snapshot directory.
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2012-06-17 23:37:26 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-fs

Over to maintainer(s).
Comment 2 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:59:18 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped
Comment 3 Graham Perrin 2021-09-26 13:30:18 UTC
With OpenZFS in FreeBSD, is this still an issue?