Bug 248286 - ZFS deadlock when rename the volumes
Summary: ZFS deadlock when rename the volumes
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.1-RELEASE
Hardware: amd64 Any
: --- Affects Many People
Assignee: Andriy Gapon
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-07-27 00:00 UTC by minmin
Modified: 2020-08-25 17:43 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description minmin 2020-07-27 00:00:11 UTC
I use bhyve very heavily, and when I ran zfs rename to clean up the VM's disk image (ZFS Volume), the ZFS deadlocked inside the kernel. The procedure to recreate it is as follows.

==========================
$ gpart add -s 20g -t freebsd-zfs -a 4k -l zdead1 ada0
ada0p9 added

$ zpool create -O compression=lz4 -m none zbad ada0p9

$ zfs list -r zbad
NAME   USED  AVAIL  REFER  MOUNTPOINT
zbad   334K  18.9G    23K  none

$ xz -dc ~minmin/tmp/zfs/zfs-deadlock-volume.zfs.xz | zfs receive zbad/vm

$ zfs list -r zbad
NAME		    USED  AVAIL  REFER	MOUNTPOINT
zbad		   2.84G  16.1G    23K	none
zbad/vm 	   2.83G  16.1G    23K	none
zbad/vm/debian10    975M  16.1G   961M	-
zbad/vm/freebsd32   977M  16.1G   977M	-
zbad/vm/idmdeb10    947M  16.1G  1.55G	-

$ zfs create zbad/vmn

$ zfs list -r zbad
NAME		    USED  AVAIL  REFER	MOUNTPOINT
zbad		   2.84G  16.1G    23K	none
zbad/vm 	   2.83G  16.1G    23K	none
zbad/vm/debian10    975M  16.1G   961M	-
zbad/vm/freebsd32   977M  16.1G   977M	-
zbad/vm/idmdeb10    947M  16.1G  1.55G	-
zbad/vmn	     23K  16.1G    23K	none

$ cat /var/tmp/zfs/deadlock
pool=zbad
zfs rename ${pool}/vm/idmdeb10	${pool}/vmn/idmdeb10
zfs rename ${pool}/vm/freebsd32 ${pool}/vmn/freebsd32

$ sh -x /var/tmp/zfs/deadlock
+ pool=zbad
+ zfs rename zbad/vm/idmdeb10 zbad/vmn/idmdeb10
+ zfs rename zbad/vm/freebsd32 zbad/vmn/freebsd32
==========================

The second rename operation is not finished.

When sleep is placed between rename operations, deadlock does not occur and the operation ends normally.

"debian10" and "freebsd32" are bhyve disk images and "idmdeb10" is a ZFS volume cloned from "debian10".

I ran the same test on multiple systems and all of them had deadlocks.

If you need it for the replication test, I can provide you with the "zfs-deadlock-volume.zfs.xz" image (about 1.5GB) that I used for the test.
Comment 1 Andriy Gapon freebsd_committer freebsd_triage 2020-07-27 08:44:27 UTC
Could you please merge base r362047 and base r362048 and see if those help?
Comment 2 minmin 2020-07-28 00:36:46 UTC
Ok. I will test with merge base r362047 and base r362048.
Comment 3 minmin 2020-07-28 00:41:27 UTC
BTW
I already tested with OpenZFS (openzfs-2020061900 and openzfs-kmod-2020061900).  It works fine well.
Comment 4 minmin 2020-07-30 06:46:22 UTC
I built and tested a merged kernel up to base r362048.
No deadlocks were encountered. I think this fix should work.