Bug 252240

Summary: "zfs send -R -I | zfs receive -F" not destroying snapshots that don't exist on the sending side
Product: Base System Reporter: Lewis Donzis <lew>
Component: kernAssignee: freebsd-fs (Nobody) <fs>
Status: New ---    
Severity: Affects Many People CC: jmc-freebsd2, john, rpp
Priority: ---    
Version: 12.2-RELEASE   
Hardware: Any   
OS: Any   

Description Lewis Donzis 2020-12-28 21:06:33 UTC
This is similar to 169964, but that was due to lack of the -R option, so I'm submitting this as a new bug.

This appears to be new to 12.2-RELEASE. If the receiving side is running 12.1 or 12.0, everything works properly, but if the receiving side is running 12.2, then snapshots that don't exist on the sending side are not destroyed on the receiving side.

The version of the sending side doesn't appear to make any difference, the problem appears to be strictly on the receive side, and consistently fails with 12.2 and withs with 12.1 or prior. In fact, on a given system, running freebsd-update to upgrade from 12.1 to 12.2 causes the problem to appear, and reverting back to 12.1, the problem disappears.

Sample commands:

zfs create ds/test
zfs snapshot ds/test@1
zfs snapshot ds/test@2
zfs snapshot ds/test@3
zfs snapshot ds/test@4
zfs snapshot ds/test@5
zfs send -R ds/test@5 | ssh desthost zfs receive -F ds/test

desthost now has the filesystem and all snapshots.

zfs snapshot ds/test@6
zfs send -R -I ds/test@5 ds/test@6 | ssh desthost zfs receive -F ds/test

desthost now has the test@6 snapshot

zfs destroy ds/test@2
zfs destroy ds/test@3
zfs send -R -I ds/test@5 ds/test@6 | ssh desthost zfs receive -F ds/test

If desthost is running 12.2, then test@2 and test@3 remain. If desthost is running 12.1 or 12.0, then test@2 and test@3 are destroyed.

And it doesn't matter whether the snapshot has previously synced or not, for example:

zfs snapshot ds/test@7
zfs send -R -I ds/test@6 ds/test@7 | ssh desthost zfs receive -F ds/test

still doesn't cause @2 and @3 to get destroyed.  However,

zfs send -R -I ds/test@1 ds/test@7 | ssh desthost zfs receive -F ds/test

does cause @2 and @3 to get destroyed.  But that's not an acceptable work-around, and this appears to be a regression from prior versions.
Comment 1 John Hay 2020-12-30 05:15:50 UTC
I also experienced this. I have a script running for many years that would create a new snapshot everyday and destroy snapshots older than a week. It would then do

zfs send -R -I filesystem@yesterday today

On the receiving side, it would do

zfs receive -Fduv localfilesystem

In the past, this would also destroy filesystems on the receive side as they get destroyed on the send side. After upgrading the receive machine to 12.2, it stopped destroying the old snapshots that had been destroyed on the send side. The send machine has not been upgraded yet. It is still running 12.1-p4.
Comment 2 rpp 2021-01-07 05:31:02 UTC
This is a me-too. We first encountered this in 12.1 revision r363264 so it predates 12.2 release. Its a showstopper for us, preventing us upgrading our 3 file servers from 11.3 (r349924). 
We have a couple of VMs set up that can quickly replicate the problem, so we can assist with testing if needed.
Comment 3 John Hay 2021-01-30 06:34:48 UTC
This has been fixed for me in 12.2-RELEASE-p3.