Bug 243730

Summary: ZFS incompatibility send/receive with Ubuntu 19.10 to FreeBSD 12.x
Product: Base System Reporter: Bert Nijhof <lammert.nijhof>
Component: binAssignee: freebsd-fs (Nobody) <fs>
Status: New ---    
Severity: Affects Some People    
Priority: ---    
Version: 12.1-STABLE   
Hardware: Any   
OS: Any   

Description Bert Nijhof 2020-01-30 14:04:02 UTC
Since June I use send/receive to send incremental backups from my Ubuntu desktop to my FreeBSD 12.x backup-server. After installing Ubuntu 19.10 with root on zfs, the weekly incremental backup failed. After some try outs with help of the community I detected that both systems had the feature large-dnode=active. However Ubuntu had dnodesize=auto and FreeBSD had dnodesize=legacy. Those values were chosen by both installers and not by me. 

I'm dealing with relative large incremental backups of ~250 GB of Virtual Machines, so an incremental backup takes 30-60 minutes. The performance bottleneck is the Pentium 4 (FreeBSD) running at 95% CPU load, while transferring at 200 Mbps.

On FreeBSD I see the size of the dataset increase nicely (Conky using zfs list). When it reaches the end of transfer the snapshot with all data just transferred is destroyed and Ubuntu gives the error message. 

After changing all relevant dnodesizes on Ubuntu to legacy and reloading all data in those datasets, my weekly incremental backups worked again.

I filed an Ubuntu bug-report long ago
https://bugs.launchpad.net/bugs/1854982

Except for helping with the bypass, nothing happened and the 2 organization don't seem to talk to each other. I've tried 12 different combinations of send/receive, but the essence of the incompatibility is described here.

Ubuntu and FreeBSD did choose different defaults for large-dnodes and dnodesizes, but to solve bugs related to feature incompatibility you have to communicate! The problem will not disappear completely, because you start using the same source. There will be probably months between release dates, so feature incompatibility probably will remain an issue.

My problems is bypassed, but the default dnodesize incompatibility between Ubuntu 19.10 and FreeBSD 12.1 remains.