Bug 210671

Summary: 'du' may report smaller than expected size(s) when using zfs
Product: Base System Reporter: ASX <as.asx64>
Component: miscAssignee: freebsd-bugs (Nobody) <bugs>
Status: New ---    
Severity: Affects Some People CC: amd64, cornelius.mund, curtis.dunham
Priority: ---    
Version: 10.3-STABLE   
Hardware: amd64   
OS: Any   

Description ASX 2016-06-28 19:20:22 UTC
Copying/creating new files and running 'du' immediately after lead to unexpected output, reporting (smaller) size(s).

This happen when using ZFS and apparently doesn't happen when using UFS.

Step to reproduce the problem:
create a script that:
- copy an enough large number of files/size (in a empty directory)
- run "du -s directory/*"

A preliminry examination on FreeBSD forum is here:
https://forums.freebsd.org/threads/56763/

test script:
mkdir test
cp /boot/kernel/kernel test/kernel1
cp /boot/kernel/kernel test/kernel2
cp /boot/kernel/kernel test/kernel3
cp /boot/kernel/kernel test/kernel4
cp /boot/kernel/kernel test/kernel5
cp /boot/kernel/kernel test/kernel6
cp /boot/kernel/kernel test/kernel7
cp /boot/kernel/kernel test/kernel8
ls -l test
du -s test/*

unexpected result:
total 64748
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel1
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel2
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel3
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel4
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel5
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel6
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel7
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel8
21153	test/kernel1
21153	test/kernel2
21153	test/kernel3
1289	test/kernel4 <--
1	test/kernel5 <--
1	test/kernel6 <--
1	test/kernel7 <--
1	test/kernel8 <--

note that running 'du' a few seconds later will give the correct result/sizes.
Comment 1 Curtis Dunham 2018-04-02 02:53:36 UTC
Can confirm this exact issue on 11.1-RELEASE.
ZFS filesystem (not UFS) does not show the correct disk usage of a file until after (presumably, per [1]) the transaction group commits.  My timings agree with the 6 second sync interval.  I do not agree that this is a feature.

Causes bup [2]'s t/test-sparse-files.sh testcase to fail as it checks disk usage immediately after (test-) restoring a backup.

Therefore it appears the failure is benign.

[1] https://forums.freebsd.org/threads/du-behavior.56763/#post-323135
[2] https://github.com/bup/bup
Comment 2 cornelius.mund 2021-08-14 10:58:00 UTC
I can confirm this behavior on ubuntu 20.04.