Bug 210671 - 'du' may report smaller than expected size(s) when using zfs
Summary: 'du' may report smaller than expected size(s) when using zfs
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: 10.3-STABLE
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-bugs mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-06-28 19:20 UTC by ASX
Modified: 2018-04-02 02:53 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description ASX 2016-06-28 19:20:22 UTC
Copying/creating new files and running 'du' immediately after lead to unexpected output, reporting (smaller) size(s).

This happen when using ZFS and apparently doesn't happen when using UFS.

Step to reproduce the problem:
create a script that:
- copy an enough large number of files/size (in a empty directory)
- run "du -s directory/*"

A preliminry examination on FreeBSD forum is here:
https://forums.freebsd.org/threads/56763/

test script:
mkdir test
cp /boot/kernel/kernel test/kernel1
cp /boot/kernel/kernel test/kernel2
cp /boot/kernel/kernel test/kernel3
cp /boot/kernel/kernel test/kernel4
cp /boot/kernel/kernel test/kernel5
cp /boot/kernel/kernel test/kernel6
cp /boot/kernel/kernel test/kernel7
cp /boot/kernel/kernel test/kernel8
ls -l test
du -s test/*

unexpected result:
total 64748
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel1
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel2
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel3
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel4
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel5
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel6
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel7
-r-xr-xr-x  1 root  wheel  21551736 Jun 28 21:06 kernel8
21153	test/kernel1
21153	test/kernel2
21153	test/kernel3
1289	test/kernel4 <--
1	test/kernel5 <--
1	test/kernel6 <--
1	test/kernel7 <--
1	test/kernel8 <--

note that running 'du' a few seconds later will give the correct result/sizes.
Comment 1 Curtis Dunham 2018-04-02 02:53:36 UTC
Can confirm this exact issue on 11.1-RELEASE.
ZFS filesystem (not UFS) does not show the correct disk usage of a file until after (presumably, per [1]) the transaction group commits.  My timings agree with the 6 second sync interval.  I do not agree that this is a feature.

Causes bup [2]'s t/test-sparse-files.sh testcase to fail as it checks disk usage immediately after (test-) restoring a backup.

Therefore it appears the failure is benign.

[1] https://forums.freebsd.org/threads/du-behavior.56763/#post-323135
[2] https://github.com/bup/bup