Bug 254210

Summary: 13.0-RC1: jail: nullfs: deleted files does not free up space
Product: Base System Reporter: rashey
Component: kernAssignee: freebsd-jail (Nobody) <jail>
Status: Open ---    
Severity: Affects Some People CC: chris, crest, dch, fs, grahamperrin, kevans, kib, r4
Priority: ---    
Version: 13.0-STABLE   
Hardware: amd64   
OS: Any   

Description rashey 2021-03-10 23:36:46 UTC
Sometimes it is necessary to share the same directory between jails and in this case there is a issue with free up space after deleting a files.

For example, the file created in jail A and then deleted in jail B will not free up space as long as jail A is running.

# cat /etc/fstab.test1
/storage/data   /usr/jail/test1/storage/data    nullfs          rw,noatime      0       0

# cat /etc/fstab.test2
/storage/data   /usr/jail/test2/storage/data    nullfs          rw,noatime      0       0

# cat /etc/jail.conf
path = "/usr/jail/${name}";
exec.clean;
exec.start      = "sh /etc/rc";
exec.stop       = "sh /etc/rc.shutdown jail";
host.hostname   = "${name}";
mount.fstab     = "/etc/fstab.${name}";
mount.devfs;
test1 {
}
test2 {
}

# df -ht ufs
Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/da0p2    8.9G    3.2G    5.0G    39%    /

# service jail start
Starting jails: test1 test2.

# df -ht ufs
Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/da0p2    8.9G    3.2G    5.0G    39%    /

# jexec test1 dd if=/dev/random of=/storage/data/random.dd bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 3.723429 secs (288374490 bytes/sec)

# df -ht ufs
Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/da0p2    8.9G    4.2G    4.0G    52%    /

# jexec test2 rm /storage/data/random.dd

# df -ht ufs
Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/da0p2    8.9G    4.2G    4.0G    52%    /

# service jail stop test2
Stopping jails: test2.

# df -ht ufs
Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/da0p2    8.9G    4.2G    4.0G    52%    /

# service jail stop test1
Stopping jails: test1.

# df -ht ufs
Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/da0p2    8.9G    3.2G    5.0G    39%    /
Comment 1 Kyle Evans freebsd_committer freebsd_triage 2021-03-13 16:32:13 UTC
Let's loop in fs as well, since this is at the intersection of jails and filesystems.
Comment 2 Konstantin Belousov freebsd_committer freebsd_triage 2021-03-13 18:06:03 UTC
Add "nocache" option to both nullfs mounts.
Comment 3 rashey 2021-03-13 20:23:49 UTC
(In reply to Konstantin Belousov from comment #2)

It is not documented and mount -t nullfs does not indicate that the option is used but it work.

One of the jails is working as file server and there is a lot of read operations.

Should I worry about hard disk head contention?
Comment 4 Ivan 2023-03-30 22:45:09 UTC
this seems to be related
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=245688
Comment 5 Ivan 2023-03-30 23:09:23 UTC
and this
https://marc.info/?l=freebsd-fs&m=150581490210158&w=2

Konstantin Belousov, according to the newsletter, this problem has been going on since at least 2017?
Comment 6 Graham Perrin freebsd_committer freebsd_triage 2023-03-31 01:01:58 UTC
*** Bug 270546 has been marked as a duplicate of this bug. ***
Comment 7 Christos Chatzaras 2023-04-26 11:53:30 UTC
FreeBSD 13.2:

The same thing happens when I use a program to optimise images (the images size shrinks but ZFS shows more usage than before). I have to restart jails to get the correct usage.