Sometimes it is necessary to share the same directory between jails and in this case there is a issue with free up space after deleting a files. For example, the file created in jail A and then deleted in jail B will not free up space as long as jail A is running. # cat /etc/fstab.test1 /storage/data /usr/jail/test1/storage/data nullfs rw,noatime 0 0 # cat /etc/fstab.test2 /storage/data /usr/jail/test2/storage/data nullfs rw,noatime 0 0 # cat /etc/jail.conf path = "/usr/jail/${name}"; exec.clean; exec.start = "sh /etc/rc"; exec.stop = "sh /etc/rc.shutdown jail"; host.hostname = "${name}"; mount.fstab = "/etc/fstab.${name}"; mount.devfs; test1 { } test2 { } # df -ht ufs Filesystem Size Used Avail Capacity Mounted on /dev/da0p2 8.9G 3.2G 5.0G 39% / # service jail start Starting jails: test1 test2. # df -ht ufs Filesystem Size Used Avail Capacity Mounted on /dev/da0p2 8.9G 3.2G 5.0G 39% / # jexec test1 dd if=/dev/random of=/storage/data/random.dd bs=1M count=1024 1024+0 records in 1024+0 records out 1073741824 bytes transferred in 3.723429 secs (288374490 bytes/sec) # df -ht ufs Filesystem Size Used Avail Capacity Mounted on /dev/da0p2 8.9G 4.2G 4.0G 52% / # jexec test2 rm /storage/data/random.dd # df -ht ufs Filesystem Size Used Avail Capacity Mounted on /dev/da0p2 8.9G 4.2G 4.0G 52% / # service jail stop test2 Stopping jails: test2. # df -ht ufs Filesystem Size Used Avail Capacity Mounted on /dev/da0p2 8.9G 4.2G 4.0G 52% / # service jail stop test1 Stopping jails: test1. # df -ht ufs Filesystem Size Used Avail Capacity Mounted on /dev/da0p2 8.9G 3.2G 5.0G 39% /
Let's loop in fs as well, since this is at the intersection of jails and filesystems.
Add "nocache" option to both nullfs mounts.
(In reply to Konstantin Belousov from comment #2) It is not documented and mount -t nullfs does not indicate that the option is used but it work. One of the jails is working as file server and there is a lot of read operations. Should I worry about hard disk head contention?
this seems to be related https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=245688
and this https://marc.info/?l=freebsd-fs&m=150581490210158&w=2 Konstantin Belousov, according to the newsletter, this problem has been going on since at least 2017?
*** Bug 270546 has been marked as a duplicate of this bug. ***
FreeBSD 13.2: The same thing happens when I use a program to optimise images (the images size shrinks but ZFS shows more usage than before). I have to restart jails to get the correct usage.