Created attachment 150961 [details] patch -- you need to run one svn cp before you run patch RFC: adding posix_fallocate(2) support to /usr/bin/truncate TL;DR ===== I have provided a patch that gives minimal posix_fallocate(2) support to /usr/bin/truncate. Now truncate(1) gives the user a choice create a sparse file or to create a pre-allocated space on disk. Before you apply the patch, you need to svn cp an existing file: svn cp usr.bin/truncate/tests/truncate_test.sh usr.bin/truncate/tests/base_test.func Problem to Solve ================ The truncate utility is often used to create large files used as disk images by emulators or as the backing storage for memory disks. One issue is that a sparse file is created -- the actual disk space is not allocated until you write. Some users might prefer to get the out-of-space errors when you initially create the large file, instead of getting out-of-space errors during future writes. The man page for truncate(1) suggests using dd(2) to create a pre-allocate space. The idiom is to use dd(2) to read data from /dev/zero and write zeroes to the file, causing the space to be allocated on disk. With the existence of the new kernel call posix_fallocate(2), it seems odd to writes zeroes to pre-allocate space. It might be useful to have a command line utility call posix_fallocate(2) as an alternative to the write zeroes idiom. Today, there is no FreeBSD command line utility that uses posix_fallocate(2). Proposed Solution ================= Add a new command line flag "-a" to the truncate(1) command line utility. When "-a" is present, truncate(1) will use posix_fallocate(2) when increasing the size of a file. Examples ======== rm -f afile truncate -s 1m afile Creates a 1048576 byte file -- truncate(2) will be used so the file is expected to be a sparse file -- not all the 1048576 bytes will be allocated to disk. rm -f afile truncate -a -s 1m afile Creates a 1048576 byte file -- posix_fallocate(2) will be used so all the 1048576 bytes will be allocated to disk -- no holes are expected to be present. rm -f afile truncate -s 1m afile truncate -a -s +1m afile Creates a 2097152 byte file -- truncate(2) will be used for the first 1048576 bytes and posix_fallocate(2) will be used for the last 1048576 bytes. The last 1048576 bytes will be allocated to disk. At least one hole is expected to be present at the start of this file. Proposed Test Strategy ====================== The existing truncate_test.sh tests can be run twice -- once with the proposed "-a" command line flag and once without the new flag. Ignoring the allocation of disk space, the behaviour of the truncate(1) utility should be the same in both cases. This will require the existing tests to be moved into a shared file used by the existing truncate_test.sh tests and a new script that will use the proposed "-a" command line flag. A new test needs to be written that will detect if the new file created by truncate(1) is sparse or not. This test will be written in C and use lseek(2)'s SEEK_HOLE feature to count the number of sparse bytes in a file. It is expected that posix_fallocate(2) will create files with a sparse byte count of zero, while truncate(2) will create files with a sparse byte count that is greater than zero. Rejected Solutions ================== 1) GNU coreutils provides both truncate and fallocate command line utilities. The GNU coreutils fallocate command line utility that will use posix_fallocate(2) for creating pre-allocated files. The FreeBSD truncate(1) utility's command line args are compatible with the GNU coreutils truncate utility -- both utilities create sparse files. FreeBSD does not have a fallocate utility. It was assumed that adding a new command line utility to FreeBSD would not be plausible. This proposal adds minimal posix_fallocate(2) support to the existing truncate(1) utility, instead of adding another command line utility. This proposed "-a" FreeBSD extension is not command line compatible with GNU coreutils truncate or fallocate. 2) It should be possible to use posix_fallocate(2) to target an existing sparse file. This should replace the existing holes with equivalently-zeroed allocated disk space. It seemed odd to add this feature to truncate(1). See Also ======== posix_fallocate(2), truncate(2), truncate(1), mdconfig(8), dd(2), lseek(2)
Adding a new fallocate command would be fine. I would prefer this for compatibility rather than adding the more incompatible '-a' flag. If it makes sense, the truncate.c can be made to change behavior to fallocate(1) when invoked through a hard link. We do this in usr.bin/w/w.c already to change behavior for uptime(1).
(In reply to Bryan Drewery from comment #1) Bryan, thanks for looking at my patch! I wrote a version of the fallocate utility that would be just a hard link to the truncate utility. I put it up on github for now. https://github.com/kirussel/bsd-truncate-utils/compare/fallocate How did you want to proceed?
(In reply to Kirk Russell from comment #2) > (In reply to Bryan Drewery from comment #1) > > Bryan, thanks for looking at my patch! > > I wrote a version of the fallocate utility that would be just a hard link to > the truncate utility. > > I put it up on github for now. > https://github.com/kirussel/bsd-truncate-utils/compare/fallocate > > How did you want to proceed? Great. I'll try to look later this week.