Bug 190735 - truncate(1) integer overflow issues with size command line arg -- diff with unit tests attached
Summary: truncate(1) integer overflow issues with size command line arg -- diff with u...
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 10.0-RELEASE
Hardware: Any Any
: --- Affects Some People
Assignee: Jilles Tjoelker
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-06-06 23:56 UTC by Kirk Russell
Modified: 2014-12-21 23:23 UTC (History)
2 users (show)

See Also:
jilles: mfc-stable10+
jilles: mfc-stable9+
jilles: mfc-stable8-


Attachments
Diff that fixes bug. And adds new unit tests. (16.74 KB, text/plain)
2014-06-06 23:56 UTC, Kirk Russell
no flags Details
A second patch with the recommend changes. (13.66 KB, patch)
2014-06-07 22:39 UTC, Kirk Russell
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Kirk Russell 2014-06-06 23:56:49 UTC
Created attachment 143473 [details]
Diff that fixes bug.  And adds new unit tests.

When you use the FreeBSD 8.4 /usr/bin/truncate, things work as expected.

- get the binary from Freebsd 8.4
# tar xvf FreeBSD-8.4-RELEASE-amd64-livefs.iso usr/bin/truncate
x usr/bin/truncate

- fail gracefully if the size is too large for int64_t
# ./usr/bin/truncate -s8388608t afile
truncate: invalid size argument `8388608t'

- using a size suffix to shrink a file
# ./usr/bin/truncate -s16t afile
# ./usr/bin/truncate -s-15t afile
# ls -lh afile 
-rw-r--r--  1 root  wheel   1.0T Jun  6 19:44 afile


With the FreeBSD 10.0 /usr/bin/truncate, we get unexpected behavior:

- when the size is too large for int64_t, we expect an error, not a zero size file
# /usr/bin/truncate -s8388608t afile
# ls -l afile
-rw-r--r--  1 root  wheel  0 Jun  6 19:47 afile

- using a size suffix to shrink a file is no longer supported
# /usr/bin/truncate -s16t afile
# /usr/bin/truncate -s-15t afile
truncate: invalid size argument `-15t'


I think this is the change that caused this regression:
http://svnweb.freebsd.org/base?view=revision&revision=204654

I have attached a diff that attempts to fix this issue.  And also adds tests using the newer kyua format.
Comment 1 Jilles Tjoelker freebsd_committer freebsd_triage 2014-06-07 16:24:08 UTC
This looks useful.

Is it possible to do without the new undocumented -D option? Perhaps you could ktrace for truncate(2) calls or add numbers to error messages. (A truncate() to something close to OFF_MAX should certainly not be expected to work, so checking for sizes of files may not be sufficient.)

Instead of the PRIdMAX macro from <inttypes.h>, please use "jd".
Comment 2 Kirk Russell 2014-06-07 22:37:02 UTC
Jilles, thanks for looking!

I have removed the new undocumented -D option and stopped using the macro from <inttypes.h>.  I like your idea of using ktrace but I will want to research that idea before including that in a diff -- I will remove this ktrace/-D stuff from the patch -- I hope that is okay.  I will attach a new patch to this bug, with the new changes.  

You can clarify your comment the expectations of off_t being OFF_MAX?  My idea is the truncate utility needs to indicate an error if OFF_MAX < size < OFF_MIN.  The function humanize_number() will allow numbers out-of-range for off_t and we should tell the user as soon as possible about this conversion problem.   What the OS does with an large off_t passed to truncate() is the OS's business.  For example, it is possible to create a huge sparse file in ZFS.  I am not sure what this sparse file is useful for, but it is nice to see that ls -h works.

# printf "%d\n" 0x7fffffffffffffff
9223372036854775807
# /usr/obj/usr/src/usr.bin/truncate/truncate -s 9223372036854775807 sparse.txt
# ls -l sparse.txt 
-rw-r--r--  1 root  wheel  9223372036854775807 Jun  7 18:30 sparse.txt
# ls -lh sparse.txt 
-rw-r--r--  1 root  wheel   8.0E Jun  7 18:33 sparse.txt
Comment 3 Kirk Russell 2014-06-07 22:39:11 UTC
Created attachment 143501 [details]
A second patch with the recommend changes.
Comment 4 commit-hook freebsd_committer freebsd_triage 2014-06-09 10:40:27 UTC
A commit references this bug:

Author: jilles
Date: Mon Jun  9 10:39:56 UTC 2014
New revision: 267265
URL: http://svnweb.freebsd.org/changeset/base/267265

Log:
  truncate: Detect integer overflow, fix relative sizes, add tests.

  The change to expand_number (r204654) broke detection of too large sizes and
  relative sizes ('+'/'-').

  Also add some tests.

  PR:		190735
  Submitted by:	Kirk Russell
  MFC after:	1 week

Changes:
  head/etc/mtree/BSD.tests.dist
  head/usr.bin/truncate/Makefile
  head/usr.bin/truncate/tests/
  head/usr.bin/truncate/tests/Makefile
  head/usr.bin/truncate/tests/truncate_test.sh
  head/usr.bin/truncate/truncate.c
Comment 5 Jilles Tjoelker freebsd_committer freebsd_triage 2014-06-09 10:44:15 UTC
Applied to 11-current, with some changes (fixing buildworld/installworld WITH_TESTS=YES, removing duplicate test, fixing some test descriptions).
Comment 6 Jilles Tjoelker freebsd_committer freebsd_triage 2014-10-29 23:04:14 UTC
Update bug to new MFC flags.
Comment 7 commit-hook freebsd_committer freebsd_triage 2014-12-07 22:31:39 UTC
A commit references this bug:

Author: jilles
Date: Sun Dec  7 22:30:55 UTC 2014
New revision: 275585
URL: https://svnweb.freebsd.org/changeset/base/275585

Log:
  MFC r267265: truncate: Detect integer overflow, fix relative sizes, add tests.

  The change to expand_number (r204654) broke detection of too large sizes and
  relative sizes ('+'/'-').

  Also add some tests.

  The usr.bin/truncate/Makefile is slightly different in the MFC because
  src.opts.mk does not exist in stable/10.

  PR:		190735
  Submitted by:	Kirk Russell

Changes:
_U  stable/10/
  stable/10/etc/mtree/BSD.tests.dist
  stable/10/usr.bin/truncate/Makefile
  stable/10/usr.bin/truncate/tests/
  stable/10/usr.bin/truncate/truncate.c
Comment 8 commit-hook freebsd_committer freebsd_triage 2014-12-21 23:22:31 UTC
A commit references this bug:

Author: jilles
Date: Sun Dec 21 23:22:14 UTC 2014
New revision: 276041
URL: https://svnweb.freebsd.org/changeset/base/276041

Log:
  MFC r267265: truncate: Detect integer overflow, fix relative sizes.

  The change to expand_number (r204654) broke detection of too large sizes and
  relative sizes ('+'/'-').

  There are no tests in the MFC because stable/9 lacks the test framework.

  PR:		190735
  Submitted by:	Kirk Russell

Changes:
_U  stable/9/usr.bin/truncate/
  stable/9/usr.bin/truncate/truncate.c