Bug 234454 - ar produces corrupt .a archive when >4GB
Summary: ar produces corrupt .a archive when >4GB
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 12.0-RELEASE
Hardware: Any Any
: --- Affects Only Me
Assignee: Ed Maste
URL:
Keywords:
Depends on:
Blocks: 233094
  Show dependency treegraph
 
Reported: 2018-12-28 01:20 UTC by Ed Maste
Modified: 2019-04-29 18:58 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Maste freebsd_committer 2018-12-28 01:20:18 UTC
Our ar (usr.bin/ar) only supports the 32-bit symbol table format, not the 64-bit /SYM64/ format, and silently truncates symbol offsets >4GB.

At the very least we should exit with an error in this case, but we should really support /SYM64/.
Comment 1 commit-hook freebsd_committer 2018-12-28 17:00:29 UTC
A commit references this bug:

Author: emaste
Date: Fri Dec 28 17:00:13 UTC 2018
New revision: 342575
URL: https://svnweb.freebsd.org/changeset/base/342575

Log:
  ar: detect and error out on 32-bit symbol table overflow

  BSD ar currently does not support the /SYM64/ 64-bit symbol table, and
  previously truncated to 32-bits, silently producing corrupted archives
  larger than 4GB.

  Note that this is only a partial fix; additional checks will come.

  PR:		234454
  Reported by:	Aijaz Baig, imp
  MFC after:	2 weeks
  Sponsored by:	The FreeBSD Foundation

Changes:
  head/usr.bin/ar/write.c
Comment 2 commit-hook freebsd_committer 2018-12-28 22:48:27 UTC
A commit references this bug:

Author: emaste
Date: Fri Dec 28 22:47:56 UTC 2018
New revision: 342580
URL: https://svnweb.freebsd.org/changeset/base/342580

Log:
  ar: detect and error out on 32-bit symbol table overflow

  BSD ar currently does not support the /SYM64/ 64-bit symbol table, and
  previously truncated to 32-bits, silently producing corrupted archives
  larger than 4GB.

  This is another overflow case in addtion to r342575.

  PR:		234454
  Reported by:	Aijaz Baig, imp
  MFC after:	2 weeks
  MFC with:	r342575
  Sponsored by:	The FreeBSD Foundation

Changes:
  head/usr.bin/ar/write.c
Comment 3 commit-hook freebsd_committer 2019-01-11 19:49:42 UTC
A commit references this bug:

Author: emaste
Date: Fri Jan 11 19:49:19 UTC 2019
New revision: 342949
URL: https://svnweb.freebsd.org/changeset/base/342949

Log:
  MFC r342575, r342580: ar: detect and error out on 32-bit symbol table overflow

  BSD ar currently does not support the /SYM64/ 64-bit symbol table, and
  previously truncated to 32-bits, silently producing corrupted archives
  larger than 4GB.

  PR:		234454

Changes:
_U  stable/12/
  stable/12/usr.bin/ar/write.c
Comment 4 commit-hook freebsd_committer 2019-01-11 19:54:50 UTC
A commit references this bug:

Author: emaste
Date: Fri Jan 11 19:53:45 UTC 2019
New revision: 342950
URL: https://svnweb.freebsd.org/changeset/base/342950

Log:
  MFC r342575, r342580: ar: detect and error out on 32-bit symbol table overflow

  BSD ar currently does not support the /SYM64/ 64-bit symbol table, and
  previously truncated to 32-bits, silently producing corrupted archives
  larger than 4GB.

  PR:		234454

Changes:
  stable/11/usr.bin/ar/write.c
Comment 5 commit-hook freebsd_committer 2019-04-10 13:13:43 UTC
A commit references this bug:

Author: emaste
Date: Wed Apr 10 13:13:35 UTC 2019
New revision: 346079
URL: https://svnweb.freebsd.org/changeset/base/346079

Log:
  ar: implement support for /SYM64/ 64-bit archives

  PR:		234454
  Submitted by:	Gerald Aryeetey <aryeeteygerald_rogers.com>
  Reviewed by:	imp (earlier)
  MFC after:	3 weeks
  Sponsored by:	The FreeBSD Foundation
  Differential Revision:	https://reviews.freebsd.org/D18793

Changes:
  head/usr.bin/ar/ar.h
  head/usr.bin/ar/read.c
  head/usr.bin/ar/write.c
Comment 6 Ed Maste freebsd_committer 2019-04-10 13:15:03 UTC
Upstream ELF Tool Chain ticket: https://sourceforge.net/p/elftoolchain/tickets/567/
Comment 7 commit-hook freebsd_committer 2019-04-29 18:35:15 UTC
A commit references this bug:

Author: emaste
Date: Mon Apr 29 18:34:12 UTC 2019
New revision: 346902
URL: https://svnweb.freebsd.org/changeset/base/346902

Log:
  MFC ar: implement support for /SYM64/ 64-bit archives

  r346079: ar: implement support for /SYM64/ 64-bit archives

  Submitted by:	Gerald Aryeetey <aryeeteygerald_rogers.com>
  Reviewed by:	imp (earlier)

  r346568: ar: test for writing 64-bit format only if symbol count is nonzero

  This is a minor simplification; if we do not have any symbols the empty
  symbol table can be in 32-bit format.

  r346569: ar: use array notation to access s_so

  This is somewhat more readable than pointer arithmetic.  Also remove an
  unnecessary cast while here.

  r346582: ar: shuffle symbol offsets during conversion for 32-bit ar archives

  During processing we maintain symbol offsets in the 64-bit s_so array,
  and when writing the archive convert to 32-bit if no offsets are greater
  than 4GB.  However, this was somewhat inefficient as we looped over the
  array twice: first, converting to big endian and second, writing each
  32-bit value one at a time (and incorrectly so on big-endian platforms).

  Instead, when writing a 32-bit archive shuffle convert symbol data to
  big endian (as required by the ar format) and shuffle to the beginning
  of the allocation at the same time.

  Also correct emission of the symbol count on big endian platforms.

  Further changes are planned, but this should fix powerpc64.

  Reported by:	jhibbits, mlinimon
  Reviewed by:	jhibbits, Gerald Aryeetey (earlier)
  Tested by:	jhibbits

  PR:		234454
  Sponsored by:	The FreeBSD Foundation

Changes:
_U  stable/12/
  stable/12/usr.bin/ar/ar.h
  stable/12/usr.bin/ar/read.c
  stable/12/usr.bin/ar/write.c
Comment 8 commit-hook freebsd_committer 2019-04-29 18:38:19 UTC
A commit references this bug:

Author: emaste
Date: Mon Apr 29 18:37:40 UTC 2019
New revision: 346903
URL: https://svnweb.freebsd.org/changeset/base/346903

Log:
  MFC ar: implement support for /SYM64/ 64-bit archives

  r346079: ar: implement support for /SYM64/ 64-bit archives

  Submitted by:	Gerald Aryeetey <aryeeteygerald_rogers.com>
  Reviewed by:	imp (earlier)

  r346568: ar: test for writing 64-bit format only if symbol count is nonzero

  This is a minor simplification; if we do not have any symbols the empty
  symbol table can be in 32-bit format.

  r346569: ar: use array notation to access s_so

  This is somewhat more readable than pointer arithmetic.  Also remove an
  unnecessary cast while here.

  r346582: ar: shuffle symbol offsets during conversion for 32-bit ar archives

  During processing we maintain symbol offsets in the 64-bit s_so array,
  and when writing the archive convert to 32-bit if no offsets are greater
  than 4GB.  However, this was somewhat inefficient as we looped over the
  array twice: first, converting to big endian and second, writing each
  32-bit value one at a time (and incorrectly so on big-endian platforms).

  Instead, when writing a 32-bit archive shuffle convert symbol data to
  big endian (as required by the ar format) and shuffle to the beginning
  of the allocation at the same time.

  Also correct emission of the symbol count on big endian platforms.

  Further changes are planned, but this should fix powerpc64.

  Reported by:	jhibbits, mlinimon
  Reviewed by:	jhibbits, Gerald Aryeetey (earlier)
  Tested by:	jhibbits

  PR:		234454
  Sponsored by:	The FreeBSD Foundation

Changes:
_U  stable/11/
  stable/11/usr.bin/ar/ar.h
  stable/11/usr.bin/ar/read.c
  stable/11/usr.bin/ar/write.c