Bug 191587 - [mips] GNU 'strip' corrupts static libraries
Summary: [mips] GNU 'strip' corrupts static libraries
Status: Closed Overcome By Events
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: mips Any
: Normal Affects Some People
Assignee: Ed Maste
URL: http://bpaste.net/show/8y6wgjoHTK5tpa...
Keywords:
Depends on:
Blocks:
 
Reported: 2014-07-03 19:58 UTC by Stacey Son
Modified: 2018-02-14 20:11 UTC (History)
14 users (show)

See Also:


Attachments
unstripped archive from ports-mgmt/pkg (535.25 KB, application/x-xz)
2014-07-25 18:13 UTC, Ed Maste
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stacey Son freebsd_committer 2014-07-03 19:58:59 UTC
In building various ports (such as devel/libpcre) I have noticed that 'strip' corrupts the static library and the port build will fail.  To reproduce do the following with an unstripped static library:

  root@erl:/tmp # ls -al libpcre.a 
  -rw-r--r--  1 sson  wheel  404446 Jun 11 18:52 libpcre.a
  root@erl:/tmp # strip --strip-debug libpcre.a 
  root@erl:/tmp # ls -al libpcre.a 
  -rw-r--r--  1 sson  wheel  403506 Jun 11 18:53 libpcre.a
  root@erl:/tmp # ranlib libpcre.a 
  ranlib: warning: can't find .strtab section
  ranlib: fatal: Invalid filename
  root@erl:/tmp # ls -al libpcre.a
  -rw-r--r--  1 sson  wheel  0 Jun 11 18:53 libpcre.a
Comment 1 Stacey Son freebsd_committer 2014-07-03 20:21:12 UTC
I should have noted that the above was on a MIPS64 Ether Router Lite.  'strip' also seems to corrupt static libraries on the MIPS arch as well but not to the point where 'ranlib' will fail with a "fatal" error.
Comment 2 Ed Maste freebsd_committer 2014-07-24 18:52:31 UTC
I'll see if this is fixed by using elftoolchain's strip
Comment 3 Sean Bruno freebsd_committer 2014-07-24 19:22:55 UTC
adding toolchain people.  I don't think anyone really wants to dig into the gcc tool chain at this point.  But, as always, I never assume I know what I'm talking about.
Comment 4 Ed Maste freebsd_committer 2014-07-25 14:53:01 UTC
elftoolchain's strip does not handle SHT_MIPS_OPTIONS
https://github.com/emaste/elftoolchain/issues/1
Comment 5 Dimitry Andric freebsd_committer 2014-07-25 17:57:01 UTC
Stacey, can you please attach a compressed version of the unstripped libpcre.a, if you still have it?  That would make it a little easier for me to debug.
Comment 6 Ed Maste freebsd_committer 2014-07-25 18:13:01 UTC
Created attachment 144973 [details]
unstripped archive from ports-mgmt/pkg
Comment 7 Ed Maste freebsd_committer 2014-07-25 18:15:07 UTC
dim: I've attached a .a from building ports-mgmt/pkg that also demonstrates the problem.
Comment 8 Sean Bruno freebsd_committer 2014-07-26 23:14:54 UTC
Updated and tried a xdev tool chain and a fully native tool chain in emulation via QEMU.  pkg still seems to be failing:

===>  Staging for pkg-1.3.0
===>   Generating temporary packing list
if test -z 'strip'; then  /usr/bin/make  INSTALL_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s"  install_sh_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" INSTALL_STRIP_FLAG=-s  install;  else  /usr/bin/make  INSTALL_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s"  install_sh_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" INSTALL_STRIP_FLAG=-s  "INSTALL_PROGRAM_ENV=STRIPPROG='strip'" install;  fi
Making install in external
/usr/bin/make  install-am
Making install in libpkg
Making install in repo
Making install in binary
Making install in .
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib'
 /bin/sh ../libtool   --mode=install /usr/bin/install -c -o root -g wheel -s  libpkg.la libpkg_static.la '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib'
libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg.so.3.0.0 /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.so.3.0.0
libtool: install: strip --strip-unneeded /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.so.3.0.0
libtool: install: (cd /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib && { ln -s -f libpkg.so.3.0.0 libpkg.so.3 || { rm -f libpkg.so.3 && ln -s libpkg.so.3.0.0 libpkg.so.3; }; })
libtool: install: (cd /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib && { ln -s -f libpkg.so.3.0.0 libpkg.so || { rm -f libpkg.so && ln -s libpkg.so.3.0.0 libpkg.so; }; })
libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg.lai /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.la
libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg_static.lai /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.la
libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg_static.a /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a
libtool: install: strip --strip-debug /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a
libtool: install: chmod 644 /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a
libtool: install: ranlib /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a
ranlib: fatal: Invalid filename
*** Error code 70

Stop.
make[6]: stopped in /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/libpkg
*** Error code 1

Stop.
make[5]: stopped in /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/libpkg
*** Error code 1

Stop.
make[4]: stopped in /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/libpkg
*** Error code 1

Stop.
make[3]: stopped in /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/ports-mgmt/pkg
*** Error code 1

Stop.
Comment 9 Sean Bruno freebsd_committer 2014-07-26 23:19:57 UTC
adding bapt for tracking.
Comment 10 Baptiste Daroussin freebsd_committer 2014-07-26 23:21:55 UTC
We should never strip static libraries should we?
I breaks way more than only mips
Comment 11 Sean Bruno freebsd_committer 2014-07-27 01:03:21 UTC
This does *not* happen on a mips32 build:

===>  Staging for pkg-1.3.0
===>   Generating temporary packing list
if test -z 'strip'; then  /usr/bin/make  INSTALL_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s"  install_sh_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" INSTALL_STRIP_FLAG=-s  install;  else  /usr/bin/make  INSTALL_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s"  install_sh_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" INSTALL_STRIP_FLAG=-s  "INSTALL_PROGRAM_ENV=STRIPPROG='strip'" install;  fi
Making install in external
/usr/bin/make  install-am
Making install in libpkg
Making install in repo
Making install in binary
Making install in .
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib'
 /bin/sh ../libtool   --mode=install /usr/bin/install -c -o root -g wheel -s  libpkg.la libpkg_static.la '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib'
libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg.so.3.0.0 /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.so.3.0.0
libtool: install: strip --strip-unneeded /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.so.3.0.0
libtool: install: (cd /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib && { ln -s -f libpkg.so.3.0.0 libpkg.so.3 || { rm -f libpkg.so.3 && ln -s libpkg.so.3.0.0 libpkg.so.3; }; })
libtool: install: (cd /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib && { ln -s -f libpkg.so.3.0.0 libpkg.so || { rm -f libpkg.so && ln -s libpkg.so.3.0.0 libpkg.so; }; })
libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg.lai /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.la
libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg_static.lai /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.la
libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg_static.a /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a
libtool: install: strip --strip-debug /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a
libtool: install: chmod 644 /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a
libtool: install: ranlib /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a
libtool: install: warning: remember to run `libtool --finish /usr/local/lib'
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/include'
 install  -o root -g wheel -m 444 pkg.h '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/include'
Making install in src
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc'
 install  -o root -g wheel -m 444 pkg.conf.sample '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc'
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin'
 STRIPPROG='strip' /bin/sh ../libtool   --mode=install /bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s pkg pkg-static '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin'
libtool: install: warning: `../libpkg/libpkg.la' has not been installed in `/usr/local/lib'
libtool: install: /bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s .libs/pkg /usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin/pkg
libtool: install: /bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s pkg-static /usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin/pkg-static
Making install in tests
Making install in scripts
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin'
 install  -o root -g wheel -m 555 sbin/pkg2ng '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin'
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/bash_completion.d'
 install  -o root -g wheel -m 444 completion/_pkg.bash '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/bash_completion.d'
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/daily'
 install  -o root -g wheel -m 444 periodic/411.pkg-backup periodic/490.status-pkg-changes '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/daily'
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/security'
 install  -o root -g wheel -m 444 periodic/410.pkg-audit periodic/460.pkg-checksum '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/security'
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/weekly'
 install  -o root -g wheel -m 444 periodic/400.status-pkg '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/weekly'
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/share/zsh/site-functions'
 install  -o root -g wheel -m 444 completion/_pkg.zsh '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/share/zsh/site-functions'
Making install in docs
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man3'
 install  -o root -g wheel -m 444 pkg_printf.3 pkg_repos.3 '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man3'
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man5'
 install  -o root -g wheel -m 444 pkg-repository.5 pkg.conf.5 '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man5'
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man8'
 install  -o root -g wheel -m 444 pkg-add.8 pkg-annotate.8 pkg-audit.8 pkg-autoremove.8 pkg-backup.8 pkg-check.8 pkg-clean.8 pkg-config.8 pkg-convert.8 pkg-create.8 pkg-delete.8 pkg-fetch.8 pkg-info.8 pkg-install.8 pkg-lock.8 pkg-query.8 pkg-register.8 pkg-repo.8 pkg-rquery.8 pkg-search.8 pkg-set.8 pkg-shell.8 pkg-shlib.8 pkg-ssh.8 pkg-stats.8 pkg-update.8 pkg-updating.8 pkg-upgrade.8 pkg-version.8 pkg-which.8 pkg.8 '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man8'
/usr/bin/make  install-data-hook
ln -s -f pkg-delete.8  /usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man8/pkg-remove.8
ln -s -f pkg-lock.8  /usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man8/pkg-unlock.8
ln -s -f pkg.8  /usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man8/pkg-static.8
 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/libdata/pkgconfig'
 install  -o root -g wheel -m 444 libpkg/pkg.pc '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/libdata/pkgconfig'
====> Compressing man pages (compress-man)
Comment 12 Ed Maste freebsd_committer 2014-07-28 14:55:32 UTC
> We should never strip static libraries should we?
> I breaks way more than only mips

In the pkg case I encountered it's "strip --strip-debug", which should only be removing debug data, and should be fine for static libraries.
Comment 13 Ed Maste freebsd_committer 2014-08-04 16:16:49 UTC
As a workaround the offending "strip --strip-debug" can be disabled in the libtool files by commenting out old_striplib.

-old_striplib="strip --strip-debug"
+#old_striplib="strip --strip-debug"
Comment 14 Sean Bruno freebsd_committer 2014-08-12 00:09:42 UTC
comment out here?

contrib/binutils/binutils/doc/binutils.texi:@item Run @code{strip --strip-debug foo}
Comment 15 Ed Maste freebsd_committer 2014-08-12 13:15:30 UTC
root@erl:/usr/ports/x11/xdpyinfo # diff -u /usr/local/bin/libtool.orig /usr/local/bin/libtool
--- /usr/local/bin/libtool.orig 2014-08-12 13:14:52.000000000 +0000
+++ /usr/local/bin/libtool      2014-08-08 14:45:45.000000000 +0000
@@ -290,7 +290,7 @@
 dlopen_self_static=no
 
 # Commands to strip libraries.
-old_striplib="strip --strip-debug"
+#old_striplib="strip --strip-debug"
 striplib="strip --strip-unneeded"
 
 
root@erl:/usr/ports/x11/xdpyinfo #
Comment 16 daniel.engberg.lists 2014-10-12 10:15:01 UTC
If you want to build pkg (for now)...
make -DWITH_DEBUG install clean

Hardware: Ubnt EdgeRouter Lite
FreeBSD 11.0-CURRENT #0 r272938: Sat Oct 11 23:39:28 UTC 2014
Comment 17 na 2014-11-20 17:09:26 UTC
No need for ports collection to reproduce this bug, simply:

$ echo 'int fun(int x){return x+42;}' > misc.c && cc -c misc.c && ar -rc misc.a misc.o && cp misc.a misc2.a
$ strip misc.a
$ ranlib misc.a
ranlib: warning: can't find .strtab section
ranlib: fatal: Invalid filename
$ ranlib misc2.a
$
Comment 18 Ed Maste freebsd_committer 2014-11-28 22:09:39 UTC
It looks like this is currently fixed in the in-tree libelf, and in upstream elftoolchain libelf. I have work in progress to switch to strip (and other tools) from the elftoolchain project, see https://reviews.freebsd.org/D1224.

After installing strip from there I see:

root@erl:~ # echo 'int fun(int x){return x+42;}' > misc.c && cc -c misc.c && ar -rc misc.a misc.o && cp misc.a misc2.a
root@erl:~ # strip misc.a
root@erl:~ # ranlib misc.a
ranlib: warning: can't find .strtab section
root@erl:~ # ranlib misc2.a
root@erl:~ #
Comment 19 Ed Maste freebsd_committer 2014-11-28 22:31:52 UTC
And for completeness,

# echo 'int fun(int x){return x+42;}' > misc.c && cc -c misc.c && ar -rc misc.a misc.o
# strip --strip-debug misc.a
# ranlib misc.a
# 
# nm misc.a     

misc.o:
0000000000000000 T fun
Comment 20 Ed Maste freebsd_committer 2015-01-14 18:39:28 UTC
WITH_ELFTOOLCHAIN_TOOLS is now the default in HEAD so this issue will be solved there.

Merging it back to stable/10 is not really feasible at the moment, so we'll have to consider what we do for the stable branch.
Comment 21 George V. Neville-Neil freebsd_committer 2015-04-20 23:23:23 UTC
If we can't MFC then do we apply the patch to 10-STABLE?
Comment 22 Ed Maste freebsd_committer 2015-05-06 01:51:13 UTC
There's no patch to apply - the issue is fixed in HEAD by switching from GNU binutils strip to ELF Tool Chain's strip. It would be possible to bring that back to stable/10 as an option, but there's a reasonably long dependency chain to address, and we'd likely need to find a way to install two different copies of libelf.
Comment 23 Ed Maste freebsd_committer 2016-06-13 15:00:57 UTC
This is fixed in -CURRENT with the migration to the ELF Tool Chain version of strip. It is not feasible to merge that change back to stable/10.

MIPS users who cannot move off stable/10 will have to use a workaround like one of those discussed in other comments in this PR.
Comment 24 Sergey 2018-02-14 13:25:08 UTC
Actually the issue preserves on 11.1, 'strip' removes not requested sections:

$ echo 'int fun(int x){return x+42;}' > misc.c && cc -c misc.c && ar -rc misc.a misc.o && cp misc.a misc2.a                               <
$ ll misc*.a                                                                                                                        [ /tmp ]
-rw-r--r--  1 x  wheel  1076 Feb 14 16:20 misc.a
-rw-r--r--  1 x  wheel  1076 Feb 14 16:20 misc2.a
$ strip misc.a                                                                                                                      [ /tmp ]
$ ranlib misc.a                                                                                                                     [ /tmp ]
ranlib: warning: can't find .strtab section
$ readelf --string-dump .strtab misc.a                                                                                              [ /tmp ]

File: misc.a(misc.o)
$ ll misc*.a                                                                                                                        [ /tmp ]
-rw-r--r--  1 x  wheel   868 Feb 14 16:21 misc.a
-rw-r--r--  1 x  wheel  1076 Feb 14 16:20 misc2.a
$ cp misc2.a misc3.a                                                                                                                [ /tmp ]
$ strip -R .comment misc2.a                                                                                                         [ /tmp ]
$ ll misc*.a                                                                                                                        [ /tmp ]
-rw-r--r--  1 x  wheel   868 Feb 14 16:21 misc.a
-rw-r--r--  1 x  wheel   708 Feb 14 16:22 misc2.a
-rw-r--r--  1 x  wheel  1076 Feb 14 16:22 misc3.a
$ readelf -e misc2.a                                                                                                                [ /tmp ]

File: misc2.a(misc.o)
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 09 00 00 00 00 00 00 00 00
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            FreeBSD
  ABI Version:                       0
  Type:                              REL (Relocatable file)
  Machine:                           Advanced Micro Devices x86-64
  Version:                           0x1
  Entry point address:               0
  Start of program headers:          0 (bytes into file)
  Start of section headers:          192 (bytes into file)
  Flags:                             0
  Size of this header:               64 (bytes)
  Size of program headers:           0 (bytes)
  Number of program headers:         0
  Size of section headers:           64 (bytes)
  Number of section headers:         6
  Section header string table index: 5

There are no program headers in this file.
There are 6 section headers, starting at offset 0xc0:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .text             PROGBITS         0000000000000000  00000040
       0000000000000011  0000000000000000  AX       0     0     16
  [ 2] .note.GNU-stack   PROGBITS         0000000000000000  00000051
       0000000000000000  0000000000000000           0     0     1
  [ 3] .eh_frame         X86_64_UNWIND    0000000000000000  00000058
       0000000000000038  0000000000000000   A       0     0     8
  [ 4] .rela.eh_frame    RELA             0000000000000000  00000090
       0000000000000000  0000000000000018   I       0     3     8
  [ 5] .shstrtab         STRTAB           0000000000000000  00000090
       0000000000000030  0000000000000000           0     0     1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings)
  I (info), L (link order), G (group), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)
$
Comment 25 Ed Maste freebsd_committer 2018-02-14 14:32:23 UTC
(In reply to Sergey from comment #24)
> Actually the issue preserves on 11.1, 'strip' removes not requested sections:

That issue is unrelated. The issue reported here (PR191587) is file corruption from GNU strip; the issue cloned to PR225890 stems from an IMO strange choice in option handling behaviour made by GNU strip and inherited by ELF Tool Chain strip.