In building various ports (such as devel/libpcre) I have noticed that 'strip' corrupts the static library and the port build will fail. To reproduce do the following with an unstripped static library: root@erl:/tmp # ls -al libpcre.a -rw-r--r-- 1 sson wheel 404446 Jun 11 18:52 libpcre.a root@erl:/tmp # strip --strip-debug libpcre.a root@erl:/tmp # ls -al libpcre.a -rw-r--r-- 1 sson wheel 403506 Jun 11 18:53 libpcre.a root@erl:/tmp # ranlib libpcre.a ranlib: warning: can't find .strtab section ranlib: fatal: Invalid filename root@erl:/tmp # ls -al libpcre.a -rw-r--r-- 1 sson wheel 0 Jun 11 18:53 libpcre.a
I should have noted that the above was on a MIPS64 Ether Router Lite. 'strip' also seems to corrupt static libraries on the MIPS arch as well but not to the point where 'ranlib' will fail with a "fatal" error.
I'll see if this is fixed by using elftoolchain's strip
adding toolchain people. I don't think anyone really wants to dig into the gcc tool chain at this point. But, as always, I never assume I know what I'm talking about.
elftoolchain's strip does not handle SHT_MIPS_OPTIONS https://github.com/emaste/elftoolchain/issues/1
Stacey, can you please attach a compressed version of the unstripped libpcre.a, if you still have it? That would make it a little easier for me to debug.
Created attachment 144973 [details] unstripped archive from ports-mgmt/pkg
dim: I've attached a .a from building ports-mgmt/pkg that also demonstrates the problem.
Updated and tried a xdev tool chain and a fully native tool chain in emulation via QEMU. pkg still seems to be failing: ===> Staging for pkg-1.3.0 ===> Generating temporary packing list if test -z 'strip'; then /usr/bin/make INSTALL_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" install_sh_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" INSTALL_STRIP_FLAG=-s install; else /usr/bin/make INSTALL_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" install_sh_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" INSTALL_STRIP_FLAG=-s "INSTALL_PROGRAM_ENV=STRIPPROG='strip'" install; fi Making install in external /usr/bin/make install-am Making install in libpkg Making install in repo Making install in binary Making install in . /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib' /bin/sh ../libtool --mode=install /usr/bin/install -c -o root -g wheel -s libpkg.la libpkg_static.la '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib' libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg.so.3.0.0 /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.so.3.0.0 libtool: install: strip --strip-unneeded /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.so.3.0.0 libtool: install: (cd /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib && { ln -s -f libpkg.so.3.0.0 libpkg.so.3 || { rm -f libpkg.so.3 && ln -s libpkg.so.3.0.0 libpkg.so.3; }; }) libtool: install: (cd /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib && { ln -s -f libpkg.so.3.0.0 libpkg.so || { rm -f libpkg.so && ln -s libpkg.so.3.0.0 libpkg.so; }; }) libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg.lai /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.la libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg_static.lai /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.la libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg_static.a /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a libtool: install: strip --strip-debug /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a libtool: install: chmod 644 /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a libtool: install: ranlib /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a ranlib: fatal: Invalid filename *** Error code 70 Stop. make[6]: stopped in /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/libpkg *** Error code 1 Stop. make[5]: stopped in /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/libpkg *** Error code 1 Stop. make[4]: stopped in /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/libpkg *** Error code 1 Stop. make[3]: stopped in /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0 *** Error code 1 Stop. make[2]: stopped in /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0 *** Error code 1 Stop. make[1]: stopped in /usr/ports/ports-mgmt/pkg *** Error code 1 Stop.
adding bapt for tracking.
We should never strip static libraries should we? I breaks way more than only mips
This does *not* happen on a mips32 build: ===> Staging for pkg-1.3.0 ===> Generating temporary packing list if test -z 'strip'; then /usr/bin/make INSTALL_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" install_sh_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" INSTALL_STRIP_FLAG=-s install; else /usr/bin/make INSTALL_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" install_sh_PROGRAM="/bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s" INSTALL_STRIP_FLAG=-s "INSTALL_PROGRAM_ENV=STRIPPROG='strip'" install; fi Making install in external /usr/bin/make install-am Making install in libpkg Making install in repo Making install in binary Making install in . /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib' /bin/sh ../libtool --mode=install /usr/bin/install -c -o root -g wheel -s libpkg.la libpkg_static.la '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib' libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg.so.3.0.0 /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.so.3.0.0 libtool: install: strip --strip-unneeded /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.so.3.0.0 libtool: install: (cd /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib && { ln -s -f libpkg.so.3.0.0 libpkg.so.3 || { rm -f libpkg.so.3 && ln -s libpkg.so.3.0.0 libpkg.so.3; }; }) libtool: install: (cd /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib && { ln -s -f libpkg.so.3.0.0 libpkg.so || { rm -f libpkg.so && ln -s libpkg.so.3.0.0 libpkg.so; }; }) libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg.lai /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg.la libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg_static.lai /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.la libtool: install: /usr/bin/install -c -o root -g wheel .libs/libpkg_static.a /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a libtool: install: strip --strip-debug /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a libtool: install: chmod 644 /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a libtool: install: ranlib /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a libtool: install: warning: remember to run `libtool --finish /usr/local/lib' /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/include' install -o root -g wheel -m 444 pkg.h '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/include' Making install in src /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc' install -o root -g wheel -m 444 pkg.conf.sample '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc' /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin' STRIPPROG='strip' /bin/sh ../libtool --mode=install /bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s pkg pkg-static '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin' libtool: install: warning: `../libpkg/libpkg.la' has not been installed in `/usr/local/lib' libtool: install: /bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s .libs/pkg /usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin/pkg libtool: install: /bin/sh /usr/ports/ports-mgmt/pkg/work/pkg-1.3.0/install-sh -c -s pkg-static /usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin/pkg-static Making install in tests Making install in scripts /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin' install -o root -g wheel -m 555 sbin/pkg2ng '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/sbin' /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/bash_completion.d' install -o root -g wheel -m 444 completion/_pkg.bash '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/bash_completion.d' /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/daily' install -o root -g wheel -m 444 periodic/411.pkg-backup periodic/490.status-pkg-changes '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/daily' /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/security' install -o root -g wheel -m 444 periodic/410.pkg-audit periodic/460.pkg-checksum '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/security' /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/weekly' install -o root -g wheel -m 444 periodic/400.status-pkg '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/etc/periodic/weekly' /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/share/zsh/site-functions' install -o root -g wheel -m 444 completion/_pkg.zsh '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/share/zsh/site-functions' Making install in docs /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man3' install -o root -g wheel -m 444 pkg_printf.3 pkg_repos.3 '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man3' /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man5' install -o root -g wheel -m 444 pkg-repository.5 pkg.conf.5 '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man5' /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man8' install -o root -g wheel -m 444 pkg-add.8 pkg-annotate.8 pkg-audit.8 pkg-autoremove.8 pkg-backup.8 pkg-check.8 pkg-clean.8 pkg-config.8 pkg-convert.8 pkg-create.8 pkg-delete.8 pkg-fetch.8 pkg-info.8 pkg-install.8 pkg-lock.8 pkg-query.8 pkg-register.8 pkg-repo.8 pkg-rquery.8 pkg-search.8 pkg-set.8 pkg-shell.8 pkg-shlib.8 pkg-ssh.8 pkg-stats.8 pkg-update.8 pkg-updating.8 pkg-upgrade.8 pkg-version.8 pkg-which.8 pkg.8 '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man8' /usr/bin/make install-data-hook ln -s -f pkg-delete.8 /usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man8/pkg-remove.8 ln -s -f pkg-lock.8 /usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man8/pkg-unlock.8 ln -s -f pkg.8 /usr/ports/ports-mgmt/pkg/work/stage/usr/local/man/man8/pkg-static.8 /bin/mkdir -p '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/libdata/pkgconfig' install -o root -g wheel -m 444 libpkg/pkg.pc '/usr/ports/ports-mgmt/pkg/work/stage/usr/local/libdata/pkgconfig' ====> Compressing man pages (compress-man)
> We should never strip static libraries should we? > I breaks way more than only mips In the pkg case I encountered it's "strip --strip-debug", which should only be removing debug data, and should be fine for static libraries.
As a workaround the offending "strip --strip-debug" can be disabled in the libtool files by commenting out old_striplib. -old_striplib="strip --strip-debug" +#old_striplib="strip --strip-debug"
comment out here? contrib/binutils/binutils/doc/binutils.texi:@item Run @code{strip --strip-debug foo}
root@erl:/usr/ports/x11/xdpyinfo # diff -u /usr/local/bin/libtool.orig /usr/local/bin/libtool --- /usr/local/bin/libtool.orig 2014-08-12 13:14:52.000000000 +0000 +++ /usr/local/bin/libtool 2014-08-08 14:45:45.000000000 +0000 @@ -290,7 +290,7 @@ dlopen_self_static=no # Commands to strip libraries. -old_striplib="strip --strip-debug" +#old_striplib="strip --strip-debug" striplib="strip --strip-unneeded" root@erl:/usr/ports/x11/xdpyinfo #
If you want to build pkg (for now)... make -DWITH_DEBUG install clean Hardware: Ubnt EdgeRouter Lite FreeBSD 11.0-CURRENT #0 r272938: Sat Oct 11 23:39:28 UTC 2014
No need for ports collection to reproduce this bug, simply: $ echo 'int fun(int x){return x+42;}' > misc.c && cc -c misc.c && ar -rc misc.a misc.o && cp misc.a misc2.a $ strip misc.a $ ranlib misc.a ranlib: warning: can't find .strtab section ranlib: fatal: Invalid filename $ ranlib misc2.a $
It looks like this is currently fixed in the in-tree libelf, and in upstream elftoolchain libelf. I have work in progress to switch to strip (and other tools) from the elftoolchain project, see https://reviews.freebsd.org/D1224. After installing strip from there I see: root@erl:~ # echo 'int fun(int x){return x+42;}' > misc.c && cc -c misc.c && ar -rc misc.a misc.o && cp misc.a misc2.a root@erl:~ # strip misc.a root@erl:~ # ranlib misc.a ranlib: warning: can't find .strtab section root@erl:~ # ranlib misc2.a root@erl:~ #
And for completeness, # echo 'int fun(int x){return x+42;}' > misc.c && cc -c misc.c && ar -rc misc.a misc.o # strip --strip-debug misc.a # ranlib misc.a # # nm misc.a misc.o: 0000000000000000 T fun
WITH_ELFTOOLCHAIN_TOOLS is now the default in HEAD so this issue will be solved there. Merging it back to stable/10 is not really feasible at the moment, so we'll have to consider what we do for the stable branch.
If we can't MFC then do we apply the patch to 10-STABLE?
There's no patch to apply - the issue is fixed in HEAD by switching from GNU binutils strip to ELF Tool Chain's strip. It would be possible to bring that back to stable/10 as an option, but there's a reasonably long dependency chain to address, and we'd likely need to find a way to install two different copies of libelf.
This is fixed in -CURRENT with the migration to the ELF Tool Chain version of strip. It is not feasible to merge that change back to stable/10. MIPS users who cannot move off stable/10 will have to use a workaround like one of those discussed in other comments in this PR.
Actually the issue preserves on 11.1, 'strip' removes not requested sections: $ echo 'int fun(int x){return x+42;}' > misc.c && cc -c misc.c && ar -rc misc.a misc.o && cp misc.a misc2.a < $ ll misc*.a [ /tmp ] -rw-r--r-- 1 x wheel 1076 Feb 14 16:20 misc.a -rw-r--r-- 1 x wheel 1076 Feb 14 16:20 misc2.a $ strip misc.a [ /tmp ] $ ranlib misc.a [ /tmp ] ranlib: warning: can't find .strtab section $ readelf --string-dump .strtab misc.a [ /tmp ] File: misc.a(misc.o) $ ll misc*.a [ /tmp ] -rw-r--r-- 1 x wheel 868 Feb 14 16:21 misc.a -rw-r--r-- 1 x wheel 1076 Feb 14 16:20 misc2.a $ cp misc2.a misc3.a [ /tmp ] $ strip -R .comment misc2.a [ /tmp ] $ ll misc*.a [ /tmp ] -rw-r--r-- 1 x wheel 868 Feb 14 16:21 misc.a -rw-r--r-- 1 x wheel 708 Feb 14 16:22 misc2.a -rw-r--r-- 1 x wheel 1076 Feb 14 16:22 misc3.a $ readelf -e misc2.a [ /tmp ] File: misc2.a(misc.o) ELF Header: Magic: 7f 45 4c 46 02 01 01 09 00 00 00 00 00 00 00 00 Class: ELF64 Data: 2's complement, little endian Version: 1 (current) OS/ABI: FreeBSD ABI Version: 0 Type: REL (Relocatable file) Machine: Advanced Micro Devices x86-64 Version: 0x1 Entry point address: 0 Start of program headers: 0 (bytes into file) Start of section headers: 192 (bytes into file) Flags: 0 Size of this header: 64 (bytes) Size of program headers: 0 (bytes) Number of program headers: 0 Size of section headers: 64 (bytes) Number of section headers: 6 Section header string table index: 5 There are no program headers in this file. There are 6 section headers, starting at offset 0xc0: Section Headers: [Nr] Name Type Address Offset Size EntSize Flags Link Info Align [ 0] NULL 0000000000000000 00000000 0000000000000000 0000000000000000 0 0 0 [ 1] .text PROGBITS 0000000000000000 00000040 0000000000000011 0000000000000000 AX 0 0 16 [ 2] .note.GNU-stack PROGBITS 0000000000000000 00000051 0000000000000000 0000000000000000 0 0 1 [ 3] .eh_frame X86_64_UNWIND 0000000000000000 00000058 0000000000000038 0000000000000000 A 0 0 8 [ 4] .rela.eh_frame RELA 0000000000000000 00000090 0000000000000000 0000000000000018 I 0 3 8 [ 5] .shstrtab STRTAB 0000000000000000 00000090 0000000000000030 0000000000000000 0 0 1 Key to Flags: W (write), A (alloc), X (execute), M (merge), S (strings) I (info), L (link order), G (group), x (unknown) O (extra OS processing required) o (OS specific), p (processor specific) $
(In reply to Sergey from comment #24) > Actually the issue preserves on 11.1, 'strip' removes not requested sections: That issue is unrelated. The issue reported here (PR191587) is file corruption from GNU strip; the issue cloned to PR225890 stems from an IMO strange choice in option handling behaviour made by GNU strip and inherited by ELF Tool Chain strip.