Bug 207091 - [exp-run] installing ELF Tool Chain elfcopy in place of binutils objcopy
Summary: [exp-run] installing ELF Tool Chain elfcopy in place of binutils objcopy
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Ed Maste
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-02-10 20:53 UTC by Ed Maste
Modified: 2016-02-26 15:48 UTC (History)
1 user (show)

See Also:


Attachments
update elftoolchain to r3395 (212.51 KB, patch)
2016-02-10 20:53 UTC, Ed Maste
no flags Details | Diff
switch to installing elfcopy as objcopy (1.12 KB, patch)
2016-02-10 20:53 UTC, Ed Maste
no flags Details | Diff
copy of libpe_utils to avoid diff issues (2.00 KB, text/x-c)
2016-02-10 22:20 UTC, Ed Maste
no flags Details
update elftoolchain to r3395 (git rename detection disabled) (213.34 KB, patch)
2016-02-10 22:28 UTC, Ed Maste
no flags Details | Diff
Allow elfcopy to convert between two non-ELF formats (1.33 KB, patch)
2016-02-14 15:37 UTC, Ed Maste
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Maste freebsd_committer 2016-02-10 20:53:18 UTC
Created attachment 166851 [details]
update elftoolchain to r3395

ELF Tool Chain has recently gained PE/COFF support, which means we can use it in the base system in place of binutils objcopy. I would like to request a ports exp-run to evaluate ports impact.

I expect that simple or minor failures will be addressed with bug fixes or small functionality additions to elfcopy, while more major issues (e.g. any significant missing functionality) should be handled by having the port require the binutils port.

There are (or will be) two patches attached to this PR: elftc-r3395.diff is an update to a later elftc version, but does not switch to elfcopy.

elfcopy-as-objcopy.diff flips the switch -- it makes WITH_ELFCOPY_AS_OBJCOPY a default YES option on all architectures.
Comment 1 Ed Maste freebsd_committer 2016-02-10 20:53:42 UTC
Created attachment 166852 [details]
switch to installing elfcopy as objcopy
Comment 2 Antoine Brodin freebsd_committer 2016-02-10 22:07:51 UTC
buildworld fails:

===> lib/libpe (obj,depend,all,install)
--- obj ---
/usr/obj/poudriere/jails/headamd64PR207091/usr/src/tmp/poudriere/jails/headamd64PR207091/usr/src/lib/libpe created for /poudriere/jails/headamd64PR207091/usr/src/lib/libpe
make[3]: make[3]: don't know how to make libpe_utils.c. Stop
Comment 3 Ed Maste freebsd_committer 2016-02-10 22:20:33 UTC
Created attachment 166856 [details]
copy of libpe_utils to avoid diff issues
Comment 4 Ed Maste freebsd_committer 2016-02-10 22:22:35 UTC
> buildworld fails:

I successfully applied both patches to a clean svn checkout to do a tinderbox build; it looks like the failure you encountered was due to overzealous git rename detection. libpe_utils.c should have been created from dwarf_str.c like so:

diff --git a/contrib/elftoolchain/libdwarf/dwarf_str.c b/contrib/elftoolchain/libpe/libpe_utils.c
similarity index 60%                                                            
copy from contrib/elftoolchain/libdwarf/dwarf_str.c                             
copy to contrib/elftoolchain/libpe/libpe_utils.c                                
index 71a7f75..9bc9a54 100644                                                   
--- a/contrib/elftoolchain/libdwarf/dwarf_str.c                                 
+++ b/contrib/elftoolchain/libpe/libpe_utils.c                                  
...

which did work in my svn checkout, but I suspect failed for you for some reason.

I've attached a plain copy of libpe_utils.c also.
Comment 5 Antoine Brodin freebsd_committer 2016-02-10 22:28:42 UTC
Please provide a patch that can be applied with svn patch,  jail creation is automated from a patch file
Comment 6 Ed Maste freebsd_committer 2016-02-10 22:28:46 UTC
Created attachment 166857 [details]
update elftoolchain to r3395 (git rename detection disabled)

update r3395 patch to avoid git rename detection
Comment 7 Ed Maste freebsd_committer 2016-02-10 22:30:52 UTC
> Please provide a patch that can be applied with svn patch,  jail creation is automated from a patch file

Updated patch without git rename detection. It's interesting to me that plain patch(1) handles it fine but svn patch does not.
Comment 8 Ed Maste freebsd_committer 2016-02-10 22:32:45 UTC
Comment on attachment 166856 [details]
copy of libpe_utils to avoid diff issues

obsolete the separate copy of libpe_utils.c - the patch has been regenerated without git rename detection
Comment 9 Antoine Brodin freebsd_committer 2016-02-12 13:02:32 UTC
Exp-run results on i386:

http://package23.nyi.freebsd.org/build.html?mastername=headi386PR207091-default&build=2016-02-10_23h02m15s

Exp-run results on amd64:

http://package22.nyi.freebsd.org/build.html?mastername=headamd64PR207091-default&build=2016-02-10_23h02m06s

New failures on i386:

+ {"origin"=>"comms/usrp", "pkgname"=>"usrp-3.4.3_4", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"math/why3", "pkgname"=>"why3-0.83_2", "phase"=>"stage", "errortype"=>"???"}
+ {"origin"=>"security/mbedtls", "pkgname"=>"mbedtls-2.2.1", "phase"=>"stage", "errortype"=>"???"}
+ {"origin"=>"security/polarssl13", "pkgname"=>"polarssl13-1.3.16", "phase"=>"stage", "errortype"=>"???"}

Failure logs on i386:

http://package23.nyi.freebsd.org/data/headi386PR207091-default/2016-02-10_23h02m15s/logs/errors/usrp-3.4.3_4.log
http://package23.nyi.freebsd.org/data/headi386PR207091-default/2016-02-10_23h02m15s/logs/errors/why3-0.83_2.log
http://package23.nyi.freebsd.org/data/headi386PR207091-default/2016-02-10_23h02m15s/logs/errors/mbedtls-2.2.1.log
http://package23.nyi.freebsd.org/data/headi386PR207091-default/2016-02-10_23h02m15s/logs/errors/polarssl13-1.3.16.log

New failures on amd64:

+ {"origin"=>"comms/usrp", "pkgname"=>"usrp-3.4.3_4", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"security/openvpn-polarssl", "pkgname"=>"openvpn-polarssl-2.3.10", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"sysutils/xen-tools", "pkgname"=>"xen-tools-4.5.2_1", "phase"=>"build", "errortype"=>"???"}

Failure logs on amd64:

http://package22.nyi.freebsd.org/data/headamd64PR207091-default/2016-02-10_23h02m06s/logs/errors/usrp-3.4.3_4.log
http://package22.nyi.freebsd.org/data/headamd64PR207091-default/2016-02-10_23h02m06s/logs/errors/openvpn-polarssl-2.3.10.log
http://package22.nyi.freebsd.org/data/headamd64PR207091-default/2016-02-10_23h02m06s/logs/errors/xen-tools-4.5.2_1.log
Comment 10 Ed Maste freebsd_committer 2016-02-12 14:30:58 UTC
Thanks Antoine.

These now have ELF Tool Chain tickets:

Strip failures due to update to ELF Tool Chain r3395:
https://sourceforge.net/p/elftoolchain/tickets/514/

Objcopy failures:
https://sourceforge.net/p/elftoolchain/tickets/515/
Comment 11 Ed Maste freebsd_committer 2016-02-12 18:57:02 UTC
The strip issue is now fixed upstream. I will go ahead with the ELF Tool Chain update including that fix (obsoleting the "update elftoolchain" patch), and then investigate the elfcopy/objcopy failures before sending a broader call for testing or requesting another exp-run.
Comment 12 commit-hook freebsd_committer 2016-02-12 20:54:17 UTC
A commit references this bug:

Author: emaste
Date: Fri Feb 12 20:54:04 UTC 2016
New revision: 295577
URL: https://svnweb.freebsd.org/changeset/base/295577

Log:
  Update ELF Tool Chain to upstream rev 3400

  Some notable improvements include:

  readelf:
  - Add AArch64 relocation definitions.
  - Report value of unknown relocation types.

  elfcopy:
  - Consider symbols with STB_GNU_UNIQUE binding as global symbols.
  - Fixed support for VMA adjustment for loadable sections found
    in relocatable objects.
  - Handle nameless global symbols.
  - Improve wildcard matching for !-prefixed symbols.
  - Add PE/COFF support.

  elfdump:
  - Improve section type reporting.
  - Add MIPS-specific section types.

  This update also includes a significant number of bug fixes.

  PR:		207091 [exp-run]
  Sponsored by:	The FreeBSD Foundation

Changes:
_U  head/contrib/elftoolchain/
  head/contrib/elftoolchain/addr2line/addr2line.c
  head/contrib/elftoolchain/addr2line/os.NetBSD.mk
_U  head/contrib/elftoolchain/ar/
  head/contrib/elftoolchain/ar/ar.c
_U  head/contrib/elftoolchain/brandelf/
  head/contrib/elftoolchain/brandelf/brandelf.c
  head/contrib/elftoolchain/common/elfdefinitions.h
  head/contrib/elftoolchain/common/native-elf-format
  head/contrib/elftoolchain/cxxfilt/cxxfilt.c
  head/contrib/elftoolchain/elfcopy/Makefile
  head/contrib/elftoolchain/elfcopy/archive.c
  head/contrib/elftoolchain/elfcopy/elfcopy.1
  head/contrib/elftoolchain/elfcopy/elfcopy.h
  head/contrib/elftoolchain/elfcopy/main.c
  head/contrib/elftoolchain/elfcopy/pe.c
  head/contrib/elftoolchain/elfcopy/sections.c
  head/contrib/elftoolchain/elfcopy/segments.c
  head/contrib/elftoolchain/elfcopy/symbols.c
_U  head/contrib/elftoolchain/elfdump/
  head/contrib/elftoolchain/elfdump/elfdump.c
  head/contrib/elftoolchain/elfdump/os.NetBSD.mk
  head/contrib/elftoolchain/libdwarf/_libdwarf.h
  head/contrib/elftoolchain/libdwarf/dwarf.3
  head/contrib/elftoolchain/libdwarf/dwarf_str.c
  head/contrib/elftoolchain/libdwarf/libdwarf.h
  head/contrib/elftoolchain/libdwarf/libdwarf_rw.c
  head/contrib/elftoolchain/libdwarf/os.NetBSD.mk
  head/contrib/elftoolchain/libelf/_libelf_config.h
  head/contrib/elftoolchain/libelftc/Makefile
  head/contrib/elftoolchain/libelftc/elftc_bfd_find_target.3
  head/contrib/elftoolchain/libelftc/elftc_copyfile.c
  head/contrib/elftoolchain/libelftc/elftc_demangle.c
  head/contrib/elftoolchain/libelftc/elftc_symbol_table_create.3
  head/contrib/elftoolchain/libelftc/libelftc.h
  head/contrib/elftoolchain/libelftc/libelftc_bfdtarget.c
  head/contrib/elftoolchain/libelftc/libelftc_dem_gnu3.c
  head/contrib/elftoolchain/libelftc/make-toolchain-version
  head/contrib/elftoolchain/libelftc/os.NetBSD.mk
  head/contrib/elftoolchain/libpe/
  head/contrib/elftoolchain/nm/os.NetBSD.mk
  head/contrib/elftoolchain/readelf/os.NetBSD.mk
  head/contrib/elftoolchain/readelf/readelf.c
  head/contrib/elftoolchain/size/os.NetBSD.mk
  head/contrib/elftoolchain/strings/os.NetBSD.mk
  head/contrib/elftoolchain/strings/strings.1
  head/contrib/elftoolchain/strings/strings.c
  head/lib/libelftc/elftc_version.c
Comment 13 Ed Maste freebsd_committer 2016-02-12 20:56:11 UTC
Comment on attachment 166857 [details]
update elftoolchain to r3395 (git rename detection disabled)

Obsolete the "update elftoolchain" patch as a later version has now been committed to HEAD.
Comment 14 Ed Maste freebsd_committer 2016-02-13 16:45:29 UTC
Remaining issues:

comms/usrp i386 & amd64
Tracked in https://sourceforge.net/p/elftoolchain/tickets/515/

sysutils/xen-tools amd64
Tracked in https://sourceforge.net/p/elftoolchain/tickets/516/

These are elfcopy/objcopy issues only; the strip regression is fixed in HEAD, which fixes these failures reported in the exp-run:

i386 math/why3, security/mbedtls, security/polarssl13
amd64 security/openvpn-polarssl
Comment 15 Ed Maste freebsd_committer 2016-02-13 22:59:45 UTC
The sysutils/xen-tools failure is an issue with that port, tracked in PR 207170.
Comment 16 Ed Maste freebsd_committer 2016-02-14 15:37:38 UTC
Created attachment 166986 [details]
Allow elfcopy to convert between two non-ELF formats
Comment 17 Ed Maste freebsd_committer 2016-02-14 15:41:30 UTC
I'd like to request another exp run.

Added a patch (elfcopy-non-elf.diff) to fix the elfcopy issue demonstrated by comms/usrp. I believe that all ports except sysutils/xen-tools should now build on HEAD with this additional patch.

sysutils/xen-tools will need to be fixed, or require GNU binutils (which is less strict about malformed ELF files).
Comment 18 Ed Maste freebsd_committer 2016-02-16 14:06:40 UTC
elfcopy-non-elf.diff now committed as r295661

royger@ submitted a patch for sysutils/xen-tools upstream: http://www.seabios.org/pipermail/seabios/2016-February/010510.html
Comment 19 Antoine Brodin freebsd_committer 2016-02-17 17:39:10 UTC
I confirm that the only new failure is xen-tools
Comment 20 Ed Maste freebsd_committer 2016-02-17 17:40:54 UTC
Thanks Antoine.
Comment 21 commit-hook freebsd_committer 2016-02-26 15:47:16 UTC
A commit references this bug:

Author: emaste
Date: Fri Feb 26 15:46:15 UTC 2016
New revision: 296096
URL: https://svnweb.freebsd.org/changeset/base/296096

Log:
  Install elftoolchain elfcopy(1) as objcopy(1) by default

  As of r295661 elfcopy supports PE format for EFI boot binaries and is a
  viable objcopy implementation for the base system and ports.

  The (temporary) src.conf knob WITHOUT_ELFCOPY_AS_OBJCOPY knob may be set
  to obtain the GNU version if necessary.

  PR:		207091 [exp-run]
  Relnotes:	Yes
  Sponsored by:	The FreeBSD Foundation

Changes:
  head/UPDATING
  head/share/mk/src.opts.mk