Bug 233056 - [exp-run] Enable the new crtbegin/crtend in head
Summary: [exp-run] Enable the new crtbegin/crtend in head
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Port Management Team
URL:
Keywords:
Depends on: 234451
Blocks: 233094
  Show dependency treegraph
 
Reported: 2018-11-07 11:56 UTC by Andrew Turner
Modified: 2019-09-16 13:42 UTC (History)
4 users (show)

See Also:
Andrew: exp-run?


Attachments
Enable the BSD crtbegin/end by default (764 bytes, patch)
2018-11-07 11:56 UTC, Andrew Turner
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Turner freebsd_committer 2018-11-07 11:56:44 UTC
Created attachment 199048 [details]
Enable the BSD crtbegin/end by default

I would like to enable the new crtbegin.o and crtend.o by default in -current. Can I get an exp-run with the attached base patch to make sure I'm not breaking building of any ports? It should apply to base r340213 or later.
Comment 1 Antoine Brodin freebsd_committer 2018-11-11 19:32:16 UTC
New failures on amd64:

+ {"origin"=>"audio/dpf-plugins-lv2", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"audio/fluidplug-lv2", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"audio/vamp-plugin-sdk", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"audio/zam-plugins-lv2", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"audio/zynaddsubfx", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"devel/libsoup", "phase"=>"configure", "errortype"=>"configure_error"}
+ {"origin"=>"devel/pear-PHP_UML", "flavor"=>"php56", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"devel/pear-PHP_UML", "flavor"=>"php70", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"devel/pear-PHP_UML", "flavor"=>"php71", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"devel/pear-PHP_UML", "flavor"=>"php73", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"devel/pear-PHP_UML", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"devel/pear-TheSeer_fXSL", "flavor"=>"php56", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"devel/pear-TheSeer_fXSL", "flavor"=>"php70", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"devel/pear-TheSeer_fXSL", "flavor"=>"php71", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"devel/pear-TheSeer_fXSL", "flavor"=>"php73", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"devel/pear-TheSeer_fXSL", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"emulators/higan", "phase"=>"extract", "errortype"=>"coredump"}
+ {"origin"=>"games/ezquake", "phase"=>"extract", "errortype"=>"process_failed"}
+ {"origin"=>"games/spring", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"graphics/librsvg2", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"graphics/tesseract-data", "phase"=>"fetch", "errortype"=>"fetch"}
+ {"origin"=>"korean/nanumfonts-otf", "phase"=>"extract", "errortype"=>"coredump"}
+ {"origin"=>"korean/nanumfonts-ttf", "phase"=>"extract", "errortype"=>"coredump"}
+ {"origin"=>"lang/afnix", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"misc/qt5-doc", "phase"=>"extract", "errortype"=>"coredump"}
+ {"origin"=>"multimedia/vlc", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"multimedia/vlc-qt4", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"multimedia/vlc3", "phase"=>"build", "errortype"=>"coredump"}
+ {"origin"=>"net-mgmt/icingaweb2", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"security/libhijack", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"textproc/py-enchant", "flavor"=>"py36", "phase"=>"configure", "errortype"=>"process_failed"}
+ {"origin"=>"www/pear-XML_GRDDL", "flavor"=>"php56", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"www/pear-XML_GRDDL", "flavor"=>"php70", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"www/pear-XML_GRDDL", "flavor"=>"php71", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"www/pear-XML_GRDDL", "flavor"=>"php73", "phase"=>"stage", "errortype"=>"process_failed"}
+ {"origin"=>"www/pear-XML_GRDDL", "phase"=>"stage", "errortype"=>"process_failed"}

Around 2000 ports skipped due to those new failures

Failure logs on amd64:

http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/dpf-plugins-lv2-1.1.1_3.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/FluidPlug-lv2-g20161020_2.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/vamp-plugin-sdk-2.6.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/zam-plugins-lv2-3.10.13_2.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/zynaddsubfx-3.0.3.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/libsoup-2.62.2.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php56-pear-PHP_UML-1.6.2.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php70-pear-PHP_UML-1.6.2.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php71-pear-PHP_UML-1.6.2.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php73-pear-PHP_UML-1.6.2.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php72-pear-PHP_UML-1.6.2.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php56-pear-TheSeer_fXSL-1.1.0.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php70-pear-TheSeer_fXSL-1.1.0.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php71-pear-TheSeer_fXSL-1.1.0.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php73-pear-TheSeer_fXSL-1.1.0.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php72-pear-TheSeer_fXSL-1.1.0.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/higan-106_2.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/ezquake-1.9_9,2.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/spring-98.0_14.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/librsvg2-2.40.20.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/tesseract-data-4.0.0.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/ko-nanumfonts-otf-20160201.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/ko-nanumfonts-ttf-20160201.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/afnix-2.6.3_3.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/qt5-doc-5.11.2.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/vlc-2.2.8_14,4.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/vlc-qt4-2.2.8_14,4.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/vlc3-3.0.4_1,4.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/icingaweb2-2.6.1.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/libhijack-0.8.0.1.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/py36-enchant-2.0.0.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php56-pear-XML_GRDDL-0.2.0.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php70-pear-XML_GRDDL-0.2.0.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php71-pear-XML_GRDDL-0.2.0.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php73-pear-XML_GRDDL-0.2.0.log
http://package18.nyi.freebsd.org/data/headamd64PR233056-default/2018-11-09_21h50m43s/logs/errors/php72-pear-XML_GRDDL-0.2.0.log
Comment 2 Ed Maste freebsd_committer 2018-11-12 14:32:44 UTC
The libhijack failure is presumably due to r340107 not the crtbegin patch.

/wrkdirs/usr/ports/security/libhijack/work/libhijack-0.8.0-1-g50804aa/libhijack/rtld.c:264:6: error: no member named 'textsize' in 'struct Struct_Obj_Entry'
        soe.textsize = round_page(aux->loadables->phdr.phdr->p_vaddr +
        ~~~ ^

Some of the failures I looked at (at least spring, ezquake) were segfaults extracting a .7z file so the p7zip port might be a good place to start investigating. That said I did try building p7zip on my desktop with BSD_CRTBEGIN enabled and it did not fail in a trivial test.
Comment 3 Andrew Turner freebsd_committer 2018-11-12 22:44:13 UTC
It looks like there's an issue with __cxa_finalize. The devel/pear-TheSeer_fXSL port fails with the following from php:

(gdb) bt
#0  0x000000080182a9f0 in ?? ()
#1  0x0000000800ac0b05 in __cxa_finalize (dso=0x0) at /tank/users/andrew/freebsd/lib/libc/stdlib/atexit.c:244
#2  0x0000000800a506e1 in __cxa_thread_atexit_hidden (dtor_func=0x1, obj=0x1, dso_symbol=0x0)
    at /tank/users/andrew/freebsd/lib/libc/stdlib/cxa_thread_atexit_impl.c:90
#3  __cxa_thread_atexit_impl (dtor_func=0x1, obj=0x1, dso_symbol=0x0) at /tank/users/andrew/freebsd/lib/libc/stdlib/cxa_thread_atexit_impl.c:81
#4  0x00000000005a24bb in main ()
Comment 4 commit-hook freebsd_committer 2018-11-13 15:28:37 UTC
A commit references this bug:

Author: andrew
Date: Tue Nov 13 15:28:27 UTC 2018
New revision: 340395
URL: https://svnweb.freebsd.org/changeset/base/340395

Log:
  Run __cxa_finalize in shared objects in the destructor path.

  When we have .dtors call them before .dtor handling, otherwise call from
  a destructor.

  PR:		233056
  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

Changes:
  head/lib/csu/common/crtbegin.c
Comment 5 Antoine Brodin freebsd_committer 2018-11-20 06:31:20 UTC
New failure on i386:

+ {"origin"=>"lang/erlang-runtime21", "phase"=>"build", "errortype"=>"coredump"}

Failure log:

http://package18.nyi.freebsd.org/data/headi386PR233056-default/2018-11-19_22h45m32s/logs/errors/erlang-runtime21-21.1.1.log
Comment 6 Ed Maste freebsd_committer 2018-11-20 14:25:38 UTC
(In reply to Antoine Brodin from comment #5)
Is the i386 failure from before or after r340395?
Comment 7 Antoine Brodin freebsd_committer 2018-11-20 14:27:19 UTC
(In reply to Ed Maste from comment #6)

It's from r340408
Comment 8 Ed Maste freebsd_committer 2018-11-20 14:35:45 UTC
(In reply to Antoine Brodin from comment #7)
Thanks Antoine.

The i386 failure seems like it might be a transient issue:
| set -e; LLVM_PROFILE_FILE="obj/i386-portbld-freebsd13.0/opt/smp/erlc-%m.profraw" \
|           ERL_FLAGS="-emu_type prof +S 1" erlc -W  -DPGO \
|   -o obj/i386-portbld-freebsd13.0/opt/smp test/estone_SUITE.erl > obj/i386-portbld-freebsd13.0/opt/smp/PROFILE_LOG
| dlsym: Resource temporarily unavailable

So presumably the issue is solved by andrew's __cxa_finalize change, with one port needing further investigation/confirmation. Would you like to run another amd64 exp-run before we enable this in HEAD?
Comment 9 Antoine Brodin freebsd_committer 2018-11-20 14:43:35 UTC
(In reply to Ed Maste from comment #8)

r340408 is higher than r340395
Comment 10 Ed Maste freebsd_committer 2018-11-20 14:59:45 UTC
(In reply to Antoine Brodin from comment #9)
> r340408 is higher than r340395

Yes, so the amd64 results in this PR are from before andrew's fix and the i386 results are after the fix.
Comment 11 Antoine Brodin freebsd_committer 2018-11-20 15:01:04 UTC
On amd64, there were 0 new failure at r340409
Comment 12 Andrew Turner freebsd_committer 2018-11-20 20:17:16 UTC
I'm getting the same error with lang/erlang-runtime21 when I use the gnu crtbegin/end code.
Comment 13 Andrew Turner freebsd_committer 2018-11-26 16:50:04 UTC
The BSD crtbegin/end has been enabled by default so close.
Comment 14 Antoine Brodin freebsd_committer 2018-12-27 18:04:18 UTC
Re-open,  there is a regression with lang/erlang-runtime21 on head-i386.

I can't reproduce the failure with gnu crtbegin / crtend.
Comment 15 Mark Millard 2018-12-31 08:30:56 UTC
(In reply to Antoine Brodin from comment #14)

I do not know if tier 2 architectures count as required for
coverage, but if I understand right the BSD crtbegin/crtend
option was disabled for at least powerpc64 because it turned
out to be incomplete and did not work without involving some
of the the old gcc-related code to some extent.

This might be another reason to not close this bugzilla entry.
Comment 16 commit-hook freebsd_committer 2019-09-16 13:42:00 UTC
A commit references this bug:

Author: andrew
Date: Mon Sep 16 13:41:30 UTC 2019
New revision: 352394
URL: https://svnweb.freebsd.org/changeset/base/352394

Log:
  MFC r339738, r339744, r339770, r339773, r339864-r339866, r339907-r339908, r339912-r339913, r339916, r339954, r340213, r340395, r340840, r340910-r340911, r341424, r342773, r342974, r351027, r352073:

  MFC the BSD crtbegin to stable/12 but keep it disabled.

  r339738:
  Implement a BSD licensed crtbegin/crtend

  These are needed for .ctors/.dtors and .jcr handling. The former needs
  all the function pointers to be called in the correct order from the
  .init/.fini section. The latter just needs to call a gcj specific function
  if it exists with a pointer to the start of the .jcr section.

  This is currently disabled until __dso_handle support is added.

  Reviewed by:	emaste
  Sponsored by:	DARPA, AFRL
  Differential Revision:	https://reviews.freebsd.org/D17587

  r339744:
  Add a missing include for src.opts.mk. Without it MK_TESTS isn't defined.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r339770:
  Drop the csu tests WARNS to 5 to fix the powerpc64 build.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r339773:
  Add __dso_handle to the BSD crtbegin. This is used to identify shared
  objects.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r339864:
  Check __dso_handle is NULL in non-DSO objects. It should only be non-NULL
  when accessed from a shared object.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r339865:
  Include the csu test directories in BSD.tests.dist

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r339866:
  Make the .ctors, .dtors, and .jcr markers as static. They shouldn't be
  accessible from out of the files they are defined in.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r339907:
  The jcr argument to _Jv_RegisterClasses is used, stop marking it otherwise.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r339908:
  Run the csu tests on a DSO. This builds the tests into a shared library,
  then runs these from the base test programs. With this we can check
  crtbeginS.o and crtendS.o are working as expected.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r339912:
  Fix the location of the static keyword.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r339913:
  Disable the .preinit_array test in DSOs, ld.bfd fails to link objects with
  the section.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r339916:
  Build the csu tests on all architectures.

  The tests haven't been run them, but this is enough to build them so I can
  get feedback on if the various crt.h headers are correct.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r339954:
  Add __used to __CTOR_LIST__ and __DTOR_LIST__

  Enabling BSD_CRTBEGIN on amd64 resulted in
  error: unused variable '__CTOR_LIST__'.

  __CTOR_LIST__ is indeed unused in crtbegin.c; it marks the beginning of
  the .ctors array and is used in crtend.c.  Annotate __DTOR_LIST__ as
  well for consistency.

  Discussed with:	andrew
  MFC with:	r339738
  Sponsored by:	The FreeBSD Foundation

  r340213:
  Add the (untested) mips and sparc64 .init call sequences.

  The BSD crtbegin/crtend code now builds on all architectures, however
  further work is needed to check if it works correctly.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r340395:
  Run __cxa_finalize in shared objects in the destructor path.

  When we have .dtors call them before .dtor handling, otherwise call from
  a destructor.

  PR:		233056
  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r340840:
  Mark the function called by the MIPS .init/.fini sequence with .local.

  As with r328939 we need to mark local symbols as such. Without this the
  assembly parser treats the symbols as global and created relocations
  against these private symbols.

  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r340910:
  Add the missing 0 at the end of the .jcr section.

  Without this the dynamic library test was failing as it was calling
  _Jv_RegisterClasses multiple times.

  r340911:
  Re-enable the dynamiclib tests. These should be fixed by r340910.

  r341424:
  Disable the BSD CRT code on powerpc and sparc64, they need extra crt*.o
  files that haven't been implemented.

  Reported by:	sbruno
  MFC with:	r339738
  Sponsored by:	DARPA, AFRL

  r342773:
  Add explicit csu test dependency

  lib/csu/tests/dynamiclib requires libh_csu.so be built first.  I'm not
  sure this is the most correct/best way to address this but it solves
  the issue in my testing.

  PR:		233734
  Sponsored by:	The FreeBSD Foundation

  r342974:
  Create crtsavres.o for powerpc builds

  Summary:
  GCC expects to link in a crtsavres.o on powerpc platforms.  On
  powerpc64 this is an empty file, but on powerpc and powerpcspe this does contain
  some save/restore functions, which may not actually be necessary for newer
  modern GCC and clang.  This appeases the in-tree gcc, though, and is needed in
  order to switch to the BSD CRTRBEGIN.

  PR:	233751
  Reviewed By: andrew
  Differential Revision: https://reviews.freebsd.org/D18826

  r351027:
  Enable BSD_CRTBEGIN on powerpc

  In r342974 jhibbits added support to build crtsavres.o. This was the
  blocker for BSD_CRTBEGIN to be enabled there. As such enable this
  option again.

  Reviewed by:	jhibbits
  Sponsored by:	DARPA, AFRL

  r352073:
  csu: Add the riscv .init call sequence

  Reviewed by:	br
  Sponsored by:	Axiado
  Differential Revision:	https://reviews.freebsd.org/D21537

Changes:
_U  stable/12/
  stable/12/etc/mtree/BSD.tests.dist
  stable/12/gnu/lib/Makefile
  stable/12/gnu/lib/csu/Makefile
  stable/12/lib/csu/Makefile
  stable/12/lib/csu/Makefile.inc
  stable/12/lib/csu/aarch64/crt.h
  stable/12/lib/csu/amd64/crt.h
  stable/12/lib/csu/arm/crt.h
  stable/12/lib/csu/common/crtbegin.c
  stable/12/lib/csu/common/crtend.c
  stable/12/lib/csu/i386/crt.h
  stable/12/lib/csu/mips/crt.h
  stable/12/lib/csu/powerpc/Makefile
  stable/12/lib/csu/powerpc/crt.h
  stable/12/lib/csu/powerpc/crtsavres.S
  stable/12/lib/csu/powerpc64/Makefile
  stable/12/lib/csu/powerpc64/crt.h
  stable/12/lib/csu/riscv/crt.h
  stable/12/lib/csu/sparc64/crt.h
  stable/12/lib/csu/tests/
  stable/12/lib/csu/tests/Makefile
  stable/12/lib/csu/tests/Makefile.tests
  stable/12/lib/csu/tests/cxx_constructors.cc
  stable/12/lib/csu/tests/dso/
  stable/12/lib/csu/tests/dynamic/Makefile
  stable/12/lib/csu/tests/dynamiclib/
  stable/12/lib/csu/tests/fini_test.c
  stable/12/lib/csu/tests/init_test.c
  stable/12/lib/csu/tests/static/Makefile
  stable/12/share/mk/src.opts.mk
  stable/12/tools/build/options/WITHOUT_BSD_CRTBEGIN
  stable/12/tools/build/options/WITH_BSD_CRTBEGIN