Bug 218187

Summary: devel/qt5-core, devel/qt5-qmake, devel/binutils: qt5 produces unusable library with binutils
Product: Ports & Packages Reporter: Dmitry Marakasov <amdmi3>
Component: Individual Port(s)Assignee: freebsd-kde (group) <kde>
Status: Closed FIXED    
Severity: Affects Many People CC: Trond.Endrestol, admin, freebsd, ganthore, george, idhameed, jau, joneum, kde, lampa, lantw44, milios, olevole, rakuco, robbak, schulz89, tcberner, vadimkolchev
Priority: --- Keywords: needs-patch, needs-qa
Version: LatestFlags: rakuco: maintainer-feedback+
tcberner: merge-quarterly+
Hardware: Any   
OS: Any   
Attachments:
Description Flags
qmake patch
none
qmake patch with UPDATING entry
none
bsd.qt.mk patch
none
Updated patch. rakuco: maintainer-approval+

Description Dmitry Marakasov freebsd_committer freebsd_triage 2017-03-28 15:43:30 UTC
There's problem in gold linker from new binutils, which produces the following problem:

- when binutils 2.28 are installed
- qmake detects gold and adds use_gold_linker to CONFIG /usr/local/lib/qt5/mkspecs/qmodule.pri:

-CONFIG +=  compile_examples no-libdl qpa largefile use_gold_linker enable_new_dtags sse2 sse3 ssse3 sse4_1 sse4_2 avx avx2 avx512f avx512er avx512cd
+CONFIG +=  compile_examples no-libdl qpa largefile enable_new_dtags sse2 sse3 ssse3 sse4_1 sse4_2 avx avx2 avx512f avx512er avx512cd

- qt5-core builds library which may not be used with conventional linker because of duplicated symbols

This leads to failure when building qt5-widgets after rebuilding qt5-core:

/usr/work/usr/ports/x11-toolkits/qt5-widgets/work/qtbase-opensource-src-5.7.1/lib/libQt5Core.so:(.dynamic+0x27aa0): multiple definition of `__bss_start@Qt_5'
/usr/work/usr/ports/x11-toolkits/qt5-widgets/work/qtbase-opensource-src-5.7.1/lib/libQt5Core.so:(.dynamic+0x27aa0): first defined here
/usr/work/usr/ports/x11-toolkits/qt5-widgets/work/qtbase-opensource-src-5.7.1/lib/libQt5Core.so:(.dynamic+0x27aa0): multiple definition of `_edata@Qt_5'
/usr/work/usr/ports/x11-toolkits/qt5-widgets/work/qtbase-opensource-src-5.7.1/lib/libQt5Core.so:(.dynamic+0x27aa0): first defined here
/usr/work/usr/ports/x11-toolkits/qt5-widgets/work/qtbase-opensource-src-5.7.1/lib/libQt5Core.so:(.dynamic+0x2b2a8): multiple definition of `_end@Qt_5'

related bugs from other distros:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=852035
https://sourceware.org/bugzilla/show_bug.cgi?id=21074

This reveals at least two problems:
- qmake5 should be fixed to never detect gold. Otherwise it creates hidden dependency on binutils which leads to build failures after binutils are deinstalled and kills consistency in general
- gold from binutils port should be fixed to not produce duplicate symbols; there's likely upstream fix for that
Comment 1 Dmitry Marakasov freebsd_committer freebsd_triage 2017-03-28 15:59:00 UTC
Created attachment 181269 [details]
qmake patch

This patch fixes qmake part, making it always ignore gold.
Comment 2 Dmitry Marakasov freebsd_committer freebsd_triage 2017-03-28 16:22:39 UTC
Created attachment 181271 [details]
qmake patch with UPDATING entry

I need kde@ approval for this patch ASAP, for I have seen at least two reports of build failures because of this.

https://lists.freebsd.org/pipermail/freebsd-ports/2017-March/107895.html
http://www.bsdportal.ru/viewtopic.php?f=58&t=28282
Comment 3 Raphael Kubo da Costa freebsd_committer freebsd_triage 2017-03-28 18:45:39 UTC
(In reply to Dmitry Marakasov from comment #2)
> Created attachment 181271 [details]
> qmake patch with UPDATING entry

This doesn't look right, all Qt5 ports built from qtbase (such as qt5-core and qt5-gui) run the configure script and perform the same gold detection that is being avoided in qmake. It makes more sense to change the configuration arguments in Mk/bsd.qt.mk.

However, it's not clear if the bug is really in Qt or in binutils itself. I was reading https://sourceware.org/bugzilla/show_bug.cgi?id=21074 and couldn't figure out if https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=40d7d93ff412f4c34cde3daa04890d5cd2e0d9c9 is supposed to fix this (I'm going to test this later).
Comment 4 Raphael Kubo da Costa freebsd_committer freebsd_triage 2017-03-28 18:50:10 UTC
(In reply to Raphael Kubo da Costa from comment #3)
> couldn't figure out if
> https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;
> h=40d7d93ff412f4c34cde3daa04890d5cd2e0d9c9 is supposed to fix this (I'm
> going to test this later).

For the record, it doesn't, it's already included in binutils 2.28.
Comment 5 Dmitry Marakasov freebsd_committer freebsd_triage 2017-03-28 19:27:45 UTC
(In reply to Raphael Kubo da Costa from comment #3)

> This doesn't look right, all Qt5 ports built from qtbase (such as qt5-core
> and qt5-gui) run the configure script and perform the same gold detection
> that is being avoided in qmake. It makes more sense to change the
> configuration arguments in Mk/bsd.qt.mk.

This is at least partially not true because

1) With this patch, all qt libs build fine with binutils present
2) Without this patch, when qmake is built with binutils present and binutils later removed, there are build failures complaining to nonexisting gold
Comment 6 Raphael Kubo da Costa freebsd_committer freebsd_triage 2017-03-28 21:32:02 UTC
If I just build qt5-core with binutils present, with or without the qmake patch, it still links libQt5Core.so with -fuse-ld=gold.
Comment 7 lampa 2017-03-29 12:02:14 UTC
Add the last line to /usr/ports/Mk/bsd.qt.mk (line 122):

CONFIGURE_ARGS+=-opensource -confirm-license \
                                -platform ${QMAKESPEC} \
                                -no-pch \
                                -no-use-gold-linker \
... etc

I've successfully compiled all the qt5 stuff with this.
Comment 8 Jochen Neumeister freebsd_committer freebsd_triage 2017-04-03 19:11:58 UTC
(In reply to lampa from comment #7)

do not work here. me bsd.qt.mk:

CONFIGURE_ARGS+=-opensource -confirm-license \
				-platform ${QMAKESPEC} \
				-no-pch \
				-no-use-gold-linker \
				-prefix ${PREFIX} \

when i will update x11-toolkits/qt5-widgets:

/usr/ports/x11-toolkits/qt5-widgets/work/qtbase-opensource-src-5.7.1/lib/libQt5Core.so:(.dynamic+0x27a98): first defined here
/usr/ports/x11-toolkits/qt5-widgets/work/qtbase-opensource-src-5.7.1/lib/libQt5Core.so:(.dynamic+0x2b2a0): multiple definition of `_end@Qt_5'
c++: error: linker command failed with exit code 1 (use -v to see invocation)
*** [../../../bin/uic] Error code 1

make[2]: stopped in /usr/ports/x11-toolkits/qt5-widgets/work/qtbase-opensource-src-5.7.1/src/tools/uic
1 error
Comment 9 Dmitry Marakasov freebsd_committer freebsd_triage 2017-04-04 18:00:19 UTC
Created attachment 181488 [details]
bsd.qt.mk patch

Ok, I don't have time to investigate thoroughly atm, but it should work no worse, as it adds -no-use-gold-linker to CONFIGURE_ARGS in both qt5-qmake and all other qt5 ports.
Comment 10 Tobias C. Berner freebsd_committer freebsd_triage 2017-04-05 20:14:48 UTC
*** Bug 218404 has been marked as a duplicate of this bug. ***
Comment 11 Chad Jacob Milios 2017-04-08 12:15:14 UTC
Add me to the list of those who have experienced this bug, found no better way around it, and has had great success using the patch (attachment 181488 [details]) to compile, run and use all my qt5-* ports. Thank you, Dmitry, Raphael, Lampa and Jochen!
Comment 12 Shawn Michael 2017-04-14 16:29:59 UTC
(In reply to Jochen Neumeister from comment #8)
You need to rebuild devel/qt5-core first before building x11-toolkits/qt5-widgets
Comment 13 Shawn Michael 2017-04-14 16:36:26 UTC
The -no-user-gold-linker modification to /usr/ports/Mk/bsd.qt.mk works for me as long as I'm building qt5 related ports.  However if I build a qt4 port it complains that "-no-user-gold-linker" is not a known option.  

I believe that this would need to be an option to configure based on the QT version being built.  It possibly needs to be added to the .else section:

. if ${_QT_VERSION:M4*}
CONFIGURE_ARGS+=...
. else  
CONFIGURE_ARGS+=-nomake examples -nomake tests \
                                -no-use-gold-linker \


I found this running portmaster -af.  I'm testing now to see if the above location works better.
Comment 14 Shawn Michael 2017-04-14 16:44:30 UTC
(In reply to freebsd from comment #13)
And if I would have read Dmitry's comment with patch 181488 that's exactly what people are reporting success with.
Comment 15 Chad Jacob Milios 2017-04-19 04:08:09 UTC
just to clarify:

Mr. Monkeyspunk in comment #13, was reporting he'd experienced failure on qt4 builds when -no-use-gold-linker was added around line 122 as per comment #7 (affecting qt4 and qt5 builds)

However then in comment #14 Mr. Monkeyspunk has noticed that since comment #9 attachment 181488 [details] is already not affecting qt4 builds anymore because it instead adds -no-use-gold-linker to the line 146 region of bsd.qt.mk (which puts it into an existing appropriate 'else' clause).

i have built many many qt4 and qt5 ports successfully today with this patch applied and binutils-2.28,1 installed. now, what i have not verified in the past 10 days is that the original problem still exists in ports/HEAD without this patch applied (that the problem has not been fixed erstwhile elsehow)

if anyone is still experiencing this problem, we should commit this patch (attachment 181488 [details]), i think.
Comment 16 Chad Jacob Milios 2017-04-19 07:44:54 UTC
Confirmed this problem DOES still exist in a clean ports/head as of:

Last Changed Rev: 438814
Last Changed Date: 2017-04-18 19:56:24 +0000 (Tue, 18 Apr 2017)

and yes then applying attachment 181488 [details] solves all of my qt5 problems while it keeps qt4 builds happy. with the patch applied i rebuilt my qt4-* and qt5-* ports happily and also built and tested using multimedia/vlc with qt5 libraries and net-p2p/namecoin with qt4 libraries.

my uname:
FreeBSD yoda.ccsys.com 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #0 r316324: Fri Mar 31 16:18:12 EDT 2017     root@yoda.ccsys.com:/usr/obj/usr/src/sys/NUOS  amd64

Many thank yous to everyone supporting FreeBSD!
Comment 17 ganthore 2017-04-28 14:53:20 UTC
This is still a problem as of 04/28/2017. 

I can confirm that the attached patch fixes the problem after applying it and rebuilding devel/qt5-core.
Comment 18 Victor Schulz 2017-05-09 01:33:43 UTC
I can also confirm that the bug is still present when compiling qt5. I applied the proposed patch and I could build everything without errors.
Comment 19 Vladimir Omelchuk 2017-05-11 05:51:33 UTC
This problem still present. 

# uname -srm
FreeBSD 11.0-RELEASE-p10 amd64

This patch is fine for me too.
Comment 20 Oleg Ginzburg 2017-05-12 13:36:35 UTC
+1, patch works for me
Comment 21 George Mitchell 2017-05-17 17:02:52 UTC
Add me to the list of users affected by the problem, and the patch works for me, too.  What are the chances of getting this committed?
Comment 22 Tobias C. Berner freebsd_committer freebsd_triage 2017-05-18 08:32:50 UTC
I suggest to bump devel/qt5-qmake's PORTREVISION additionally.
Comment 23 Vadim Kolchev 2017-05-21 20:02:44 UTC
Works for me as well.
Comment 24 Kubilay Kocak freebsd_committer freebsd_triage 2017-05-27 14:14:05 UTC
Notify and assign to kde@ (maintainer). Given they werent' notified (officially, other than cc) on issue creation, reset maintainer timeout.

@Dmitry Its always best to assign maintainers (group email, in the case of teams) to issues so that a timeout-reassignment is explicit. If however in this case bapt@ is the only person on kde@ (I don't know), this issue can be considered maintainer timeout immediately, please re-assign accordingly.

I believe the patch also needs to be updated to include the QT4 conditional as outlined in comment 13
Comment 25 Tobias C. Berner freebsd_committer freebsd_triage 2017-05-29 13:33:34 UTC
*** Bug 219633 has been marked as a duplicate of this bug. ***
Comment 26 Raphael Kubo da Costa freebsd_committer freebsd_triage 2017-05-29 22:03:07 UTC
Dmitry, can you attach a patch bumping PORTREVISION? I think we're good to go with this version we have now.
Comment 27 Tobias C. Berner freebsd_committer freebsd_triage 2017-05-30 13:37:38 UTC
Created attachment 183069 [details]
Updated patch.

Updated patch, with current date, and bumped devel/qt5-core and devel/qt5-qmake.
Comment 28 Raphael Kubo da Costa freebsd_committer freebsd_triage 2017-05-30 16:40:38 UTC
Comment on attachment 183069 [details]
Updated patch.

Thanks, please go ahead and commit. Don't forget the MFH tag.
Comment 29 commit-hook freebsd_committer freebsd_triage 2017-06-02 20:03:21 UTC
A commit references this bug:

Author: tcberner
Date: Fri Jun  2 20:03:12 UTC 2017
New revision: 442460
URL: https://svnweb.freebsd.org/changeset/ports/442460

Log:
  Do not use gold linker for Qt5 ports.

    gold linker from binutils 2.28 may produce duplicate library
    symbols, which makes shared libraries created with it not usable
    with conventional ld linker.

  PR:		218187
  Submitted by:	amdmi3

Changes:
  head/Mk/bsd.qt.mk
  head/UPDATING
  head/devel/qt5-core/Makefile
  head/devel/qt5-qmake/Makefile
Comment 30 commit-hook freebsd_committer freebsd_triage 2017-06-12 17:29:54 UTC
A commit references this bug:

Author: tcberner
Date: Mon Jun 12 17:29:04 UTC 2017
New revision: 443455
URL: https://svnweb.freebsd.org/changeset/ports/443455

Log:
  MFH: r442460

  Do not use gold linker for Qt5 ports.

    gold linker from binutils 2.28 may produce duplicate library
    symbols, which makes shared libraries created with it not usable
    with conventional ld linker.

  PR:		218187
  Submitted by:	amdmi3

  Approved by:	ports-secteam (miwi)

Changes:
_U  branches/2017Q2/
  branches/2017Q2/Mk/bsd.qt.mk
  branches/2017Q2/UPDATING
  branches/2017Q2/devel/qt5-core/Makefile
  branches/2017Q2/devel/qt5-qmake/Makefile
Comment 31 Tobias C. Berner freebsd_committer freebsd_triage 2017-06-12 17:32:19 UTC
Sorry for the timeout.