Bug 204400 - dns/dnstable fails to build with GCC 4.9: /usr/local/lib/libxs.so.2: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20'
Summary: dns/dnstable fails to build with GCC 4.9: /usr/local/lib/libxs.so.2: undefine...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Some People
Assignee: Don Lewis
URL:
Keywords:
Depends on:
Blocks: 196712
  Show dependency treegraph
 
Reported: 2015-11-09 15:16 UTC by Gerald Pfeifer
Modified: 2015-12-08 17:45 UTC (History)
1 user (show)

See Also:
bugzilla: maintainer-feedback? (truckman)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Gerald Pfeifer freebsd_committer 2015-11-09 15:16:25 UTC
We are working to move the default version of GCC in ports (lang/gcc)
from GCC 4.8 to GCC 4.9.

This port is one of a few that blocks this update, cf. 

http://package18.nyi.freebsd.org/data/93amd64-default-PR196712/2015-10-26_13h36m37s/logs/errors/dnstable-0.8.0.log

My guess is that dnstable uses a library that uses GCC, but fails to
use (and link with) GCC itself, hence this undefined reference.  In a
nutshell, if your port uses C++ and links with some C++ code built
with GCC, then the port itself needs to use GCC.
Comment 1 Koop Mast freebsd_committer 2015-11-09 16:59:48 UTC
libxs has USE_GCC=yes. I think the only solution is to add USE_GCC=yes so it is inline with libxs.

I don't know why it doesn't break with GCC 4.8 but perhaps 4.9 builds it differently, and I assume it only breaks on 9.x because either 1) libstc++ is still in base and it gets confused or 2) GCC doesn't know about the undefined symbol, but clang in 10+ does.
Comment 2 Don Lewis freebsd_committer 2015-11-09 21:27:46 UTC
dnstable itself is pure C.  It picks up the libxs dependency via net/nmsg (also pure C and built without USE_GCC).  Adding USE_GCC to dnstable won't help because it also depends on devel/mtbl, which links to libsnappy.so, which is built with the base compiler and apparently contains some C++ code because it is linked against libc++.so.

My preferred hammer for this particular nail is USES=compiler:c++11-lib because it avoids bringing the entire gcc toolchain from ports on FreeBSD 10+.  Users hate it when installing a trivial package also install gcc, binutils, and a whole bunch of other stuff.
Comment 3 Don Lewis freebsd_committer 2015-11-09 21:40:33 UTC
I think that if devel/libxs and archivers/snappy can come to an agreement on the toolchain, then the dnstable build will happy again.  The question then becomes what else gets broken when one or both of those is changed.
Comment 4 Don Lewis freebsd_committer 2015-11-10 00:21:07 UTC
The toolchain disagreement between devel/libxs and archivers/snappy results in this breakage on FreeBSD 10:

% ldd /usr/local/bin/dnstable_convert 
/usr/local/bin/dnstable_convert:
	libdnstable.so.0 => /usr/local/lib/libdnstable.so.0 (0x800823000)
	libmtbl.so.0 => /usr/local/lib/libmtbl.so.0 (0x800a2d000)
	libnmsg.so.6 => /usr/local/lib/libnmsg.so.6 (0x800c3d000)
	libwdns.so.1 => /usr/local/lib/libwdns.so.1 (0x800e5d000)
	libthr.so.3 => /lib/libthr.so.3 (0x80106b000)
	libc.so.7 => /lib/libc.so.7 (0x801290000)
	libyajl.so.2 => /usr/local/lib/libyajl.so.2 (0x80163c000)
	liblz4.so.1 => /usr/local/lib/liblz4.so.1 (0x801845000)
	libz.so.6 => /lib/libz.so.6 (0x801a58000)
	libsnappy.so.1 => /usr/local/lib/libsnappy.so.1 (0x801c6e000)
	libpcap.so.8 => /lib/libpcap.so.8 (0x801e75000)
	libprotobuf-c.so.1 => /usr/local/lib/libprotobuf-c.so.1 (0x8020ba000)
	libxs.so.2 => /usr/local/lib/libxs.so.2 (0x8022c4000)
	libm.so.5 => /lib/libm.so.5 (0x802508000)
->	libc++.so.1 => /usr/lib/libc++.so.1 (0x802731000)
	libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x8029f0000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x802c0d000)
	librt.so.1 => /usr/lib/librt.so.1 (0x802e1b000)
->	libstdc++.so.6 => /usr/local/lib/gcc48/libstdc++.so.6 (0x803021000)

It may build successfully, but this c++ library combo cause it to croak if you try to run it.
Comment 5 Don Lewis freebsd_committer 2015-11-10 01:13:47 UTC
These are the ports that depend on devel/libxs:

dns/dnstable
dns/py-pydnstable
net/axa
net/nmsg
net/p5-Net-Nmsg
net/py-pynmsg
net/sie-nmsg

I don't see anything there that would break if libxs was changed from USE_GCC=yes to USES=compiler:c++11-lib.  That would fix the libc++ vs. libstdc++ in dnstable on FreeBSD 10.

These are the ports that depend on archivers/snappy and their compiler overrides:

archivers/php5-snappy
archivers/py-python-snappy
databases/hbase
databases/leveldb       compiler:c++0x
databases/levigo        custom
databases/mongodb
databases/p5-Mango
databases/p5-Tie-LevelDB
databases/pecl-leveldb
databases/py-leveldb
databases/rocksdb       compiler:c++11-lib
devel/eblob
devel/hadoop2
devel/hive
devel/mtbl
devel/oozie
devel/pig
devel/py-pymtbl
devel/smack
devel/spark
dns/dnstable
dns/py-pydnstable
games/freeminer         compiler:c++11-lib
games/freeminer-default
net-mgmt/unifi2
net-mgmt/unifi3
net-mgmt/unifi4
net-p2p/twister
security/openvpn-devel
/usr/ports/security/py-crits
security/py-pyaff4
security/py-rekall
sysutils/osquery        compiler:c++11-lib
www/chromium            compiler:c++11-lib
www/sahi                compiler:c++11-lib

Clearly changing snappy to USE_GCC=yes would be a bad idea.  Whether or not USES=compiler:c++11-lib would be a good path forward is less clear.

In any case, changing devel/libxs to USES=compiler:c++11-lib would allow this to be added to dnstable, which would get rid of the linker error on FreeBSD 9.
Comment 6 Don Lewis freebsd_committer 2015-11-10 17:34:42 UTC
Sigh ... libxs doesn't build with clang34 on FreeBSD 10.  It does build with clang37 on HEAD.
Comment 7 Don Lewis freebsd_committer 2015-11-11 08:30:54 UTC
The problem with libxs turns out to be a bug in the libc++ include files that are bundled with clang < 3.6.

The patch included in <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204461> eliminates the need for it to use USE_GCC=yes.  That should fix the gcc 4.9 linker error when building dnstable on FreeBSD 9 without having to modify the dnstable port.  It should also fix the linking of both libc++ and libstdc++ when building dnstable on FreeBSD >= 10.
Comment 8 commit-hook freebsd_committer 2015-12-08 01:43:05 UTC
A commit references this bug:

Author: truckman
Date: Tue Dec  8 01:42:11 UTC 2015
New revision: 403247
URL: https://svnweb.freebsd.org/changeset/ports/403247

Log:
  Remove USE_GCC=yes from devel/libxs and always build with the base
  compiler.

  There is a defect in the libc++ header files bundled with clang < 3.6
  that broke the libxs build.  Because of this breakage, USE_GCC=yes
  was added to the port Makefile in r330486.

  Unfortunately that breaks dns/dnstable in two different ways.
  Dnstable itself is pure-C code, but it links to two different
  libraries that contain C++ code, libxs and archivers/snappy, the
  latter of which is built with the base c++ compiler.

    * On FreeBSD 9, snappy is generally built with g++ 4.2 from base
      and linked to libstdc++ in base, whereas libxs is built with g++
      from ports and linked to libstdc++ from ports.  When building
      dnstable, the linker seems to load libsnappy first, which brings
      in libstdc++ from base.  This seems to work fine with ports gcc
      4.8 or older, but when the default ports version is upgraded
      to 4.9, the linker fails with the error:
        "/usr/local/lib/libxs.so.2: undefined reference to
  	`std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20'"

    * On FreeBSD >= 10 where clang is the base compiler and snappy
      is linked to libc++, the build succeeds but the resulting
      executables will fail at runtime because they link to both libc++
      from base and libstdc++ from ports.

  When building libxs on FreeBSD 10 with clang 3.4, the build error is:

    CXX    libxs_la-io_thread.lo
  --- libxs_la-encoder.lo ---
  In file included from encoder.cpp:23:
  In file included from ./encoder.hpp:28:
  In file included from /usr/include/c++/v1/algorithm:626:
  /usr/include/c++/v1/utility:254:9: error: field has incomplete type 'xs::io_thread_t::timer_info_t'
      _T2 second;
  	^

  Patching the code to work around the build failure does not look
  possible, so instead, fix the problem in a rather hackish way when
  compiling with clang < 3.6 and using its bundled c++ headers:

    * Make a local copy of the two defective header files.

    * Apply the upstream change to those files from
      <http://llvm.org/viewvc/llvm-project?view=revision&revision=231119>
      "Allow declaration of map and multimap iterator with incomplete
      mapped type. Patch from eugenis"

    * Add the directory containing the updated header files to the
      CPPFLAGS.

  This fix is not needed when building with base clang on FreeBSD 9
  because it uses the stdc++ headers.

  PR:		204461
  PR:		204400
  PR:		196712
  Approved by:	vg (maintainer)
  MFH:		2015Q4
  Sponsored by:	Farsight Security, Inc.

Changes:
  head/devel/libxs/Makefile
  head/devel/libxs/files/
  head/devel/libxs/files/extra-patch-map
Comment 9 Don Lewis freebsd_committer 2015-12-08 02:23:40 UTC
Fixed by devel/libxs r403247.
Comment 10 commit-hook freebsd_committer 2015-12-08 17:45:24 UTC
A commit references this bug:

Author: truckman
Date: Tue Dec  8 17:44:36 UTC 2015
New revision: 403317
URL: https://svnweb.freebsd.org/changeset/ports/403317

Log:
  MFH: r403247

  Remove USE_GCC=yes from devel/libxs and always build with the base
  compiler.

  There is a defect in the libc++ header files bundled with clang < 3.6
  that broke the libxs build.  Because of this breakage, USE_GCC=yes
  was added to the port Makefile in r330486.

  Unfortunately that breaks dns/dnstable in two different ways.
  Dnstable itself is pure-C code, but it links to two different
  libraries that contain C++ code, libxs and archivers/snappy, the
  latter of which is built with the base c++ compiler.

    * On FreeBSD 9, snappy is generally built with g++ 4.2 from base
      and linked to libstdc++ in base, whereas libxs is built with g++
      from ports and linked to libstdc++ from ports.  When building
      dnstable, the linker seems to load libsnappy first, which brings
      in libstdc++ from base.  This seems to work fine with ports gcc
      4.8 or older, but when the default ports version is upgraded
      to 4.9, the linker fails with the error:
        "/usr/local/lib/libxs.so.2: undefined reference to
  	`std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20'"

    * On FreeBSD >= 10 where clang is the base compiler and snappy
      is linked to libc++, the build succeeds but the resulting
      executables will fail at runtime because they link to both libc++
      from base and libstdc++ from ports.

  When building libxs on FreeBSD 10 with clang 3.4, the build error is:

    CXX    libxs_la-io_thread.lo
  --- libxs_la-encoder.lo ---
  In file included from encoder.cpp:23:
  In file included from ./encoder.hpp:28:
  In file included from /usr/include/c++/v1/algorithm:626:
  /usr/include/c++/v1/utility:254:9: error: field has incomplete type 'xs::io_thread_t::timer_info_t'
      _T2 second;
  	^

  Patching the code to work around the build failure does not look
  possible, so instead, fix the problem in a rather hackish way when
  compiling with clang < 3.6 and using its bundled c++ headers:

    * Make a local copy of the two defective header files.

    * Apply the upstream change to those files from
      <http://llvm.org/viewvc/llvm-project?view=revision&revision=231119>
      "Allow declaration of map and multimap iterator with incomplete
      mapped type. Patch from eugenis"

    * Add the directory containing the updated header files to the
      CPPFLAGS.

  This fix is not needed when building with base clang on FreeBSD 9
  because it uses the stdc++ headers.

  PR:		204461
  PR:		204400
  Approved by:	vg (maintainer)
  Approved by:	ports-secteam (feld)
  Sponsored by:	Farsight Security, Inc.

Changes:
_U  branches/2015Q4/
  branches/2015Q4/devel/libxs/Makefile
  branches/2015Q4/devel/libxs/files/