Bug 204461 - [patch] devel/libxs - build with base toolchain whenever possible
Summary: [patch] devel/libxs - build with base toolchain whenever possible
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Don Lewis
URL:
Keywords: patch
Depends on:
Blocks: 196712
  Show dependency treegraph
 
Reported: 2015-11-11 08:27 UTC by Don Lewis
Modified: 2015-12-08 17:46 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (vg)
truckman: merge-quarterly?


Attachments
patch to remove USE_GCC=yes from devel/libxs (8.99 KB, patch)
2015-11-11 08:27 UTC, Don Lewis
no flags Details | Diff
simplified patch to remove USE_GCC=yes from devel/libxs (8.83 KB, patch)
2015-11-11 12:44 UTC, Don Lewis
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Don Lewis freebsd_committer 2015-11-11 08:27:20 UTC
Created attachment 163001 [details]
patch to remove USE_GCC=yes from devel/libxs

The libxs port contains C++ code and currently uses USE_GCC=yes.  This causes two problems for the port dns/dnstable, which is pure C code and uses the base system toolchain, and which also links to the archivers/snappy port (which also contains C++ code and always uses the base system compiler to build):

  On FreeBSD 9, libsnappy is linked to the libstdc++ in base and libxs is
  linked to the ports version of libstdc++ that is bundled with gcc from
  ports.  When linking dnstable, the linker may bring in the base version
  of libstdc++ before it links in libxs and sees that rpath information
  that would tell it to bring in libstdc++ from ports instead.  This may
  result in a link error.  See PRs 196712 and 204400.

  On FreeBSD >= 10, libsnappy is linked to libc++ from base, so the resulting
  executables get linked to both libstdc++ and libc++, which causes crashes at
  runtime.

The attached patch eliminates the need for libxs to use USE_GCC=yes, except
if it is built in a FreeBSD 9 environment where clang is the primary compiler, which is not the default.  With this patch, libxs will link to base libstdc++ on FreeBSD 9, and will link to base libc++ on FreeBSD >= 10.

That said, this patch is extremely hackish.  The biggest problem is that libxs does not build with clang < 3.6.  The problem is not in the code, but rather a bug in the libc++ header files.  Since libc++ is not bundled with the clang ports, using clang from ports does not fix the problem.  The best solution that I could find was to make local copies of the two broken header files, patch them with the specific upstream commit that fixed the bug, and include their location in the compiler include path.

It turns out that libxs does build with the old version of gcc that is in base on FreeBSD 9, so using USE_GCC=yes is not necessary there.   The only case where it is necessary is on a FreeBSD 9 machine that has been configured with clang is the primary compiler and with gcc omitted.
Comment 1 Don Lewis freebsd_committer 2015-11-11 12:44:06 UTC
Created attachment 163006 [details]
simplified patch to remove USE_GCC=yes from devel/libxs

It turns out that clang 3.4.1 in FreeBSD 9.3 base is capable of building libxs without requiring any c++ header patches.  Simplify the Makefile changes so that the base compiler is always used.
Comment 2 Don Lewis freebsd_committer 2015-11-12 17:26:37 UTC
These are the ports that depend on libxs:
 devel/libxs
 dns/dnstable
 dns/py-pydnstable
 net/axa
 net/nmsg
 net/p5-Net-Nmsg
 net/py-pynmsg
 net/sie-nmsg

I've built all of them with the patch attached to this PR on FreeBSD 9.3, 10.1, and 11.0-CURRENT, and on both amd64 and i386.  No problems were detected.

After libxs is fixed, it would probably be a good idea to merge the change to the quarterly ports branch to fix the runtime issue on FreeBSD 10 caused by the libstdc++ vs libc++ conflict.
Comment 3 Veniamin Gvozdikov freebsd_committer 2015-12-03 17:24:45 UTC
Please add build logs for 9.X/10.X/CURRENT and commit it.
Comment 4 Don Lewis freebsd_committer 2015-12-05 22:02:10 UTC
This thread on developers@ starting on October 15 "pkg install nginx-netflix Was:
/usr/local Was: Meanwhile, in another community" discouraged attaching poudriere build logs to bugzilla tickets.

I've uploaded the build logs for FreeBSD 9.3, 10.1, and 11.0-CURRENT amd64 here:
<https://people.freebsd.org/~truckman/libxs/>
Comment 5 Veniamin Gvozdikov freebsd_committer 2015-12-07 09:37:50 UTC
Approved, commit it.
Comment 6 commit-hook freebsd_committer 2015-12-08 01:43:03 UTC
A commit references this bug:

Author: truckman
Date: Tue Dec  8 01:42:11 UTC 2015
New revision: 403247
URL: https://svnweb.freebsd.org/changeset/ports/403247

Log:
  Remove USE_GCC=yes from devel/libxs and always build with the base
  compiler.

  There is a defect in the libc++ header files bundled with clang < 3.6
  that broke the libxs build.  Because of this breakage, USE_GCC=yes
  was added to the port Makefile in r330486.

  Unfortunately that breaks dns/dnstable in two different ways.
  Dnstable itself is pure-C code, but it links to two different
  libraries that contain C++ code, libxs and archivers/snappy, the
  latter of which is built with the base c++ compiler.

    * On FreeBSD 9, snappy is generally built with g++ 4.2 from base
      and linked to libstdc++ in base, whereas libxs is built with g++
      from ports and linked to libstdc++ from ports.  When building
      dnstable, the linker seems to load libsnappy first, which brings
      in libstdc++ from base.  This seems to work fine with ports gcc
      4.8 or older, but when the default ports version is upgraded
      to 4.9, the linker fails with the error:
        "/usr/local/lib/libxs.so.2: undefined reference to
  	`std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20'"

    * On FreeBSD >= 10 where clang is the base compiler and snappy
      is linked to libc++, the build succeeds but the resulting
      executables will fail at runtime because they link to both libc++
      from base and libstdc++ from ports.

  When building libxs on FreeBSD 10 with clang 3.4, the build error is:

    CXX    libxs_la-io_thread.lo
  --- libxs_la-encoder.lo ---
  In file included from encoder.cpp:23:
  In file included from ./encoder.hpp:28:
  In file included from /usr/include/c++/v1/algorithm:626:
  /usr/include/c++/v1/utility:254:9: error: field has incomplete type 'xs::io_thread_t::timer_info_t'
      _T2 second;
  	^

  Patching the code to work around the build failure does not look
  possible, so instead, fix the problem in a rather hackish way when
  compiling with clang < 3.6 and using its bundled c++ headers:

    * Make a local copy of the two defective header files.

    * Apply the upstream change to those files from
      <http://llvm.org/viewvc/llvm-project?view=revision&revision=231119>
      "Allow declaration of map and multimap iterator with incomplete
      mapped type. Patch from eugenis"

    * Add the directory containing the updated header files to the
      CPPFLAGS.

  This fix is not needed when building with base clang on FreeBSD 9
  because it uses the stdc++ headers.

  PR:		204461
  PR:		204400
  PR:		196712
  Approved by:	vg (maintainer)
  MFH:		2015Q4
  Sponsored by:	Farsight Security, Inc.

Changes:
  head/devel/libxs/Makefile
  head/devel/libxs/files/
  head/devel/libxs/files/extra-patch-map
Comment 7 commit-hook freebsd_committer 2015-12-08 17:45:22 UTC
A commit references this bug:

Author: truckman
Date: Tue Dec  8 17:44:36 UTC 2015
New revision: 403317
URL: https://svnweb.freebsd.org/changeset/ports/403317

Log:
  MFH: r403247

  Remove USE_GCC=yes from devel/libxs and always build with the base
  compiler.

  There is a defect in the libc++ header files bundled with clang < 3.6
  that broke the libxs build.  Because of this breakage, USE_GCC=yes
  was added to the port Makefile in r330486.

  Unfortunately that breaks dns/dnstable in two different ways.
  Dnstable itself is pure-C code, but it links to two different
  libraries that contain C++ code, libxs and archivers/snappy, the
  latter of which is built with the base c++ compiler.

    * On FreeBSD 9, snappy is generally built with g++ 4.2 from base
      and linked to libstdc++ in base, whereas libxs is built with g++
      from ports and linked to libstdc++ from ports.  When building
      dnstable, the linker seems to load libsnappy first, which brings
      in libstdc++ from base.  This seems to work fine with ports gcc
      4.8 or older, but when the default ports version is upgraded
      to 4.9, the linker fails with the error:
        "/usr/local/lib/libxs.so.2: undefined reference to
  	`std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20'"

    * On FreeBSD >= 10 where clang is the base compiler and snappy
      is linked to libc++, the build succeeds but the resulting
      executables will fail at runtime because they link to both libc++
      from base and libstdc++ from ports.

  When building libxs on FreeBSD 10 with clang 3.4, the build error is:

    CXX    libxs_la-io_thread.lo
  --- libxs_la-encoder.lo ---
  In file included from encoder.cpp:23:
  In file included from ./encoder.hpp:28:
  In file included from /usr/include/c++/v1/algorithm:626:
  /usr/include/c++/v1/utility:254:9: error: field has incomplete type 'xs::io_thread_t::timer_info_t'
      _T2 second;
  	^

  Patching the code to work around the build failure does not look
  possible, so instead, fix the problem in a rather hackish way when
  compiling with clang < 3.6 and using its bundled c++ headers:

    * Make a local copy of the two defective header files.

    * Apply the upstream change to those files from
      <http://llvm.org/viewvc/llvm-project?view=revision&revision=231119>
      "Allow declaration of map and multimap iterator with incomplete
      mapped type. Patch from eugenis"

    * Add the directory containing the updated header files to the
      CPPFLAGS.

  This fix is not needed when building with base clang on FreeBSD 9
  because it uses the stdc++ headers.

  PR:		204461
  PR:		204400
  Approved by:	vg (maintainer)
  Approved by:	ports-secteam (feld)
  Sponsored by:	Farsight Security, Inc.

Changes:
_U  branches/2015Q4/
  branches/2015Q4/devel/libxs/Makefile
  branches/2015Q4/devel/libxs/files/