Bug 220714 - devel/boost-all: Update to 1.65.1
Summary: devel/boost-all: Update to 1.65.1
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-office mailing list
URL: https://reviews.freebsd.org/D11582
Keywords: patch
Depends on: 218835 220715 220717 220718 220719 220723 220724 220725 220760 220762 222132 222190 222207 222439
Blocks: 220783 223922
  Show dependency treegraph
 
Reported: 2017-07-13 20:06 UTC by Jan Beich
Modified: 2017-11-27 22:17 UTC (History)
8 users (show)

See Also:
bugzilla: maintainer-feedback? (office)
jbeich: exp-run?


Attachments
Build failure log for powerpc64 (661.22 KB, text/plain)
2017-07-16 19:03 UTC, Justin Hibbits
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Comment 1 Jan Beich freebsd_committer 2017-07-14 07:29:06 UTC
A few consumers are busted on 11.0 amd64. Maintainers have been notified. At worst on update those can be marked BROKEN per timeout.

### /usr/local/etc/poudriere.d/PR220714-make.conf ###
OPTIONS_SET += BOOST
audio_pms_SET += REGEX
databases_mariadb100-server_SET += OQGRAPH
databases_mariadb101-server_SET += OQGRAPH
databases_mariadb102-server_SET += OQGRAPH
graphics_vigra_SET += NUMPY
textproc_kiss-templates_SET += TEST
www_nghttp2_SET += ASIO
Comment 2 Matthew Rezny freebsd_committer 2017-07-15 13:19:03 UTC
review D11582 is an update to Boost 1.65 Beta 1, not to 1.65.0 as that has not been released. Are you proposing to update Boost in ports to a beta version?
Comment 3 Jan Beich freebsd_committer 2017-07-15 14:35:03 UTC
No. Boost is neither a leaf port nor has frequent security issues. On release (probably in ~2 weeks) I plan to check consumers again and maybe request another exp-run.
Comment 4 Jan Beich freebsd_committer 2017-07-15 14:49:31 UTC
Can someone check the update builds fine on powerpc*? For one, boost-1.65 adds Stacktrace library which uses <unwind.h> from libcxxrt.
Comment 5 Justin Hibbits freebsd_committer 2017-07-16 02:51:41 UTC
Kicked off a poudriere build of devel/boost-all now, for ppc64
Comment 6 Mark Millard 2017-07-16 03:53:02 UTC
(In reply to Jan Beich from comment #4)

What variations are desired to be covered?:

A) a system gcc 4.2.1 based build?
B) some modern lang/gcc* based build(s)?
C) a devel/llvm40 based build?
D) an experimental system-clang-4 based build?

Which of these even involve libcxxrt use
in the code produced and which do not?
Is libcxxrt common to all of the options?

Note: (C) and (D) might be a problem for
running the built code because clang targeting
either 32-bit or 64-bit powerpc does not yet
produce code that handles thrown C++
exceptions in a FreeBSD context: a program
crash results (last I knew).

Also:

*) Variations that experiment with lib32 code
   under 64-bit FreeBSD?

@) 32-bit FreeBSD vs. 64-bit FreeBSD?

There are a lot of combinations overall.


(Note: It may be days before I can get to
such experiments and at the time I'll likely
only have access to one PowerPC machine,
a so-called "Quad-core" PowerMac G5 [2 dual
core processors]. It can run 32-bit FreeBSD
as well though.)


I'm guessing that Justin H. is testing (A)
for 64-bit code on 64-bit powerpc64 FreeBSD.
Off the top of my head I do not know if that
involves libcxxrt for the produced code or
not.
Comment 8 Jan Beich freebsd_committer 2017-07-16 10:07:42 UTC
(In reply to Justin Hibbits from comment #5)
> Kicked off a poudriere build of devel/boost-all now, for ppc64

Can you enable PYTHON option or just build devel/boost-python-libs later? The specific port rarely caused issues in the past.

(In reply to Mark Millard from comment #6)
Thanks for useful questions.

> A) a system gcc 4.2.1 based build?

Mainly this one.

> B) some modern lang/gcc* based build(s)?

After bug 218788 non-x86/non-arm platforms may end up using lang/gcc5 when building boost-libs in C++11 mode. For testing try the following:

devel/boost-all/compiled.mk:
-USES+=		compiler:features
+USES+=		compiler:c++11-lib
+USE_CXXSTD=	c++11

> D) an experimental system-clang-4 based build?

Have you tried if C++ exceptions work on /branches/clang500-import? If "no" then let's focus on GCC for now.

> Is libcxxrt common to all of the options?

<unwind.h> comes from either libcxxrt (if clang/libc++ is enabled) or GCC.
_Unwind_Backtrace() comes from either LLVM libunwind or GCC.

However, GCC 4.2 in base may lack both <unwind.h> and _Unwind_Backtrace(). Whether Boost build is smart to figure this out is an open question. ;)

> @) 32-bit FreeBSD vs. 64-bit FreeBSD?

Only 64-bit would be fine but each Boost update has medium risk of breaking on some architecture. For one, bug 199601 skipped several versions and ended up losing sparc64 support.
Comment 9 Jan Beich freebsd_committer 2017-07-16 12:11:26 UTC
(In reply to Antoine Brodin from comment #7)
No json diffs anymore? There're false positives:
- databases/rocksdb-lite
- devel/arduino-tools rocks
- net-im/ring-daemon
Comment 10 Justin Hibbits freebsd_committer 2017-07-16 19:03:06 UTC
Failed as you suspected it might, with no unwind.h:

In file included from ./boost/stacktrace/safe_dump_to.hpp:209,
                 from ./boost/stacktrace/frame.hpp:20,
                 from ./boost/stacktrace/detail/frame_unwind.ipp:15,
                 from libs/stacktrace/build/../src/addr2line.cpp:10:
./boost/stacktrace/detail/collect_unwind.ipp:18:20: error: unwind.h: No such file or directory

Full log is attached.  Using gcc6.
Comment 11 Justin Hibbits freebsd_committer 2017-07-16 19:03:41 UTC
Created attachment 184409 [details]
Build failure log for powerpc64
Comment 12 Jan Beich freebsd_committer 2017-07-16 19:38:28 UTC
OK, let's try to switch to C++11. Tier3 archs like mips* are blocked by devel/icu requiring lang/gcc*, anyway.

(In reply to Justin Hibbits from comment #10)
Do you have unwind.h under /usr/local/lib/gcc6/ on powerpc*?
Comment 13 Justin Hibbits freebsd_committer 2017-07-16 20:30:57 UTC
(In reply to Jan Beich from comment #12)
Yes, there is a unwind.h in /usr/local/lib/gcc6.  I'll retry with a USES+= compiler:c++11-lib which I thought was already there for boost, so mistakenly thought it was using gcc6 (didn't closely check the log).  Just kicked off another poudriere build, so we'll see how that goes with it.
Comment 14 Justin Hibbits freebsd_committer 2017-07-17 00:43:26 UTC
The build failed again, this time at package phase:

pkg-static: Unable to access file /wrkdirs/usr/ports/devel/boost-libs/work/stage/usr/local/lib/libboost_stacktrace_addr2line.a:No such file or directory
pkg-static: Unable to access file /wrkdirs/usr/ports/devel/boost-libs/work/stage/usr/local/lib/libboost_stacktrace_addr2line.so:No such file or directory
pkg-static: Unable to access file /wrkdirs/usr/ports/devel/boost-libs/work/stage/usr/local/lib/libboost_stacktrace_addr2line.so.1.65.0:No such file or directory
Comment 15 Jan Beich freebsd_committer 2017-07-17 01:24:58 UTC
(In reply to Justin Hibbits from comment #14)
> pkg-static: Unable to access file .../libboost_stacktrace_addr2line.so: No such file or directory

Can you build and run has_addr2line.cpp standalone? What exit code it has? If non-zero why does it fail? C++ exceptions again? If so why only there? For one, Boost runs this code at configure and saves the output in ${BUILD_WRKSRC}/bin.v2/config.log where you can inspect CXXFLAGS on suspiciousness.

https://github.com/boostorg/stacktrace/blob/master/build/has_addr2line.cpp
https://github.com/boostorg/stacktrace/blob/master/build/Jamfile.v2
Comment 16 Jan Beich freebsd_committer 2017-07-17 01:58:30 UTC
Can you try building on powerpc64 with the new patch? I've added USE_CXXSTD=gnu++11 (per bug 220783) which maybe a bit risky on Tier2 archs.
Comment 17 Justin Hibbits freebsd_committer 2017-07-17 14:51:25 UTC
I see what it's doing.  By default it calls /usr/bin/addr2line -h for its test, which returns 1.  /usr/local/bin/addr2line returns 0.
Comment 18 Justin Hibbits freebsd_committer 2017-07-17 14:52:55 UTC
This can be changed by setting BOOST_STACKTRACE_ADDR2LINE_LOCATION preprocessor variable.
Comment 19 Jan Beich freebsd_committer 2017-07-17 15:05:33 UTC
(In reply to Justin Hibbits from comment #17)
> I see what it's doing.  By default it calls /usr/bin/addr2line -h for its test,
> which returns 1.  /usr/local/bin/addr2line returns 0.

This is the same behavior on x86.

$ cat >a.cc
#include <cstdlib>

int main() {
    return std::system("/usr/bin/false");
}

$ c++ a.cc
$ ./a.out
$ echo $?
0
Comment 20 Justin Hibbits freebsd_committer 2017-07-17 15:07:26 UTC
Explicitly setting that via CXXFLAGS didn't change anything, so now I'm at a further loss.  I'm not sure what it really expects.
Comment 21 Jan Beich freebsd_committer 2017-07-17 15:53:21 UTC
std::system() can fail for a number of reasons: broken C++ toolchain, command not found, command is NULL, waitpid() or fork() failure, non-zero status from waitpid(). Looking at the code in our libc and musl, empty argument or non-zero exit status of the last command run by shell aren't those. For one, POSIX states "The termination status shall be as defined for the sh utility; otherwise, the termination status is unspecified." which is kind of confusing given sh -c "foo" does inherit exit status of foo.

Did you really try to compile and run the code snippet?
Comment 22 Justin Hibbits freebsd_committer 2017-07-17 16:08:18 UTC
I just did try running the snippet you pasted, and it returned 0.  Something really doesn't smell right with that.
Comment 23 John Hein 2017-07-17 16:56:32 UTC
Maybe it should be 'return WEXITSTATUS(std::system(...'?
Comment 24 Jan Beich freebsd_committer 2017-07-17 18:04:46 UTC
(In reply to John Hein from comment #23)
Thanks. Fixed: https://reviews.freebsd.org/D11582?vs=30849&id=30878
Comment 26 Mark Millard 2017-07-26 08:42:42 UTC
(In reply to Jan Beich from comment #8)

Jan wrote:

Have you tried if C++ exceptions work on /branches/clang500-import? If "no" then let's focus on GCC for now.

Reply:

I've finally tried clang 5 on powerpc64 targeting
powerpc64 (under -r321493 ) and:

#include <exception>
int main(void)
{
    try { throw std::exception(); }
    catch (std::exception& e) {}
    return 0;
}

still gets SIGSEGV when the a.out produced is
launched.

(TARGET_ARCH=powerpc does not even buildworld
buildkernel via either clang 5 or gcc 4.2.1
currently.)
Comment 27 Shane 2017-08-05 08:29:37 UTC
Just for info, currently the FindBoost cmake module ony recognizes up to boost 1.64 - the test is `Boost_VERSION VERSION_LESS 106500`

This still exists in cmake 3.9.0 so will also need updating upstream.

While it doesn't break builds (unless the project tests _Boost_IMPORTED_TARGETS), it posts the configure error message "Imported targets not available for Boost version 106500" and sets _Boost_IMPORTED_TARGETS to FALSE which can lead you on a wasteful search for what is missing.
Comment 28 Jan Beich freebsd_committer 2017-09-08 00:01:56 UTC
Antoine, can you queue another exp-run with 1.65.1 patch?

(In reply to Shane from comment #27)
> currently the FindBoost cmake module ony recognizes up to boost 1.64 - the test is `Boost_VERSION VERSION_LESS 106500`

Can someone from kde@ take a look? CMake-3.9.2 is still affected.
Comment 29 Jan Beich freebsd_committer 2017-09-08 04:03:52 UTC
Ideally, this should land before 2017Q4 branches even at the cost of a few BROKEN consumers. Backporting (security) updates to quarterly that depend on different Boost versions increases risk of bustage, especially after bug 220783.
Comment 30 Tobias C. Berner freebsd_committer 2017-09-08 05:51:08 UTC
(In reply to Jan Beich from comment #28)
https://github.com/Kitware/CMake/blob/10a3835cbd94d7c7fa6176c142c7a12449dfe160/Modules/FindBoost.cmake

From what I can tell, this is still the case.
Comment 31 Roger Leigh 2017-09-08 06:34:58 UTC
Note that you'll want to pick up https://gitlab.kitware.com/cmake/cmake/merge_requests/1241 to have CMake's FindBoost work with 1.65.1.
Comment 32 Antoine Brodin freebsd_committer 2017-09-10 08:54:06 UTC
Exp-run results on 10.3 i386:

http://package18.nyi.freebsd.org/build.html?mastername=103i386-default-PR220714&build=2017-09-10_08h14m47s

New failures:

+ {"origin"=>"games/pokerth", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"games/stuntrally", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"graphics/aqsis", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"graphics/mitsuba", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"security/botan110", "phase"=>"build-depends", "errortype"=>"???"}
+ {"origin"=>"security/clamfs", "phase"=>"build", "errortype"=>"???"}

6 ports were skipped due to those failures

New failure logs:

http://package18.nyi.freebsd.org/data/103i386-default-PR220714/2017-09-10_08h14m47s/logs/errors/pokerth-1.1.1_15.log
http://package18.nyi.freebsd.org/data/103i386-default-PR220714/2017-09-10_08h14m47s/logs/errors/stuntrally-2.6.log
http://package18.nyi.freebsd.org/data/103i386-default-PR220714/2017-09-10_08h14m47s/logs/errors/aqsis-1.8.2_13.log
http://package18.nyi.freebsd.org/data/103i386-default-PR220714/2017-09-10_08h14m47s/logs/errors/mitsuba-0.5.0_4.log
http://package18.nyi.freebsd.org/data/103i386-default-PR220714/2017-09-10_08h14m47s/logs/errors/botan110-1.10.13_3.log
http://package18.nyi.freebsd.org/data/103i386-default-PR220714/2017-09-10_08h14m47s/logs/errors/clamfs-1.0.1_3.log
Comment 33 Antoine Brodin freebsd_committer 2017-09-10 18:41:03 UTC
Exp-run results on 10.3 amd64:

http://package18.nyi.freebsd.org/build.html?mastername=103amd64-default-PR220714&build=2017-09-10_08h35m34s

New failures:

+ {"origin"=>"games/spring", "phase"=>"build", "errortype"=>"clang"}
+ {"origin"=>"games/stuntrally", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"graphics/aqsis", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"graphics/mitsuba", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"security/botan110", "phase"=>"build-depends", "errortype"=>"???"}
+ {"origin"=>"security/clamfs", "phase"=>"build", "errortype"=>"???"}

New failure logs:

http://package18.nyi.freebsd.org/data/103amd64-default-PR220714/2017-09-10_08h35m34s/logs/errors/spring-98.0_5.log
http://package18.nyi.freebsd.org/data/103amd64-default-PR220714/2017-09-10_08h35m34s/logs/errors/stuntrally-2.6.log
http://package18.nyi.freebsd.org/data/103amd64-default-PR220714/2017-09-10_08h35m34s/logs/errors/aqsis-1.8.2_13.log
http://package18.nyi.freebsd.org/data/103amd64-default-PR220714/2017-09-10_08h35m34s/logs/errors/mitsuba-0.5.0_4.log
http://package18.nyi.freebsd.org/data/103amd64-default-PR220714/2017-09-10_08h35m34s/logs/errors/botan110-1.10.13_3.log
http://package18.nyi.freebsd.org/data/103amd64-default-PR220714/2017-09-10_08h35m34s/logs/errors/clamfs-1.0.1_3.log
Comment 34 Antoine Brodin freebsd_committer 2017-09-13 05:24:22 UTC
New failures on 11.0 i386:

+ {"origin"=>"games/pokerth", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"games/stuntrally", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"graphics/aqsis", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"graphics/mitsuba", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"security/botan110", "phase"=>"build-depends", "errortype"=>"???"}
+ {"origin"=>"security/clamfs", "phase"=>"build", "errortype"=>"???"}
Comment 35 Antoine Brodin freebsd_committer 2017-09-13 05:26:49 UTC
New failures on 11.0 amd64:

+ {"origin"=>"games/stuntrally", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"graphics/aqsis", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"graphics/mitsuba", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"net/ceph", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"net/ceph-devel", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"security/botan110", "phase"=>"build-depends", "errortype"=>"???"}
+ {"origin"=>"security/clamfs", "phase"=>"build", "errortype"=>"???"}

ceph failure logs:

http://package18.nyi.freebsd.org/data/110amd64-default-PR220714/2017-09-12_08h04m20s/logs/errors/ceph-v12.2.0.log
http://package18.nyi.freebsd.org/data/110amd64-default-PR220714/2017-09-12_08h04m20s/logs/errors/ceph-devel-w.v2017.08.15.1.log
Comment 36 Antoine Brodin freebsd_committer 2017-09-13 15:09:37 UTC
New failures on head i386:

+ {"origin"=>"games/pokerth", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"graphics/aqsis", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"graphics/mitsuba", "phase"=>"build", "errortype"=>"???"}
+ {"origin"=>"security/botan110", "phase"=>"build-depends", "errortype"=>"???"}
+ {"origin"=>"security/clamfs", "phase"=>"build", "errortype"=>"???"}
Comment 37 Jan Beich freebsd_committer 2017-09-21 18:57:33 UTC
Thanks. I'll land the patch on 2017-09-25, marking those ports BROKEN. office@ timed out on review/approval 1 month ago (since 1.65.0 actual release).
Comment 38 Jan Beich freebsd_committer 2017-09-25 00:11:55 UTC
Thanks. Landed. Going to watch pkg-fallout@ and bugzilla for a week.
Comment 39 Jan Beich freebsd_committer 2017-09-25 00:14:24 UTC
Oops, ports r450560 referenced different bug.