Bug 224185

Summary: [cmake] net/ceph and net/ceph-devel: fails to build with boost 1.66
Product: Ports & Packages Reporter: Jan Beich <jbeich>
Component: Individual Port(s)Assignee: freebsd-ports-bugs (Nobody) <ports-bugs>
Status: Closed FIXED    
Severity: Affects Only Me CC: adridg, kde, wjw
Priority: --- Keywords: needs-patch
Version: LatestFlags: bugzilla: maintainer-feedback? (wjw)
Hardware: Any   
OS: Any   
Bug Depends on: 222207    
Bug Blocks: 223922    
Description Flags
Help find Boost
Sample CMakeLists that looks for a boost library
net/ceph v12.2.2 net shar-file none

Description Jan Beich freebsd_committer 2017-12-08 16:08:17 UTC
As part of exp-run in bug 223922 this port was found to break on Boost update. If the issue won't disappear on its own by Boost release I plan to mark the port as BROKEN. Can you check with upstream maintainer or help with a fix?

$ fetch -qo- 'https://reviews.freebsd.org/D13279?download=true' | patch -Efsp0 -d /usr/ports
$ make all deinstall install clean -C/usr/ports/devel/boost-libs
$ make -C/usr/ports/net/ceph
/usr/bin/ld: cannot find -lBoost::thread

Comment 1 Adriaan de Groot freebsd_committer 2017-12-08 16:53:18 UTC
It's not really ceph's fault, it's the boost CMake module, which has this list of known Boost versions coded in. There is a cmake 3.10 PR as well, which is now in the post-exp-run-fixing-other-ports phase, I'll add this port to the list to check.
Comment 2 Willem Jan Withagen 2017-12-09 12:23:33 UTC
(In reply to Jan Beich from comment #0)

Hi Jan,

As Adraan writes, there is a thigh lock in between Boost Cmake and the CmakeBoost detection module. So to successfully upgrade boost, this also needs to be fixed in Cmake.
Otherwise Cmake will not be able to detect the last update version.

This happens every time Boost upgrades.

Reported this also when we went from 1.64 to 1.65. Not sure on how to make this äutomagically" work for future updates.
Comment 3 Adriaan de Groot freebsd_committer 2017-12-18 09:44:06 UTC
Created attachment 188925 [details]
Help find Boost

It's a coordination problem, really:
 - Boost 1.66 isn't released yet
 - .. therefore (?) CMake can't know about it yet
 - .. so CMake won't find that Boost version (yet)

Generally a new CMake release with an updated FindBoost is released shortly after a Boost release. That causes an bigger coordination problem (for us packagers), since now you have a boost update delayed because it breaks stuff that uses cmake, and a week or more passes while cmake is released, and then another week for cmake's exp-run and attendant fallout.


CMake git doesn't have a (pre) updated FindBoost, either.

For testing purposes, please find a (git) patch which you could apply to /usr/local/share/cmake/Modules/FindBoost.cmake .. it might do the trick to at least *find* the beta Boost version.
Comment 4 Adriaan de Groot freebsd_committer 2017-12-18 10:20:18 UTC
Other approach: add CMAKE_ARGS+=-DBoost_ADDITIONAL_VERSIONS="1.66" (learned about this in #cmake) to ceph as part of the boost update, which might work *if* ceph doesn't use imported targets.

Upstream CMake bug report is https://gitlab.kitware.com/cmake/cmake/issues/17575
Comment 5 Willem Jan Withagen 2017-12-18 10:26:47 UTC
That is a definite BLEH....

But the override would allow to run an exp-run with 1.66. set.
And that should prevent fallout from Ceph I guess.
Not sure what imported targets are within Cmake, but I guess we'll see.
Comment 6 Jan Beich freebsd_committer 2017-12-18 15:38:19 UTC
Boost 1.66.0 has been finally released.

(In reply to Adriaan de Groot from comment #4)
> Other approach: add CMAKE_ARGS+=-DBoost_ADDITIONAL_VERSIONS="1.66"

Doesn't help: 1.66 - http://sprunge.us/QIgI, 1.66.0 - http://sprunge.us/iPPP
Comment 7 Willem Jan Withagen 2017-12-20 07:52:30 UTC
So now it is up to CMake to do the "right thing".

The Ceph project has also noticed the release, and is going to "force" 1.66 usage due to new features in Beast. So lets hope for a fast fix in CMake, or a patch on the current version.
Comment 8 Adriaan de Groot freebsd_committer 2017-12-23 10:47:55 UTC
I'm told (see upstream bug report) that "recent" CMake should find Boost 1.66 and reduce the problems caused by new Boost versions being not-found. I just committed CMake 3.10.1 to the ports tree, so I hope that this blocker for Boost update / ceph has gone away now.
Comment 9 Jan Beich freebsd_committer 2017-12-23 23:10:40 UTC
(In reply to Adriaan de Groot from comment #8)
> I just committed CMake 3.10.1 to the ports tree, so I hope that this
> blocker for Boost update / ceph has gone away now.

No joy.

11.1R + boost 1.66 + ceph       - https://clbin.com/lJxer
11.1R + boost 1.66 + ceph-devel - https://clbin.com/b9KUT
12.0C + boost 1.66 + ceph       - https://clbin.com/JRGuo
12.0C + boost 1.66 + ceph-devel - https://clbin.com/LhGd2
Comment 10 Willem Jan Withagen 2017-12-24 10:52:25 UTC
(In reply to Jan Beich from comment #9)
The roor is:
[ 39%] Built target manpages
/usr/bin/ld: cannot find -lBoost::thread
c++: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[3]: *** [src/CMakeFiles/ceph-common.dir/build.make:421: lib/libceph-common.so.0] Error 1

Which suggests that between 1.65 and 1.66 something changed in the packaging of Boost libraries? And it needs to be loaded in the dependencies?

Currently rather busy with X-mas preps, so might need so time to look into this.
Comment 11 Jan Beich freebsd_committer 2017-12-27 01:58:31 UTC
https://gitlab.kitware.com/cmake/cmake/commit/21f53ec9af07 doesn't help.

11.1R + boost 1.66 + ceph       - https://ptpb.pw/zZKs
11.1R + boost 1.66 + ceph-devel - https://ptpb.pw/HPVs
12.0C + boost 1.66 + ceph       - https://ptpb.pw/vdcq
12.0C + boost 1.66 + ceph-devel - https://ptpb.pw/6jAa
Comment 12 Willem Jan Withagen 2017-12-27 09:21:33 UTC
(In reply to Jan Beich from comment #11)

Hi Jan,

A regular ceph build in jenkinst would look a like:

and if you there search for boost_thread you'll see that in that build the libraries are bound in a totally different way:

So I'm not why all of a sudden Cmake want to transform this into -lBoost::thread.
Where I assume that Cmake expects Clang to find the correct lib with this option?

I certainly do not follow the development of CMake as it is, and guess I'll have to look into this. But I'm rather busy ATM.

Is Ceph the only one giving you such a hard time?
Comment 13 Adriaan de Groot freebsd_committer 2017-12-27 17:29:21 UTC
The magic is this: CMake's FindBoost module defines something called an *imported target*, which is called something like Boost::Thread, or Boost::date_time. These imported targets do a couple of things when they are linked-to (e.g. by doing `target_link_libraries(ceph Boost::Thread)` ):
 - add include paths
 - expand to a bunch of -l flags

When the imported target isn't defined, CMake thinks it's just a regular library name.
Comment 14 Willem Jan Withagen 2017-12-27 17:57:23 UTC
(In reply to Adriaan de Groot from comment #13)

I've also asked the Ceph CMake wizard to take a look at this.

It sort of annoys me that an upgrade would break in this weird way.
Could be that Ceph was using a deprecated feature, but usually these are prevented far before they get final.

And testing this, I have to be careful not to break my working environment.
Comment 15 Adriaan de Groot freebsd_committer 2017-12-28 11:02:55 UTC
Created attachment 189155 [details]
Sample CMakeLists that looks for a boost library

This is a tarball (extracts to cmake-boost/) with a sample CMakeLists.txt and a stupidly simple program that uses Boost's date-time library. There is also a Makefile that creates a build directory, invokes CMake, and builds the result -- I *imagine* it will fail at the link stage if you have Boost 1.66 installed.
Comment 16 Adriaan de Groot freebsd_committer 2017-12-28 14:06:54 UTC
Upstream hasn't got a commit yet for Boost 1.66, so we're (in the most general sense: any project that uses CMake, and Boost, and uses imported targets) kind of stuck: a new Boost will break stuff. Ceph might switch to not using the imported targets, but that means they have to handle all the magic of imported targets by hand.

I think it's time to give CMake upstream a kicking again (or we should just apply the patch "Help find Boost" attached to this PR, as a patch to CMake and bump the portrevision; that would usually involve an exp-run which we should fold into any new Boost exp-run).
Comment 17 Jan Beich freebsd_committer 2017-12-28 20:09:18 UTC
Comment on attachment 188925 [details]
Help find Boost

After applying the patch Boost 1.66 is detected fine but the build is still busted. Ceph bundles Beast library which is now part of Boost, not sure if there're any API differences.

11.1 amd64 + net/ceph       - https://ptpb.pw/tX2s
11.1 amd64 + net/ceph-devel - https://ptpb.pw/1XaE
12.0 amd64 + net/ceph       - https://ptpb.pw/a4Qb
12.0 amd64 + net/ceph-devel - https://ptpb.pw/ZjKR

In file included from src/rgw/rgw_asio_frontend.cc:13:
In file included from src/rgw/../Beast/include/beast/http/read.hpp:12:
src/rgw/../Beast/include/beast/core/async_completion.hpp:73:11: error: no matching constructor for initialization of 'boost::asio::async_result<handler_type>' (aka 'async_result<coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, unsigned long> >')
        , result(handler)
          ^      ~~~~~~~
src/rgw/../Beast/include/beast/http/impl/async_read.ipp:584:40: note: in instantiation of member function 'beast::async_completion<boost::asio::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::executor> >, void (boost::system::error_code, unsigned long)>::async_completion' requested here
        void(error_code, std::size_t)> completion{handler};
src/rgw/../Beast/include/beast/http/impl/async_read.ipp:650:20: note: in instantiation of function template specialization 'beast::http::detail::async_read_some<boost::asio::basic_stream_socket<boost::asio::ip::tcp>, beast::basic_flat_streambuf<std::__1::allocator<char> >, true, beast::http::message_parser<true, rgw::asio::streaming_body, beast::http::basic_fields<std::__1::allocator<char> > >, boost::asio::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::executor> > >' requested here
    return detail::async_read_some(stream, dynabuf, parser,
src/rgw/rgw_asio_frontend.cc:86:33: note: in instantiation of function template specialization 'beast::http::async_read_some<boost::asio::basic_stream_socket<boost::asio::ip::tcp>, beast::basic_flat_streambuf<std::__1::allocator<char> >, true, true, beast::http::message_parser<true, rgw::asio::streaming_body, beast::http::basic_fields<std::__1::allocator<char> > >, boost::asio::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::executor> > >' requested here
      auto bytes = beast::http::async_read_some(socket, buffer, parser, yield[ec]);
/usr/local/include/boost/asio/async_result.hpp:50:7: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'handler_type' (aka 'coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, unsigned long>') to 'const boost::asio::async_result<boost::asio::detail::coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, unsigned long>, void>' for 1st argument
class async_result
/usr/local/include/boost/asio/impl/spawn.hpp:316:12: note: candidate constructor not viable: no known conversion from 'handler_type' (aka 'coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, unsigned long>') to 'typename detail::coro_async_result<executor_binder<void (*)(), executor>, void>::completion_handler_type &' (aka 'coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, void> &') for 1st argument
  explicit async_result(
1 error generated.
Comment 18 Adriaan de Groot freebsd_committer 2017-12-29 09:32:47 UTC
At least we've moved on from CMake being the problem, to ceph C++ code being the problem. I'll wager that smacking ceph with a hammer to use the upstream Beast is the right thing to do -- Beast is header-only, so it doesn't need any fancy CMakery.
Comment 19 Adriaan de Groot freebsd_committer 2017-12-29 14:51:28 UTC
.. wager lost. The embedded Beast is a rather old version, so just removing it shows up compatibility problems between the ceph code and Boost::asio. I don't feel like taking shots in the dark at fixing this, since I'm not familiar enough with Boost changes.

So status right now is: ceph needs to fix their C++ code.

Please include https://reviews.freebsd.org/D13662 in any future exp-run for Boost 1.66.
Comment 20 Willem Jan Withagen 2017-12-29 15:50:20 UTC
(In reply to Adriaan de Groot from comment #19)

This was sort of expected, since the person that wanted Ceph to move to Boost 1.66 was excited about the new Beast features...
I'll try and see if I can work this out with him before Ceph goes 1.66.

And it is sort of my own problem, since I opted to remove Boost from with the Ceph build tree, so its source distro gets a lot smaller. But then that means we have to live with what is available. And that also trhows a fit once in while.

Thanx for all the help Adriaan
Comment 21 Willem Jan Withagen 2017-12-30 22:32:14 UTC
(In reply to Willem Jan Withagen from comment #20)

I was able to build an upgraded version of the Ceph source v12.2.2 with
boost 1.66 and a patched CMake (with the already available patch in the PR)

It will exclude the building of the one tool that uses beast::asio, but there is an alternate version for this.

So once CMake is fixed to actually allow for Boost 1.66, I can submit an upgrade for the net/ceph port to v12.2.2. But that upgrade will also work with the current Boost 1.65.

So now I will have to sharpen my phabricator skills again.
I will also upload the net/ceph shar here for those that want to try.
Comment 22 Willem Jan Withagen 2017-12-30 22:37:03 UTC
Created attachment 189230 [details]
net/ceph v12.2.2 net shar-file

net/ceph needs a upgrade to allow for a upgrade in ports to Boost 1.66.
Note that it first needs an upgrade of CMake to 3.10.1 with a patch to actually make it recognize Boost 1.66
Comment 23 Willem Jan Withagen 2017-12-30 23:05:16 UTC
(In reply to Willem Jan Withagen from comment #22)
Up for phabricator review on:
Comment 24 commit-hook freebsd_committer 2018-01-01 01:36:40 UTC
A commit references this bug:

Author: jbeich
Date: Mon Jan  1 01:36:00 UTC 2018
New revision: 457752
URL: https://svnweb.freebsd.org/changeset/ports/457752

  net/ceph: update to 12.2.2

  Changes:	http://ceph.com/releases/v12-2-1-luminous-released/
  Changes:	http://ceph.com/releases/v12-2-2-luminous-released/
  PR:		224185
  Submitted by:	Willem Jan Withagen (maintainer)
  Differential Revision:	https://reviews.freebsd.org/D13699

Comment 25 commit-hook freebsd_committer 2018-01-18 04:01:55 UTC
A commit references this bug:

Author: jbeich
Date: Thu Jan 18 04:01:24 UTC 2018
New revision: 459314
URL: https://svnweb.freebsd.org/changeset/ports/459314

  devel/cmake: unbreak net/ceph with boost 1.66

  PR:		224185
  Submitted by:	adridg
  Exp-run by:	antoine
  Differential Revision:	https://reviews.freebsd.org/D13662

Comment 26 Jan Beich freebsd_committer 2018-01-18 04:24:01 UTC
Boost 1.66 has landed. I've marked net/ceph-devel as BROKEN per comment 17.
Comment 27 Willem Jan Withagen 2018-01-18 08:30:49 UTC
Once there is a dev release in Ceph mimic (13.x) I'll update ceph-dev, and unBROKEN it at that time.
Thanx for all you help
Comment 28 Adriaan de Groot freebsd_committer 2018-04-17 20:58:37 UTC
I guess we can close this one then? In the meantime, there's a new generation of broken-with-boost-1.67 PRs.
Comment 29 Jan Beich freebsd_committer 2018-04-18 09:10:31 UTC
OK. When Boost 1.67 lands it will exacerbate BROKEN in ceph-devel, anyway.
Comment 30 Willem Jan Withagen 2018-04-18 09:30:58 UTC
(In reply to Jan Beich from comment #29)

Just this morning one the the core developers for Ceph started the dicussion of EOL-ing all stuff that depends on 2.7, and only work on using 3.x. Because 2.7 is EOL in 2020....
So I'm expecting even more fallout.

Question what is going to be needed to keep FreeBSD packages happy. But I guess I'll find out in due time.
Comment 31 Jan Beich freebsd_committer 2018-07-17 09:26:28 UTC
ceph-devel has EXPIRATION_DATE via ports r474787. Do you plan to fix or update the port?
Comment 32 Willem Jan Withagen 2018-07-17 12:40:05 UTC
(In reply to Jan Beich from comment #31)

I think we should not...

I'm in the process of updating the V12 port, and creating a V13 port.
Ceph-dev would be 14.0-Current in FreeBSD speak.

But people really wanting that should actually get ceph/master, since that now a days readily builds on FreeBSD.

So I guess I can let it go.