As part of exp-run in bug 223922 this port was found to break on Boost update. If the issue won't disappear on its own by Boost release I plan to mark the port as BROKEN. Can you check with upstream maintainer or help with a fix? $ fetch -qo- 'https://reviews.freebsd.org/D13279?download=true' | patch -Efsp0 -d /usr/ports $ make all deinstall install clean -C/usr/ports/devel/boost-libs $ make -C/usr/ports/net/ceph [...] /usr/bin/ld: cannot find -lBoost::thread http://package22.nyi.freebsd.org/data/111amd64-default-PR223922/2017-12-03_09h31m05s/logs/errors/ceph-v12.2.0_2.log http://package22.nyi.freebsd.org/data/111amd64-default-PR223922/2017-12-03_09h31m05s/logs/errors/ceph-devel-w.v2017.08.15.1_2.log
It's not really ceph's fault, it's the boost CMake module, which has this list of known Boost versions coded in. There is a cmake 3.10 PR as well, which is now in the post-exp-run-fixing-other-ports phase, I'll add this port to the list to check.
(In reply to Jan Beich from comment #0) Hi Jan, As Adraan writes, there is a thigh lock in between Boost Cmake and the CmakeBoost detection module. So to successfully upgrade boost, this also needs to be fixed in Cmake. Otherwise Cmake will not be able to detect the last update version. This happens every time Boost upgrades. Reported this also when we went from 1.64 to 1.65. Not sure on how to make this äutomagically" work for future updates.
Created attachment 188925 [details] Help find Boost It's a coordination problem, really: - Boost 1.66 isn't released yet - .. therefore (?) CMake can't know about it yet - .. so CMake won't find that Boost version (yet) Generally a new CMake release with an updated FindBoost is released shortly after a Boost release. That causes an bigger coordination problem (for us packagers), since now you have a boost update delayed because it breaks stuff that uses cmake, and a week or more passes while cmake is released, and then another week for cmake's exp-run and attendant fallout. Bleh. CMake git doesn't have a (pre) updated FindBoost, either. For testing purposes, please find a (git) patch which you could apply to /usr/local/share/cmake/Modules/FindBoost.cmake .. it might do the trick to at least *find* the beta Boost version.
Other approach: add CMAKE_ARGS+=-DBoost_ADDITIONAL_VERSIONS="1.66" (learned about this in #cmake) to ceph as part of the boost update, which might work *if* ceph doesn't use imported targets. Upstream CMake bug report is https://gitlab.kitware.com/cmake/cmake/issues/17575
That is a definite BLEH.... But the override would allow to run an exp-run with 1.66. set. And that should prevent fallout from Ceph I guess. Not sure what imported targets are within Cmake, but I guess we'll see.
Boost 1.66.0 has been finally released. (In reply to Adriaan de Groot from comment #4) > Other approach: add CMAKE_ARGS+=-DBoost_ADDITIONAL_VERSIONS="1.66" Doesn't help: 1.66 - http://sprunge.us/QIgI, 1.66.0 - http://sprunge.us/iPPP
So now it is up to CMake to do the "right thing". The Ceph project has also noticed the release, and is going to "force" 1.66 usage due to new features in Beast. So lets hope for a fast fix in CMake, or a patch on the current version.
I'm told (see upstream bug report) that "recent" CMake should find Boost 1.66 and reduce the problems caused by new Boost versions being not-found. I just committed CMake 3.10.1 to the ports tree, so I hope that this blocker for Boost update / ceph has gone away now.
(In reply to Adriaan de Groot from comment #8) > I just committed CMake 3.10.1 to the ports tree, so I hope that this > blocker for Boost update / ceph has gone away now. No joy. 11.1R + boost 1.66 + ceph - https://clbin.com/lJxer 11.1R + boost 1.66 + ceph-devel - https://clbin.com/b9KUT 12.0C + boost 1.66 + ceph - https://clbin.com/JRGuo 12.0C + boost 1.66 + ceph-devel - https://clbin.com/LhGd2
(In reply to Jan Beich from comment #9) The roor is: [ 39%] Built target manpages /usr/bin/ld: cannot find -lBoost::thread c++: error: linker command failed with exit code 1 (use -v to see invocation) gmake[3]: *** [src/CMakeFiles/ceph-common.dir/build.make:421: lib/libceph-common.so.0] Error 1 Which suggests that between 1.65 and 1.66 something changed in the packaging of Boost libraries? And it needs to be loaded in the dependencies? Currently rather busy with X-mas preps, so might need so time to look into this.
https://gitlab.kitware.com/cmake/cmake/commit/21f53ec9af07 doesn't help. 11.1R + boost 1.66 + ceph - https://ptpb.pw/zZKs 11.1R + boost 1.66 + ceph-devel - https://ptpb.pw/HPVs 12.0C + boost 1.66 + ceph - https://ptpb.pw/vdcq 12.0C + boost 1.66 + ceph-devel - https://ptpb.pw/6jAa
(In reply to Jan Beich from comment #11) Hi Jan, A regular ceph build in jenkinst would look a like: http://cephdev.digiware.nl:8180/jenkins/job/ceph-master/lastBuild/console and if you there search for boost_thread you'll see that in that build the libraries are bound in a totally different way: /usr/local/lib/libboost_thread.so So I'm not why all of a sudden Cmake want to transform this into -lBoost::thread. Where I assume that Cmake expects Clang to find the correct lib with this option? I certainly do not follow the development of CMake as it is, and guess I'll have to look into this. But I'm rather busy ATM. Is Ceph the only one giving you such a hard time?
The magic is this: CMake's FindBoost module defines something called an *imported target*, which is called something like Boost::Thread, or Boost::date_time. These imported targets do a couple of things when they are linked-to (e.g. by doing `target_link_libraries(ceph Boost::Thread)` ): - add include paths - expand to a bunch of -l flags When the imported target isn't defined, CMake thinks it's just a regular library name.
(In reply to Adriaan de Groot from comment #13) I've also asked the Ceph CMake wizard to take a look at this. It sort of annoys me that an upgrade would break in this weird way. Could be that Ceph was using a deprecated feature, but usually these are prevented far before they get final. And testing this, I have to be careful not to break my working environment.
Created attachment 189155 [details] Sample CMakeLists that looks for a boost library This is a tarball (extracts to cmake-boost/) with a sample CMakeLists.txt and a stupidly simple program that uses Boost's date-time library. There is also a Makefile that creates a build directory, invokes CMake, and builds the result -- I *imagine* it will fail at the link stage if you have Boost 1.66 installed.
Upstream hasn't got a commit yet for Boost 1.66, so we're (in the most general sense: any project that uses CMake, and Boost, and uses imported targets) kind of stuck: a new Boost will break stuff. Ceph might switch to not using the imported targets, but that means they have to handle all the magic of imported targets by hand. I think it's time to give CMake upstream a kicking again (or we should just apply the patch "Help find Boost" attached to this PR, as a patch to CMake and bump the portrevision; that would usually involve an exp-run which we should fold into any new Boost exp-run).
Comment on attachment 188925 [details] Help find Boost After applying the patch Boost 1.66 is detected fine but the build is still busted. Ceph bundles Beast library which is now part of Boost, not sure if there're any API differences. 11.1 amd64 + net/ceph - https://ptpb.pw/tX2s 11.1 amd64 + net/ceph-devel - https://ptpb.pw/1XaE 12.0 amd64 + net/ceph - https://ptpb.pw/a4Qb 12.0 amd64 + net/ceph-devel - https://ptpb.pw/ZjKR In file included from src/rgw/rgw_asio_frontend.cc:13: In file included from src/rgw/../Beast/include/beast/http/read.hpp:12: src/rgw/../Beast/include/beast/core/async_completion.hpp:73:11: error: no matching constructor for initialization of 'boost::asio::async_result<handler_type>' (aka 'async_result<coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, unsigned long> >') , result(handler) ^ ~~~~~~~ src/rgw/../Beast/include/beast/http/impl/async_read.ipp:584:40: note: in instantiation of member function 'beast::async_completion<boost::asio::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::executor> >, void (boost::system::error_code, unsigned long)>::async_completion' requested here void(error_code, std::size_t)> completion{handler}; ^ src/rgw/../Beast/include/beast/http/impl/async_read.ipp:650:20: note: in instantiation of function template specialization 'beast::http::detail::async_read_some<boost::asio::basic_stream_socket<boost::asio::ip::tcp>, beast::basic_flat_streambuf<std::__1::allocator<char> >, true, beast::http::message_parser<true, rgw::asio::streaming_body, beast::http::basic_fields<std::__1::allocator<char> > >, boost::asio::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::executor> > >' requested here return detail::async_read_some(stream, dynabuf, parser, ^ src/rgw/rgw_asio_frontend.cc:86:33: note: in instantiation of function template specialization 'beast::http::async_read_some<boost::asio::basic_stream_socket<boost::asio::ip::tcp>, beast::basic_flat_streambuf<std::__1::allocator<char> >, true, true, beast::http::message_parser<true, rgw::asio::streaming_body, beast::http::basic_fields<std::__1::allocator<char> > >, boost::asio::basic_yield_context<boost::asio::executor_binder<void (*)(), boost::asio::executor> > >' requested here auto bytes = beast::http::async_read_some(socket, buffer, parser, yield[ec]); ^ /usr/local/include/boost/asio/async_result.hpp:50:7: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'handler_type' (aka 'coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, unsigned long>') to 'const boost::asio::async_result<boost::asio::detail::coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, unsigned long>, void>' for 1st argument class async_result ^ /usr/local/include/boost/asio/impl/spawn.hpp:316:12: note: candidate constructor not viable: no known conversion from 'handler_type' (aka 'coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, unsigned long>') to 'typename detail::coro_async_result<executor_binder<void (*)(), executor>, void>::completion_handler_type &' (aka 'coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, void> &') for 1st argument explicit async_result( ^ 1 error generated.
At least we've moved on from CMake being the problem, to ceph C++ code being the problem. I'll wager that smacking ceph with a hammer to use the upstream Beast is the right thing to do -- Beast is header-only, so it doesn't need any fancy CMakery.
.. wager lost. The embedded Beast is a rather old version, so just removing it shows up compatibility problems between the ceph code and Boost::asio. I don't feel like taking shots in the dark at fixing this, since I'm not familiar enough with Boost changes. So status right now is: ceph needs to fix their C++ code. Please include https://reviews.freebsd.org/D13662 in any future exp-run for Boost 1.66.
(In reply to Adriaan de Groot from comment #19) This was sort of expected, since the person that wanted Ceph to move to Boost 1.66 was excited about the new Beast features... I'll try and see if I can work this out with him before Ceph goes 1.66. And it is sort of my own problem, since I opted to remove Boost from with the Ceph build tree, so its source distro gets a lot smaller. But then that means we have to live with what is available. And that also trhows a fit once in while. Thanx for all the help Adriaan
(In reply to Willem Jan Withagen from comment #20) I was able to build an upgraded version of the Ceph source v12.2.2 with boost 1.66 and a patched CMake (with the already available patch in the PR) It will exclude the building of the one tool that uses beast::asio, but there is an alternate version for this. So once CMake is fixed to actually allow for Boost 1.66, I can submit an upgrade for the net/ceph port to v12.2.2. But that upgrade will also work with the current Boost 1.65. So now I will have to sharpen my phabricator skills again. I will also upload the net/ceph shar here for those that want to try.
Created attachment 189230 [details] net/ceph v12.2.2 net shar-file net/ceph needs a upgrade to allow for a upgrade in ports to Boost 1.66. Note that it first needs an upgrade of CMake to 3.10.1 with a patch to actually make it recognize Boost 1.66
(In reply to Willem Jan Withagen from comment #22) Up for phabricator review on: https://reviews.freebsd.org/D13699
A commit references this bug: Author: jbeich Date: Mon Jan 1 01:36:00 UTC 2018 New revision: 457752 URL: https://svnweb.freebsd.org/changeset/ports/457752 Log: net/ceph: update to 12.2.2 Changes: http://ceph.com/releases/v12-2-1-luminous-released/ Changes: http://ceph.com/releases/v12-2-2-luminous-released/ PR: 224185 Submitted by: Willem Jan Withagen (maintainer) Differential Revision: https://reviews.freebsd.org/D13699 Changes: head/net/ceph/Makefile head/net/ceph/distinfo head/net/ceph/files/ceph.in head/net/ceph/files/file-git_version head/net/ceph/files/patch-src_tools_ceph__kvstore__tool.cc head/net/ceph/pkg-plist
A commit references this bug: Author: jbeich Date: Thu Jan 18 04:01:24 UTC 2018 New revision: 459314 URL: https://svnweb.freebsd.org/changeset/ports/459314 Log: devel/cmake: unbreak net/ceph with boost 1.66 PR: 224185 Submitted by: adridg Exp-run by: antoine Differential Revision: https://reviews.freebsd.org/D13662 Changes: head/devel/cmake/Makefile head/devel/cmake/files/patch-Modules_FindBoost.cmake
Boost 1.66 has landed. I've marked net/ceph-devel as BROKEN per comment 17.
Once there is a dev release in Ceph mimic (13.x) I'll update ceph-dev, and unBROKEN it at that time. Thanx for all you help
I guess we can close this one then? In the meantime, there's a new generation of broken-with-boost-1.67 PRs.
OK. When Boost 1.67 lands it will exacerbate BROKEN in ceph-devel, anyway.
(In reply to Jan Beich from comment #29) Just this morning one the the core developers for Ceph started the dicussion of EOL-ing all stuff that depends on 2.7, and only work on using 3.x. Because 2.7 is EOL in 2020.... So I'm expecting even more fallout. Question what is going to be needed to keep FreeBSD packages happy. But I guess I'll find out in due time.
ceph-devel has EXPIRATION_DATE via ports r474787. Do you plan to fix or update the port?
(In reply to Jan Beich from comment #31) I think we should not... I'm in the process of updating the V12 port, and creating a V13 port. Ceph-dev would be 14.0-Current in FreeBSD speak. But people really wanting that should actually get ceph/master, since that now a days readily builds on FreeBSD. So I guess I can let it go. --WjW