Created attachment 248698 [details] foundationdb.diff * Pick up maintainership, as requested by Dave Cottlehuber in bug #266207 * Separate port for client library * Separate port for 7.1.X and 7.3.X * Update 7.1.X to 7.1.57 Changelog: https://github.com/apple/foundationdb/blob/7.1.57/documentation/sphinx/source/release-notes/release-notes-710.rst * Add 7.3.33 Changelog: https://github.com/apple/foundationdb/blob/7.3.33/documentation/sphinx/source/release-notes/release-notes-730.rst
Created attachment 248742 [details] foundationdb.diff some cleanup
happy to commit this, 2 questions first: - do we really want/need 3 fdb versions in tree? I'm sure you have a reason, but IIRC the protocol should support clients built against a lower version to use a newer fdb server, and its a significant build impact on the pkg servers for a comparatively small user base. - do you have a git branch somewhere I can fetch it from? poudriere is building atm ...
(In reply to Dave Cottlehuber from comment #2) > do we really want/need 3 fdb versions in tree? Not 3, only 2 7.1.X and 7.3.X > do you have a git branch somewhere I can fetch it from? No, only attached patch, apply it via "git apply".
none of these built in poudriere, https://pkg.skunkwerks.at/poudriere/build.html?mastername=current_x64-default&build=2024-02-28_16h19m03s -- ADDING SIMULATOR TEST 194 status/separate_not_enough_servers -- ADDING SIMULATOR TEST 195 status/single_process_too_many_config_params CMake Error at /usr/local/share/cmake/Modules/ExternalProject.cmake:2910 (message): error: could not find git for clone of googlebenchmark Call Stack (most recent call first): /usr/local/share/cmake/Modules/ExternalProject.cmake:4418 (_ep_add_download_command) CMakeLists.txt:5 (ExternalProject_Add) and CMakeFiles/fdbserver.dir/DDTeamCollection.actor.g.cpp.o /wrkdirs/usr/ports/databases/foundationdb73-client/work/.build/fdbserver/DDTeamCollection.actor.g.cpp 1. /wrkdirs/usr/ports/databases/foundationdb73-client/work/foundationdb-7.3.33/fdbserver/DDTeamCollection.actor.cpp:6437:3 <Spelling=/wrkdirs/usr/ports/databases/foundationdb73-client/work/foundationdb-7.3.33/flow/include/flow/Error.h:128:51>: current parser token ')' 2. /wrkdirs/usr/ports/databases/foundationdb73-client/work/foundationdb-7.3.33/fdbserver/DDTeamCollection.actor.cpp:5864:1: parsing struct/union/class body 'DDTeamCollectionUnitTest' 3. /wrkdirs/usr/ports/databases/foundationdb73-client/work/.build/fdbserver/DDTeamCollection.actor.g.cpp:28669:2: parsing function body 'DDTeamCollectionUnitTest::GetTeam_ServerUtilizationNearCutoffActorState::a_body1cont1' 4. /wrkdirs/usr/ports/databases/foundationdb73-client/work/.build/fdbserver/DDTeamCollection.actor.g.cpp:28669:2: in compound statement ('{}') 5. /wrkdirs/usr/ports/databases/foundationdb73-client/work/foundationdb-7.3.33/fdbserver/DDTeamCollection.actor.cpp:6437:3 <Spelling=/wrkdirs/usr/ports/databases/foundationdb73-client/work/foundationdb-7.3.33/flow/include/flow/Error.h:127:5>: in compound statement ('{}') I'll try on 14.0-amd64 today and let you know how that goes.
poudriere 14.o-amd64 reports: client:configure - same issue as before [00:20:18] Built ports: devel/libfmt databases/foundationdb73-client databases/foundationdb73-server [00:20:18] Failed ports: databases/foundationdb71-client:configure [00:20:18] Skipped ports: databases/foundationdb71-server [00:20:16] [01] [00:09:38] Finished databases/foundationdb73-server | foundationdb73-server-7.3.33: Success https://pkg.skunkwerks.at/poudriere/build.html?mastername=14_0_x64-default&build=2024-02-29_08h03m49s
(In reply to Dave Cottlehuber from comment #5) I am building it on FreeBSD:14:amd64, maybe there's something wrong with the patch.
are you doing it in poudriere? if not, it will pick up stray state from your filesystem, and also will have network access, which is what's breaking those. If it can't build in poudriere on a 15.0-CURRENT kernel, and a 14.0-RELEASE jail, then this will also fail in official package builders, which are the same setup. Do you have a git repo with a branch and your commits in? I can see if that helps. Also please clarify why you're splitting the ports, FDB clients specifically import a .h file with DFDB_API_VERSION to allow a client to connect to a newer server version than what the client knows of: "$CFLAGS -I/usr/local/include -Ic_src/ -g -Wall -Werror -DFDB_API_VERSION=" ++ MaxAPIVersion ... (from erlfdb port as example). I'd suggest we get 1 update and working version in place, and then split into separate versions as required. Up to you!
(In reply to Dave Cottlehuber from comment #7) > are you doing it in poudriere? Yes, of course. > Do you have a git repo with a branch and your commits in? I can see if that helps. https://gitlab.com/dwagin/freebsd-ports/-/commit/08794524940435ff75779530842f4e2cf83cd86f > Also please clarify why you're splitting the ports In order for packaging to be the same as in other systems. > I'd suggest we get 1 update and working version in place, and then split into separate versions as required. Up to you! I'm currently using the 7.3.X branch, 7.1.X has been left for those who can use it.
thanks for the git link, much easier! Dmitry you are right, 7.3 won't build on FreeBSD current (at least in poudriere) because llvm version is newer and stuff breaks in subtle ways. This doesn't need to block the port update ofc, but we'll need a `BROKEN_FreeBSD_15= Incompatible with LLVM17 in 15-CURRENT` while we find a fix. For the port itself, foundationdb already provides a "backwards-compatible" header file for clients to use and compile against. This means there's no need to have 2 separate port versions in the ports tree, and clients are able to negotiate the required FDB protocol directly. > FDB clients specifically import a .h file with DFDB_API_VERSION to allow > a client to connect to a newer server version than what the client knows: > "$CFLAGS -I/usr/local/include -Ic_src/ -g -Wall -Werror -DFDB_API_VERSION=" ++ MaxAPIVersion ... Can you add a new update that does the nice split of server & client, but doesn't make a separate 7.1 & 7.3 port? IMO this should be ready to go!
Created attachment 249218 [details] single fdb version, appease linters, split client & server, update erlfdb how does this look? I just renamed your 7.33.3, tidied up portclippy & portfmt, and included my LLVM version constraint for 15 builds. Looking good! poudriere tests for 14.0 amd64 and 15-CURRENT amd64 are running atm.
(In reply to Dave Cottlehuber from comment #10) https://apple.github.io/foundationdb/api-general.html#api-versions By default, client and server binaries installed for a given cluster must have the same version. However, there is also a multi-version feature that allows a client to connect to a cluster whose server processes have a different version. This feature works by loading a separate client library that is version-compatible with the cluster, which then proxies all API calls. https://forums.foundationdb.org/t/client-library-backwards-compatibility-issues/919 For a given FDB version major.minor.patch, only .patch-level releases are protocol compatible. This means any major or minor changes to the server imply a new protocol across the cluster. The fdb_c library uses this same protocol to talk to the cluster, so it impacts all clients, as well. Please provide a link to the "backwards compatible" client library documentation. In my experience, you need the same version of the server and fdb_c library. I don’t have a test stand right now to confirm this.
TLDR you are right :-) From the docs https://apple.github.io/foundationdb/api-general.html ~~~ In general, a given client will support both its corresponding API version and earlier API versions. A client selects the API version it will use by explicitly specifying the version number upon initialization. The purpose of this design is to allow the server, client libraries, or bindings to be upgraded without having to modify application code. You can therefore upgrade to more recent packages (and so receive their various improvements) without having to change application code. ~~~ I've mis-interpreted this as being able to upgrade the server & client without downtime. I had a longer read of docs, and tried this in practice. I was confused by the difference between client requested API version, and the wire binary/network protocol, these things are not the same. This means that any upgrade requires: - installing both libfdb_c.so versions on *all* clients and restarting them before upgrading servers - bouncing all servers within a very short timeframe once client updates are done IMO the ports tree doesn't need to cater for this scenario, users will need to plan their upgrades themselves and manage their server versions very very closely. The alternative is that the ports tree accumulates multiple FDB versions, for every single MAJOR.MINOR release, which is a tremendous waste of build time and space in the cluster. Keeping fdb versions to "latest" version in main repo seems a reasonable compromise. What do you think?
the alternative is to have each version in ports install a different-versioned libfdb_c.so such that the clients can be installed side-by-side. Is that an option?
(In reply to Dave Cottlehuber from comment #13) > included my LLVM version constraint for 15 builds. Hello Dave, please tell me more about this. I also added portclippy changes to the gitlab branch. https://gitlab.com/dwagin/freebsd-ports/-/commits/dwagin-foundationdb?ref_type=heads
We discovered that FDB won't build on 15.0-CURRENT because the newer LLVM produces errors. While that bug is addressed, we can work around it in ports by requiring LLVM=<17: USES= cmake compiler:c++20-lang llvm:max=16 mono:build python:build \ shebangfix ssl On 13 & 14, builtin llvm is always max 16, so this only alters behaviour on CURRENT.
Created attachment 249612 [details] foundationdb.diff * max llvm 16 * 7.1.59 and 7.3.35 * erlfdb * portclippy
(In reply to Dave Cottlehuber from comment #13) IMHO, the ports tree should contain multiple versions of FDB for each individual MAJOR.MINOR release. All databases in the ports tree do this.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=c7911fcef4f31ad76b2c6369ca03d5a100b236d2 commit c7911fcef4f31ad76b2c6369ca03d5a100b236d2 Author: Dave Cottlehuber <dch@FreeBSD.org> AuthorDate: 2024-04-11 10:04:28 +0000 Commit: Dave Cottlehuber <dch@FreeBSD.org> CommitDate: 2024-04-11 10:04:30 +0000 database/foundationdb: update maintainer PR: 277262 databases/foundationdb/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Created attachment 249961 [details] foundationdb.diff * 7.1.59 and 7.3.37 * rebase