Created attachment 231394 [details] Patch - Default BDB 5 to 18 First of all, I appologize to open bug on "Individual Ports" component. I know that "Ports Framework" should be correct, but only "Individual Port(s)" appears for me. Maybe I don't have permission. So, if you can change the fields, that would be great. db5 is very outdated (with possible security issues), and since today (2022-01-27), this port was abandoned by maintainer. The current version db18 is the most up-to-date and is maintained by ports. db is required as needed (directly or indirectly) by many applications commonly used like Apache Httpd Server and Git-svn, so I proposed this patch for testing. Please perform an exp-run with this patch applied. Thanks
databases/db5 is set EXPIRATION_DATE=2022-06-30 so I think it's reasonable to change default.
Created attachment 231406 [details] Updated patch file (In reply to Rafael Grether from comment #0) Remove codes related to WITH_BDB6_PERMITTED from Mk/Uses/bdb.mk. Unfortunately original patch doesn't work as is expected as currently user needs to set 'WITH_BDB6_PERMITTED=yes' in /etc/make.conf to use db18 as default version of BDB. The variable is introduced for user who doesn't want to accept AGPLv3 when upstream changed license with the release of 6.0. However, after db5 is removed it becomes meaningless as db18 is the only available version of BDB. Moreover it makes it impossible to change default version BDB to 18. So remove codes related to it.
Some new failure logs: http://package22.nyi.freebsd.org/data/130amd64-default-foo/2022-02-04_18h50m50s/logs/py38-berkeleydb-18.1.4.log http://package22.nyi.freebsd.org/data/130amd64-default-foo/2022-02-04_18h50m50s/logs/py38-bsddb3-6.2.9.log http://package22.nyi.freebsd.org/data/130amd64-default-foo/2022-02-04_18h50m50s/logs/kdesvn-2.1.0_2.log http://package22.nyi.freebsd.org/data/130amd64-default-foo/2022-02-04_18h50m50s/logs/log4cxx-0.11.0.log http://package22.nyi.freebsd.org/data/130amd64-default-foo/2022-02-04_18h50m50s/logs/rsvndump-0.6_2.log http://package22.nyi.freebsd.org/data/130amd64-default-foo/2022-02-04_18h50m50s/logs/fastresolve-2.10_6.log http://package22.nyi.freebsd.org/data/130amd64-default-foo/2022-02-04_18h50m50s/logs/poedit-1.8.4_35.log http://package22.nyi.freebsd.org/data/130amd64-default-foo/2022-02-04_18h50m50s/logs/ftwin-0.8.10.log http://package22.nyi.freebsd.org/data/130amd64-default-foo/2022-02-04_18h50m50s/logs/httest-2.4.24.log http://package22.nyi.freebsd.org/data/130amd64-default-foo/2022-02-04_18h50m50s/logs/mod_webauth-4.7.0_2.log
Created attachment 231569 [details] Patch file Fix build error of following ports. * databases/py-berkeleydb * databases/py-bsddb3 * dns/fastresolve * editors/poedit As for other ports listed in comment #3, the source of build error is devel/apr1. Built of it succeeds even if databases/db18 is default version of BDB. But it fails to detect berkeley db 18.x at build time. As a result `apu-1-config` command returns wrong library name of berkeley db. I checked source tree of apr-util 1.6.1 and configure script doesn't seem to support berkeley db 18.x. There is dbm.m4 in source archives and macros to detect berkeley db are defined in it. So in order to support berkeley db 18.x we need to hack it. To do it, however, knowledge about GNU Autotools are required. And unfortunately I don't have it. So we need help of someone who is familiar with it.
postfix 3.6.5 should now also wotk with db18: http://www.postfix.org/announcements/postfix-3.6.5.html
Created attachment 231617 [details] git patch to permit build apr-1 with Oracle Berkeley DB 18 This minimal patch for "git am" seems to do the job and add Berkeley DB 18 support at build-time. I have not done run-time tests, I know next to nothing about apr and apr-util and I also use nginx. Someone knowledgeable about APR and Apache please give this a thorough spin and test.
I am available for build testing if provided with a list of stuff to build.
Consider bug #262136 as done although I have to chase some deps for DEPRECATING db5.
Created attachment 235409 [details] Updated patch file * Remove compatiblity hack to upgrade older plussed versions to 5+ aa there isn't such case in ports tree anymore. * Remove INVALID_BDB_VER and replace it with argument of 'USE=bsb'. * Stop checking if there is already installed version of berkeley db. While testing migration from db5 to db18, I found following confuging situation. (1) Currently default version of bdb is 5. So if a port uses 'USES=bdb', then install packages depends of db5. (2) To migrate to db18, add 'DEFAULT_VERSIONS+=bdb=18' to /etc/make.conf. (3) Rebuild package installed at (1). (4) Now it is expected the package depends on db18. But what really happens is that it still depends on db5. It happens because when db5 is already installed it is used as default version of bdb even if there is 'DEFAULT_VERSIONS+=bdb=18' in /etc/make.conf. It's against user's expectation. So remove the logic. * Add entry to UPDATING. * Bump PORTREVISION of ports that use bdb.
@antoine Would you please try another exp-run with updated patch?
Comment on attachment 235409 [details] Updated patch file The entry in the patch for UPDATING appears to be wrong... + by yourself, you can keep db5 as default by adding + 'DEFAULT_VERSIONS+=bdb=18' in /etc/make.conf. I suspect that setting bdb=18 will not keep db5 as default.
Created attachment 235422 [details] Updated patch file Fix the way to keep db5 as default in UPDATING.
(In reply to J.R. Oldroyd from comment #11) Fixed. Thanks for pointing out.
Hello, Could you refresh the patch? Thanks.
He's already done: 14 by yourself, you can keep db5 as default by adding 15 'DEFAULT_VERSIONS+=bdb=5' in /etc/make.conf. Which refresh are you requesting?
This problem went away for me after recent upgrades to: * 12.3-RELEASE-p7 * 13.1-RELEASE-p2 Months of alert fatigue gone.
(In reply to Dan Langille from comment #16) I'm wrong. Sorry, I was looking at the wrong monitoring console. *sigh*
Created attachment 236499 [details] Updated patch file Chase update of ports tree.
(In reply to Antoine Brodin from comment #14) Sorry I overlooked your reply. Updated patch should be applied to latest ports tree cleanly. So please do exp-run with it.
(In reply to Yasuhiro Kimura from comment #19) Am I doing this wrong? This patch applies cleanly to a recent ports tree. I built devel/apr1: [pkg01 dan ~/ports/head] % sudo poudriere bulk -j 131amd64 -p default -z primary -Ci devel/apr1 ... root@131amd64-default-primary:~ # pkg info -x db db5-5.3.28_8 gdbm-1.23 Still db5 here for me Next, I added this to /usr/local/etc/poudriere.d/make.conf: DEFAULT_VERSIONS+=bdb=18 root@131amd64-default-primary:~ # pkg info -d apr apr-1.7.0.1.6.1_2: expat-2.4.8 gdbm-1.23 db5-5.3.28_8
(In reply to Dan Langille from comment #20) I'm sorry, I build against the wrong tree. I am redoing my tests. Sorry for the noise.
(In reply to Dan Langille from comment #21) I applied latest patch to ports 5ca22080ae08 and made clean build of devel/apr1 with pouriere and 13.1-RELEASE amd64 jail. Then the package depends on db18 as following. yasu@rolling-vm-freebsd2[1161]% pkg info -F /usr/local/poudriere/data/packages/131amd64-default/All/apr-1.7.0.1.6.1_3.pkg -d apr-1.7.0.1.6.1_3: db18-18.1.40 expat-2.4.8 gdbm-1.23 yasu@rolling-vm-freebsd2[1162]%
(In reply to Yasuhiro Kimura from comment #22) Further testing against the correct ports tree results in: root@131amd64-dvl:~ # pkg info -d apr apr-1.7.0.1.6.1_3: expat-2.4.8 gdbm-1.23 db18-18.1.40 Thank you. Sorry for my errors.
All these ports that will now use db18 may have to add AGPLv3 to their LICENSE. Furthermore if a particular port's current license prevents being a derived work from an AGPLv3 license, then it may be incompatible to the extent that the db18-dependent function should be removed. GPLv2, for instance, is compatible with AGPLv3, so you can link an AGPLv3 with a GPLv2 work in that case (making the entire derived work subject to terms of the AGPLv3). But that compatibility with AGPLv3 may not extend to other licenses. Note that linking with an AGPLv3 licensed library can add a requirement for full source code distribution of the derived work (if the derived work is used publicly). Here is an example of how one port handled this: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=264024 Quoting from that bug: "bdb18 is licensed with AGPLv3. This forces the derived work (mutt) to be AGPLv3. mutt is currently licensed with GPLv2 or later. This allows the derived work to be linked with an AGPLv3 library. If it were strictly GPLv2 only, then linking with an AGPLv3 library would not be permitted." As always, license issues can be tricky, but this should definitely be considered / reviewed before throwing a global switch for unsuspecting ports to use db18.
Did someone have a look at some bdb forks with the previous license, maybe like https://github.com/bloomberg/comdb2/tree/master/berkdb ?
(In reply to Antoine Brodin from comment #25) Look at them for what purpose?
(In reply to Dan Langille from comment #26) Oh I see you were replying to #24
Created attachment 237252 [details] Updated patch file Chase update of ports tree.
I don't think it's a good idea to link by default on AGPLv3 software. BDB option should be disabled/removed where it no longer makes senses (where more modern database exists and has existed for a long time for instance).
(In reply to Antoine Brodin from comment #29) Yes - its going to be interesting for those that use options for apache (via apr) or ISC's bind (via dlz_bdb used by samba). Is there a possibility of using the FreeBSD's "Berkeley DB 1.85" in place of Oracles db18, part of the standard distribution?
PR 263123 is referenced as blocking, but this is circular. I pledge to give the issue db5 -> db18 priority now for ending this tedious drama. Thank you.
The priority should be to disable by default a dependency on a legacy database that has no future (where appropriate).
Good Morning, I've bumped into this db5 issue again and again in the last months, and I can see that if I manually install db18 and then(!) build apr1/apache24, both will nicely use db18. This is great. But - binary packages are still built against db5, which causes never-ending annoyance because db5 keeps being brought back. On a FreeBSD14 system, pkg "latest" of today... $ doas pkg install apache24 ... New packages to be INSTALLED: apache24: 2.4.55 apr: 1.7.0.1.6.1_2 db5: 5.3.28_9 gdbm: 1.23 jansson: 2.14 is there a way to make db5 go away for binpkg builds? thanks.
(In reply to Gert Doering from comment #33) As detailed in bug 261525: apr runs fine without any BDB, using native NDBM. As far as I can tell apr using BDB is mostly for historical reasons.
(In reply to Oliver Heesakkers from comment #34) As I understand it, no one disputes that. The problem is for those who use the official package repository, the binary packages are still built to be dependent on this unmaintained antiquated package, so they are forced to install something the system will warn them is a vulnerability.
(In reply to K J Petrie from comment #35) Agreed. I have been able to rid myself of nagging messages *only* because I build my own packages. Not everyone can do that. Alert fatigue is real. People start ignoring alerts under such circumstances.
(In reply to Antoine Brodin from comment #32) Questioning inclusion of db18 in ports tree, it is pulling from a github repo not affiliated with Oracle and given above licensing considerations it seems logical to remove it and force maintainers to adopt their own solutions.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=f473a726dba249ec936d340b8b1167ce5996bd30 commit f473a726dba249ec936d340b8b1167ce5996bd30 Author: Bernard Spil <brnrd@FreeBSD.org> AuthorDate: 2023-04-01 15:42:03 +0000 Commit: Bernard Spil <brnrd@FreeBSD.org> CommitDate: 2023-04-01 15:42:03 +0000 devel/apr1: Update to 1.7.3 * Update APR-util to 1.6.3 * Fix build with MySQL 8 [2] * databases/db5 is depreacated since 2022-06-30 [3] * Switch default BDB to 18 PR: 269857, 230538 [2], 261523 [3] Submitted by: ngie With hat: apache devel/apr1/Makefile | 15 ++- devel/apr1/distinfo | 10 +- devel/apr1/files/patch-PR59332 (new) | 18 +++ devel/apr1/files/patch-PR61517 (gone) | 126 --------------------- ...r-1.7.0_configure => patch-apr-1.7.3_configure} | 4 +- ...kqueue.c => patch-apr-1.7.3_poll_unix_kqueue.c} | 14 +-- devel/apr1/files/patch-bdb18 | 4 +- 7 files changed, 43 insertions(+), 148 deletions(-)
(In reply to commit-hook from comment #38) Linking with db18 means the devel/apr1 port should be subject to AGPLv3. See also comment 24.
(In reply to commit-hook from comment #38) "Switch default BDB to 18": Is this still missing from Mk/bsd.default-versions.mk?
(In reply to Martin Birgmeier from comment #40) I think the 'switch default' comment is just applicable to the specific port (devel/apr1) in the commit.
(In reply to John Hein from comment #39) The AGPLv3 license issue (and another breakage from https://cgit.freebsd.org/ports/commit/?id=f473a726dba249ec936d340b8b1167ce5996bd30) is addressed by the patch in bug 269857, comment 5 - attachment 241267 [details]
(In reply to John Hein from comment #41) I am wondering what devel/subversion is using because it depends on both apr and BDB. ldd shows the following after the upgrade of apr; does apr indeed depend on BDB? [0]# ldd /usr/local/bin/svn /usr/local/bin/svn: libsvn_client-1.so.0 => /usr/local/lib/libsvn_client-1.so.0 (0x8002ad000) libsvn_wc-1.so.0 => /usr/local/lib/libsvn_wc-1.so.0 (0x80033f000) libsvn_ra-1.so.0 => /usr/local/lib/libsvn_ra-1.so.0 (0x8003f0000) libsvn_diff-1.so.0 => /usr/local/lib/libsvn_diff-1.so.0 (0x800400000) libsvn_ra_local-1.so.0 => /usr/local/lib/libsvn_ra_local-1.so.0 (0x80041b000) libsvn_repos-1.so.0 => /usr/local/lib/libsvn_repos-1.so.0 (0x800427000) libsvn_fs-1.so.0 => /usr/local/lib/libsvn_fs-1.so.0 (0x800468000) libsvn_fs_fs-1.so.0 => /usr/local/lib/libsvn_fs_fs-1.so.0 (0x800478000) libsvn_fs_x-1.so.0 => /usr/local/lib/libsvn_fs_x-1.so.0 (0x8004cf000) libsvn_fs_base-1.so.0 => /usr/local/lib/libsvn_fs_base-1.so.0 (0x800526000) libsvn_fs_util-1.so.0 => /usr/local/lib/libsvn_fs_util-1.so.0 (0x800559000) libsvn_ra_svn-1.so.0 => /usr/local/lib/libsvn_ra_svn-1.so.0 (0x80055e000) libsvn_ra_serf-1.so.0 => /usr/local/lib/libsvn_ra_serf-1.so.0 (0x800587000) libsvn_delta-1.so.0 => /usr/local/lib/libsvn_delta-1.so.0 (0x8005bd000) libsvn_subr-1.so.0 => /usr/local/lib/libsvn_subr-1.so.0 (0x8005df000) libdb-5.3.so.0 => /usr/local/lib/libdb-5.3.so.0 (0x800677000) libserf-1.so.1 => /usr/local/lib/libserf-1.so.1 (0x80081d000) libaprutil-1.so.0 => /usr/local/lib/libaprutil-1.so.0 (0x80083b000) libapr-1.so.0 => /usr/local/lib/libapr-1.so.0 (0x80086b000) libexpat.so.1 => /usr/local/lib/libexpat.so.1 (0x8008ac000) libz.so.6 => /lib/libz.so.6 (0x8008de000) libsqlite3.so.0 => /usr/local/lib/libsqlite3.so.0 (0x8008fb000) libmagic.so.4 => /usr/lib/libmagic.so.4 (0x800a95000) liblz4.so.1 => /usr/local/lib/liblz4.so.1 (0x800ac0000) libutf8proc.so.2 => /usr/local/lib/libutf8proc.so.2 (0x800af0000) libintl.so.8 => /usr/local/lib/libintl.so.8 (0x800b48000) libc.so.7 => /lib/libc.so.7 (0x800b57000) libthr.so.3 => /lib/libthr.so.3 (0x800f4b000) libssl.so.111 => /usr/lib/libssl.so.111 (0x800f79000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x80101d000) libcrypt.so.5 => /lib/libcrypt.so.5 (0x801314000) libgdbm.so.6 => /usr/local/lib/libgdbm.so.6 (0x801335000) libm.so.5 => /lib/libm.so.5 (0x801348000) [0]# ldd /usr/local/lib/libapr-1.so.0 /usr/local/lib/libaprutil-1.so.0 /usr/local/lib/libapr-1.so.0: libcrypt.so.5 => /lib/libcrypt.so.5 (0x8006d0000) libthr.so.3 => /lib/libthr.so.3 (0x8006f1000) libc.so.7 => /lib/libc.so.7 (0x80024f000) /usr/local/lib/libaprutil-1.so.0: libexpat.so.1 => /usr/local/lib/libexpat.so.1 (0x8006c6000) libapr-1.so.0 => /usr/local/lib/libapr-1.so.0 (0x8006f8000) libcrypt.so.5 => /lib/libcrypt.so.5 (0x800739000) libthr.so.3 => /lib/libthr.so.3 (0x80075a000) libc.so.7 => /lib/libc.so.7 (0x80024f000) libm.so.5 => /lib/libm.so.5 (0x800788000) [0]# -- Martin
The new port doesn't build on Poudriere here. I get: ~]$ tail -n 15 /usr/local/poudriere/data/logs/bulk/13amd64-local-kjpservers/latest/logs/errors/apr-1.7.3.1.6.3_1.log =========================================================================== =======================<phase: package >============================ ===> Building package for apr-1.7.3.1.6.3_1 pkg-static: Unable to access file /wrkdirs/usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbm_gdbm-1.so:No such file or directory pkg-static: Unable to access file /wrkdirs/usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbm_gdbm.a:No such file or directory pkg-static: Unable to access file /wrkdirs/usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbm_gdbm.so:No such file or directory *** Error code 1 Stop. make: stopped in /usr/ports/devel/apr1 =>> Cleaning up wrkdir ===> Cleaning for apr-1.7.3.1.6.3_1 build of devel/apr1 | apr-1.7.3.1.6.3_1 ended at Mon Apr 3 08:49:27 BST 2023 build time: 00:00:30 !!! build failure encountered !!! Is it missing a symbolic link apr-util-1 > apr-util-1.6.3?
(In reply to K J Petrie from comment #44) To get around that, I had to disable GDBM in OPTIONS.
(In reply to void from comment #45) Yes, I have just tried it with DBM re-enabled and it builds - so it won't build a package with GDBM if you don't want the bloat of db18.
(In reply to K J Petrie from comment #46) The gdbm build failure was a stray backslash in devel/apr1 (my fault). See bug 269857, comment 13
(In reply to dewayne from comment #30) See bug 269857, comment 17 (and review D40366). Using bdb from base certainly seems like a reasonable option for devel/apr1 - and probably a better default than the AGPLv3 flavor of Berkeley DB.
Just want to reiterate that it would be very much appreciated if a decision could be made and something could be updated so that those using the binary package of apache24 can stop getting security warnings about db5 being EOL.