Bug 261523 - Mk/bsd.default-versions.mk: Change default db5 (deprecated and EoL) to db18
Summary: Mk/bsd.default-versions.mk: Change default db5 (deprecated and EoL) to db18
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Port Management Team
Keywords: needs-qa
Depends on: 261638 262135 262137 262138 262139 262210 263010 263012 263013
Blocks: 263123
  Show dependency treegraph
Reported: 2022-01-27 19:30 UTC by Rafael Grether
Modified: 2023-08-27 18:31 UTC (History)
38 users (show)

See Also:
lwhsu: exp-run?

Patch - Default BDB 5 to 18 (1019 bytes, patch)
2022-01-27 19:30 UTC, Rafael Grether
no flags Details | Diff
Updated patch file (2.21 KB, patch)
2022-01-28 08:18 UTC, Yasuhiro Kimura
no flags Details | Diff
Patch file (7.17 KB, patch)
2022-02-05 21:16 UTC, Yasuhiro Kimura
no flags Details | Diff
git patch to permit build apr-1 with Oracle Berkeley DB 18 (2.22 KB, patch)
2022-02-07 18:32 UTC, Matthias Andree
no flags Details | Diff
Updated patch file (40.70 KB, patch)
2022-07-21 16:13 UTC, Yasuhiro Kimura
no flags Details | Diff
Updated patch file (40.70 KB, patch)
2022-07-22 10:55 UTC, Yasuhiro Kimura
no flags Details | Diff
Updated patch file (36.79 KB, patch)
2022-09-11 11:53 UTC, Yasuhiro Kimura
no flags Details | Diff
Updated patch file (40.38 KB, patch)
2022-10-13 05:28 UTC, Yasuhiro Kimura
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Rafael Grether 2022-01-27 19:30:27 UTC
Created attachment 231394 [details]
Patch - Default BDB 5 to 18

First of all, I appologize to open bug on "Individual Ports" component.
I know that "Ports Framework" should be correct, but only "Individual Port(s)" appears for me. Maybe I don't have permission.

So, if you can change the fields, that would be great.

db5 is very outdated (with possible security issues), and since today (2022-01-27), this port was abandoned by maintainer.

The current version db18 is the most up-to-date and is maintained by ports.

db is required as needed (directly or indirectly) by many applications commonly used like Apache Httpd Server and Git-svn, so I proposed this patch for testing.

Please perform an exp-run with this patch applied.

Comment 1 Li-Wen Hsu freebsd_committer 2022-01-27 19:39:13 UTC
databases/db5 is set EXPIRATION_DATE=2022-06-30 so I think it's reasonable to change default.
Comment 2 Yasuhiro Kimura freebsd_committer 2022-01-28 08:18:31 UTC
Created attachment 231406 [details]
Updated patch file

(In reply to Rafael Grether from comment #0)

Remove codes related to WITH_BDB6_PERMITTED from Mk/Uses/bdb.mk.

Unfortunately original patch doesn't work as is expected as currently user needs to set 'WITH_BDB6_PERMITTED=yes' in /etc/make.conf to use db18 as default version of BDB. The variable is introduced for user who doesn't want to accept AGPLv3 when upstream changed license with the release of 6.0. However, after db5 is removed it becomes meaningless as db18 is the only available version of BDB. Moreover it makes it impossible to change default version BDB to 18. So remove codes related to it.
Comment 4 Yasuhiro Kimura freebsd_committer 2022-02-05 21:16:13 UTC
Created attachment 231569 [details]
Patch file

Fix build error of following ports.

* databases/py-berkeleydb
* databases/py-bsddb3
* dns/fastresolve
* editors/poedit

As for other ports listed in comment #3, the source of build error is devel/apr1. Built of it succeeds even if databases/db18 is default version of BDB. But it fails to detect berkeley db 18.x at build time. As a result `apu-1-config` command returns wrong library name of berkeley db. I checked source tree of apr-util 1.6.1 and configure script doesn't seem to support berkeley db 18.x. There is dbm.m4 in source archives and macros to detect berkeley db are defined in it. So in order to support berkeley db 18.x we need to hack it. To do it, however, knowledge about GNU Autotools are required. And unfortunately I don't have it. So we need help of someone who is familiar with it.
Comment 5 Matthias Fechner freebsd_committer 2022-02-07 17:02:08 UTC
postfix 3.6.5 should now also wotk with db18:
Comment 6 Matthias Andree freebsd_committer 2022-02-07 18:32:32 UTC
Created attachment 231617 [details]
git patch to permit build apr-1 with Oracle Berkeley DB 18

This minimal patch for "git am" seems to do the job and add Berkeley DB 18 support at build-time. 

I have not done run-time tests, I know next to nothing about apr and apr-util and I also use nginx. 

Someone knowledgeable about APR and Apache please give this a thorough spin and test.
Comment 7 Dan Langille freebsd_committer 2022-02-15 23:46:50 UTC
I am available for build testing if provided with a list of stuff to build.
Comment 8 Muhammad Moinur Rahman freebsd_committer 2022-02-26 05:41:22 UTC
Consider bug #262136 as done although I have to chase some deps for DEPRECATING db5.
Comment 9 Yasuhiro Kimura freebsd_committer 2022-07-21 16:13:58 UTC
Created attachment 235409 [details]
Updated patch file

* Remove compatiblity hack to upgrade older plussed versions to 5+ aa there isn't such case in ports tree anymore.
* Remove INVALID_BDB_VER and replace it with argument of 'USE=bsb'.
* Stop checking if there is already installed version of berkeley db. While testing migration from db5 to db18, I found following confuging situation.
  (1) Currently default version of bdb is 5. So if a port uses 'USES=bdb', then install packages depends of db5.
  (2) To migrate to db18, add 'DEFAULT_VERSIONS+=bdb=18' to /etc/make.conf.
  (3) Rebuild package installed at (1).
  (4) Now it is expected the package depends on db18. But what really       happens is that it still depends on db5. It happens because when db5 is already installed it is used as default version of bdb even if there is 'DEFAULT_VERSIONS+=bdb=18' in /etc/make.conf. It's against user's expectation. So remove the logic.
* Add entry to UPDATING.
* Bump PORTREVISION of ports that use bdb.
Comment 10 Yasuhiro Kimura freebsd_committer 2022-07-21 16:15:46 UTC
@antoine Would you please try another exp-run with updated patch?
Comment 11 J.R. Oldroyd 2022-07-21 17:57:24 UTC
Comment on attachment 235409 [details]
Updated patch file

The entry in the patch for UPDATING appears to be wrong...

+  by yourself, you can keep db5 as default by adding
+  'DEFAULT_VERSIONS+=bdb=18' in /etc/make.conf.

I suspect that setting bdb=18 will not keep db5 as default.
Comment 12 Yasuhiro Kimura freebsd_committer 2022-07-22 10:55:58 UTC
Created attachment 235422 [details]
Updated patch file

Fix the way to keep db5 as default in UPDATING.
Comment 13 Yasuhiro Kimura freebsd_committer 2022-07-22 10:56:34 UTC
(In reply to J.R. Oldroyd from comment #11)

Fixed. Thanks for pointing out.
Comment 14 Antoine Brodin freebsd_committer 2022-08-12 19:40:41 UTC
Could you refresh the patch?
Comment 15 K J Petrie 2022-08-14 17:06:50 UTC
He's already done:
14 	by yourself, you can keep db5 as default by adding
15 	'DEFAULT_VERSIONS+=bdb=5' in /etc/make.conf.

Which refresh are you requesting?
Comment 16 Dan Langille freebsd_committer 2022-09-01 21:32:48 UTC
This problem went away for me after recent upgrades to:

* 12.3-RELEASE-p7
* 13.1-RELEASE-p2

Months of alert fatigue gone.
Comment 17 Dan Langille freebsd_committer 2022-09-01 21:35:33 UTC
(In reply to Dan Langille from comment #16)

I'm wrong. Sorry, I was looking at the wrong monitoring console.

Comment 18 Yasuhiro Kimura freebsd_committer 2022-09-11 11:53:11 UTC
Created attachment 236499 [details]
Updated patch file

Chase update of ports tree.
Comment 19 Yasuhiro Kimura freebsd_committer 2022-09-11 11:54:35 UTC
(In reply to Antoine Brodin from comment #14)

Sorry I overlooked your reply. Updated patch should be applied to latest ports tree cleanly. So please do exp-run with it.
Comment 20 Dan Langille freebsd_committer 2022-09-11 13:21:59 UTC
(In reply to Yasuhiro Kimura from comment #19)

Am I doing this wrong?

This patch applies cleanly to a recent ports tree.

I built devel/apr1:

[pkg01 dan ~/ports/head] % sudo poudriere bulk -j 131amd64 -p default -z primary -Ci devel/apr1
root@131amd64-default-primary:~ # pkg info -x db

Still db5 here for me

Next, I added this to /usr/local/etc/poudriere.d/make.conf:


root@131amd64-default-primary:~ # pkg info -d apr
Comment 21 Dan Langille freebsd_committer 2022-09-11 13:24:07 UTC
(In reply to Dan Langille from comment #20)
I'm sorry, I build against the wrong tree. I am redoing my tests. Sorry for the noise.
Comment 22 Yasuhiro Kimura freebsd_committer 2022-09-11 13:33:54 UTC
(In reply to Dan Langille from comment #21)

I applied latest patch to ports 5ca22080ae08 and made clean build of devel/apr1 with pouriere and 13.1-RELEASE amd64 jail. Then the package depends on db18 as following.

yasu@rolling-vm-freebsd2[1161]% pkg info -F /usr/local/poudriere/data/packages/131amd64-default/All/apr- -d
Comment 23 Dan Langille freebsd_committer 2022-09-11 14:24:10 UTC
(In reply to Yasuhiro Kimura from comment #22)

Further testing against the correct ports tree results in:

root@131amd64-dvl:~ # pkg info -d apr

Thank you.

Sorry for my errors.
Comment 24 John Hein 2022-09-13 17:34:08 UTC
All these ports that will now use db18 may have to add AGPLv3 to their LICENSE.  Furthermore if a particular port's current license prevents being a derived work from an AGPLv3 license, then it may be incompatible to the extent that the db18-dependent function should be removed. GPLv2, for instance, is compatible with AGPLv3, so you can link an AGPLv3 with a GPLv2 work in that case (making the entire derived work subject to terms of the AGPLv3). But that compatibility with AGPLv3 may not extend to other licenses.  Note that linking with an AGPLv3 licensed library can add a requirement for full source code distribution of the derived work (if the derived work is used publicly).

Here is an example of how one port handled this:


Quoting from that bug:

"bdb18 is licensed with AGPLv3. This forces the derived work (mutt) to be AGPLv3.

mutt is currently licensed with GPLv2 or later. This allows the derived
work to be linked with an AGPLv3 library.  If it were strictly GPLv2
only, then linking with an AGPLv3 library would not be permitted."

As always, license issues can be tricky, but this should definitely be considered / reviewed before throwing a global switch for unsuspecting ports to use db18.
Comment 25 Antoine Brodin freebsd_committer 2022-09-15 09:33:22 UTC
Did someone have a look at some bdb forks with the previous license,  maybe like https://github.com/bloomberg/comdb2/tree/master/berkdb ?
Comment 26 Dan Langille freebsd_committer 2022-09-28 00:11:14 UTC
(In reply to Antoine Brodin from comment #25)

Look at them for what purpose?
Comment 27 Dan Langille freebsd_committer 2022-09-28 00:12:08 UTC
(In reply to Dan Langille from comment #26)
Oh I see you were replying to #24
Comment 28 Yasuhiro Kimura freebsd_committer 2022-10-13 05:28:26 UTC
Created attachment 237252 [details]
Updated patch file

Chase update of ports tree.
Comment 29 Antoine Brodin freebsd_committer 2022-10-13 07:52:21 UTC
I don't think it's a good idea to link by default on AGPLv3 software.
BDB option should be disabled/removed where it no longer makes senses (where more modern database exists and has existed for a long time for instance).
Comment 30 dewayne 2022-10-13 08:53:43 UTC
(In reply to Antoine Brodin from comment #29)
Yes - its going to be interesting for those that use options for apache (via apr) or ISC's bind (via dlz_bdb used by samba).  Is there a possibility of using the FreeBSD's "Berkeley DB 1.85" in place of Oracles db18, part of the standard distribution?
Comment 31 p5B2EA84B3 2023-01-04 14:25:55 UTC
PR 263123 is referenced as blocking, but this is circular. 

I pledge to give the issue db5 -> db18 priority now for ending this tedious drama.

Thank you.
Comment 32 Antoine Brodin freebsd_committer 2023-01-04 14:53:45 UTC
The priority should be to disable by default a dependency on a legacy database that has no future (where appropriate).
Comment 33 Gert Doering 2023-02-07 07:33:16 UTC
Good Morning,

I've bumped into this db5 issue again and again in the last months, and I can see that if I manually install db18 and then(!) build apr1/apache24, both will nicely use db18.  This is great.

But - binary packages are still built against db5, which causes never-ending annoyance because db5 keeps being brought back.

On a FreeBSD14 system, pkg "latest" of today...

$ doas pkg install apache24
New packages to be INSTALLED:
        apache24: 2.4.55
        db5: 5.3.28_9
        gdbm: 1.23
        jansson: 2.14

is there a way to make db5 go away for binpkg builds?

Comment 34 Oliver Heesakkers 2023-02-07 11:52:10 UTC
(In reply to Gert Doering from comment #33)
As detailed in bug 261525: apr runs fine without any BDB, using native NDBM.

As far as I can tell apr using BDB is mostly for historical reasons.
Comment 35 K J Petrie 2023-02-07 12:03:56 UTC
(In reply to Oliver Heesakkers from comment #34)
As I understand it, no one disputes that. The problem is for those who use the official package repository, the binary packages are still built to be dependent on this unmaintained antiquated package, so they are forced to install something the system will warn them is a vulnerability.
Comment 36 Dan Langille freebsd_committer 2023-02-07 14:10:52 UTC
(In reply to K J Petrie from comment #35)
Agreed. I have been able to rid myself of nagging messages *only* because I build my own packages. Not everyone can do that.

Alert fatigue is real. People start ignoring alerts under such circumstances.
Comment 37 ctipper 2023-02-09 17:13:02 UTC
(In reply to Antoine Brodin from comment #32)

Questioning inclusion of db18 in ports tree, it is pulling from a github repo not affiliated with Oracle and given above licensing considerations it seems logical to remove it and force maintainers to adopt their own solutions.
Comment 38 commit-hook freebsd_committer 2023-04-01 15:46:13 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=f473a726dba249ec936d340b8b1167ce5996bd30

commit f473a726dba249ec936d340b8b1167ce5996bd30
Author:     Bernard Spil <brnrd@FreeBSD.org>
AuthorDate: 2023-04-01 15:42:03 +0000
Commit:     Bernard Spil <brnrd@FreeBSD.org>
CommitDate: 2023-04-01 15:42:03 +0000

    devel/apr1: Update to 1.7.3

     * Update APR-util to 1.6.3
     * Fix build with MySQL 8 [2]
     * databases/db5 is depreacated since 2022-06-30 [3]
     * Switch default BDB to 18

    PR:             269857, 230538 [2], 261523 [3]
    Submitted by:   ngie
    With hat:       apache

 devel/apr1/Makefile                                |  15 ++-
 devel/apr1/distinfo                                |  10 +-
 devel/apr1/files/patch-PR59332 (new)               |  18 +++
 devel/apr1/files/patch-PR61517 (gone)              | 126 ---------------------
 ...r-1.7.0_configure => patch-apr-1.7.3_configure} |   4 +-
 ...kqueue.c => patch-apr-1.7.3_poll_unix_kqueue.c} |  14 +--
 devel/apr1/files/patch-bdb18                       |   4 +-
 7 files changed, 43 insertions(+), 148 deletions(-)
Comment 39 John Hein 2023-04-01 18:27:38 UTC
(In reply to commit-hook from comment #38)
Linking with db18 means the devel/apr1 port should be subject to AGPLv3.

See also comment 24.
Comment 40 Martin Birgmeier 2023-04-02 08:48:50 UTC
(In reply to commit-hook from comment #38)

"Switch default BDB to 18": Is this still missing from Mk/bsd.default-versions.mk?
Comment 41 John Hein 2023-04-02 16:16:00 UTC
(In reply to Martin Birgmeier from comment #40)
I think the 'switch default' comment is just applicable to the specific port (devel/apr1) in the commit.
Comment 42 John Hein 2023-04-02 16:36:45 UTC
(In reply to John Hein from comment #39)
The AGPLv3 license issue (and another breakage from https://cgit.freebsd.org/ports/commit/?id=f473a726dba249ec936d340b8b1167ce5996bd30) is addressed by the patch in bug 269857, comment 5 - attachment 241267 [details]
Comment 43 Martin Birgmeier 2023-04-03 07:04:38 UTC
(In reply to John Hein from comment #41)

I am wondering what devel/subversion is using because it depends on both apr and BDB. ldd shows the following after the upgrade of apr; does apr indeed depend on BDB?

[0]# ldd /usr/local/bin/svn
        libsvn_client-1.so.0 => /usr/local/lib/libsvn_client-1.so.0 (0x8002ad000)
        libsvn_wc-1.so.0 => /usr/local/lib/libsvn_wc-1.so.0 (0x80033f000)
        libsvn_ra-1.so.0 => /usr/local/lib/libsvn_ra-1.so.0 (0x8003f0000)
        libsvn_diff-1.so.0 => /usr/local/lib/libsvn_diff-1.so.0 (0x800400000)
        libsvn_ra_local-1.so.0 => /usr/local/lib/libsvn_ra_local-1.so.0 (0x80041b000)
        libsvn_repos-1.so.0 => /usr/local/lib/libsvn_repos-1.so.0 (0x800427000)
        libsvn_fs-1.so.0 => /usr/local/lib/libsvn_fs-1.so.0 (0x800468000)
        libsvn_fs_fs-1.so.0 => /usr/local/lib/libsvn_fs_fs-1.so.0 (0x800478000)
        libsvn_fs_x-1.so.0 => /usr/local/lib/libsvn_fs_x-1.so.0 (0x8004cf000)
        libsvn_fs_base-1.so.0 => /usr/local/lib/libsvn_fs_base-1.so.0 (0x800526000)
        libsvn_fs_util-1.so.0 => /usr/local/lib/libsvn_fs_util-1.so.0 (0x800559000)
        libsvn_ra_svn-1.so.0 => /usr/local/lib/libsvn_ra_svn-1.so.0 (0x80055e000)
        libsvn_ra_serf-1.so.0 => /usr/local/lib/libsvn_ra_serf-1.so.0 (0x800587000)
        libsvn_delta-1.so.0 => /usr/local/lib/libsvn_delta-1.so.0 (0x8005bd000)
        libsvn_subr-1.so.0 => /usr/local/lib/libsvn_subr-1.so.0 (0x8005df000)
        libdb-5.3.so.0 => /usr/local/lib/libdb-5.3.so.0 (0x800677000)
        libserf-1.so.1 => /usr/local/lib/libserf-1.so.1 (0x80081d000)
        libaprutil-1.so.0 => /usr/local/lib/libaprutil-1.so.0 (0x80083b000)
        libapr-1.so.0 => /usr/local/lib/libapr-1.so.0 (0x80086b000)
        libexpat.so.1 => /usr/local/lib/libexpat.so.1 (0x8008ac000)
        libz.so.6 => /lib/libz.so.6 (0x8008de000)
        libsqlite3.so.0 => /usr/local/lib/libsqlite3.so.0 (0x8008fb000)
        libmagic.so.4 => /usr/lib/libmagic.so.4 (0x800a95000)
        liblz4.so.1 => /usr/local/lib/liblz4.so.1 (0x800ac0000)
        libutf8proc.so.2 => /usr/local/lib/libutf8proc.so.2 (0x800af0000)
        libintl.so.8 => /usr/local/lib/libintl.so.8 (0x800b48000)
        libc.so.7 => /lib/libc.so.7 (0x800b57000)
        libthr.so.3 => /lib/libthr.so.3 (0x800f4b000)
        libssl.so.111 => /usr/lib/libssl.so.111 (0x800f79000)
        libcrypto.so.111 => /lib/libcrypto.so.111 (0x80101d000)
        libcrypt.so.5 => /lib/libcrypt.so.5 (0x801314000)
        libgdbm.so.6 => /usr/local/lib/libgdbm.so.6 (0x801335000)
        libm.so.5 => /lib/libm.so.5 (0x801348000)
[0]# ldd /usr/local/lib/libapr-1.so.0 /usr/local/lib/libaprutil-1.so.0
        libcrypt.so.5 => /lib/libcrypt.so.5 (0x8006d0000)
        libthr.so.3 => /lib/libthr.so.3 (0x8006f1000)
        libc.so.7 => /lib/libc.so.7 (0x80024f000)
        libexpat.so.1 => /usr/local/lib/libexpat.so.1 (0x8006c6000)
        libapr-1.so.0 => /usr/local/lib/libapr-1.so.0 (0x8006f8000)
        libcrypt.so.5 => /lib/libcrypt.so.5 (0x800739000)
        libthr.so.3 => /lib/libthr.so.3 (0x80075a000)
        libc.so.7 => /lib/libc.so.7 (0x80024f000)
        libm.so.5 => /lib/libm.so.5 (0x800788000)

-- Martin
Comment 44 K J Petrie 2023-04-03 09:01:33 UTC
The new port doesn't build on Poudriere here. I get:

~]$ tail -n 15 /usr/local/poudriere/data/logs/bulk/13amd64-local-kjpservers/latest/logs/errors/apr- 
=======================<phase: package        >============================
===>  Building package for apr-
pkg-static: Unable to access file /wrkdirs/usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbm_gdbm-1.so:No such file or directory
pkg-static: Unable to access file /wrkdirs/usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbm_gdbm.a:No such file or directory
pkg-static: Unable to access file /wrkdirs/usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbm_gdbm.so:No such file or directory
*** Error code 1

make: stopped in /usr/ports/devel/apr1
=>> Cleaning up wrkdir
===>  Cleaning for apr-
build of devel/apr1 | apr- ended at Mon Apr  3 08:49:27 BST 2023
build time: 00:00:30
!!! build failure encountered !!!

Is it missing a symbolic link apr-util-1 > apr-util-1.6.3?
Comment 45 void 2023-04-03 09:07:05 UTC
(In reply to K J Petrie from comment #44)

To get around that, I had to disable GDBM in OPTIONS.
Comment 46 K J Petrie 2023-04-03 09:21:10 UTC
(In reply to void from comment #45)
Yes, I have just tried it with DBM re-enabled and it builds - so it won't build a package with GDBM if you don't want the bloat of db18.
Comment 47 John Hein 2023-04-03 15:00:08 UTC
(In reply to K J Petrie from comment #46)
The gdbm build failure was a stray backslash in devel/apr1 (my fault).
See bug 269857, comment 13
Comment 48 John Hein 2023-07-07 21:20:13 UTC
(In reply to dewayne from comment #30)
See bug 269857, comment 17 (and review D40366).  Using bdb from base certainly seems like a reasonable option for devel/apr1 - and probably a better default than the AGPLv3 flavor of Berkeley DB.
Comment 49 Will 2023-08-27 18:31:44 UTC
Just want to reiterate that it would be very much appreciated if a decision could be made and something could be updated so that those using the binary package of apache24 can stop getting security warnings about db5 being EOL.