Bug 296519 - mail/bogofilter: build broken with databases/lmdb >= 1.0.0
Summary: mail/bogofilter: build broken with databases/lmdb >= 1.0.0
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Matthias Andree
URL:
Keywords:
Depends on: 296530
Blocks:
  Show dependency treegraph
 
Reported: 2026-07-04 17:13 UTC by Craig Leres
Modified: 2026-07-09 19:28 UTC (History)
6 users (show)

See Also:
mandree: maintainer-feedback+
mandree: merge-quarterly?


Attachments
Proposed patch (2.15 KB, patch)
2026-07-05 06:47 UTC, Xin LI
mandree: maintainer-approval-
Details | Diff
[PATCH] mail/bogofilter*: Prepare LMDB 1.0 transition; add bogofilter-lmdb1 port for half a year (7.72 KB, patch)
2026-07-06 21:13 UTC, Matthias Andree
mandree: maintainer-approval-
Details | Diff
[PATCH v2] mail/bogofilter*: Prepare LMDB 1.0 transition; add bogofilter-lmdb1 port for half a year (10.36 KB, patch)
2026-07-07 21:26 UTC, Matthias Andree
mandree: maintainer-approval+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Craig Leres freebsd_committer freebsd_triage 2026-07-04 17:13:30 UTC
bogofilter stopped building for me after 0cea189105efe, reverting databases/lmdb to 0.9.35 fixes it. portsfallout.com see this as well.

    https://portsfallout.com/fallout/1332871/
Comment 1 Xin LI freebsd_committer freebsd_triage 2026-07-05 06:47:38 UTC
Created attachment 272506 [details]
Proposed patch

Looks like mandree@ already have fixed this upstream.  Could you please try this patch?
Comment 2 Matthias Andree freebsd_committer freebsd_triage 2026-07-05 10:05:48 UTC
leres@, delphij@ thanks for the report and linking to my upstream patch.

delphij@, thanks for maintaining LMDB, but please revert the change and plan the 0.9 -> 1.0 transition properly, I'll file a new PR for that.  For a transitional period of 3 ... 6 months, we'll need both versions in the tree.  I'll tell you why:

IMPORTANT: The LMDB update to 1.0 "breaks" users' on-disk databases - not permanently - but LMDB 1.0 cannot read/migrate 0.9 databases, or vice versa!  The end user will perceive this as loss of the entire database.

SO: DO NOT COMMIT THIS PATCH, YET!  (https://bugs.freebsd.org/bugzilla/attachment.cgi?id=272506)  We're lucky no FreeBSD bogofilter package currently uses LMDB 1.0 due to enforced self-tests, so no users can break their database by just "pkg upgrade".  Arch Linux has a bigger effort getting this done.

Users need to manually identify and enumerate ALL their LMDB databases, dump them, then update, and then reimport.

IOW, we'll need to have TWO versions of LMDB, and TWO versions of bogofilter and possibly other packages using LMDB, so that users have a clear migration path, and applications such as mail/postfix have time to catch up with LMDB 1.0.

I must admit the LMDB 1.0.0 update took me by surprise, and I saw first reports on the Postfix mailing list some three days ago where people were proposing rather complex patches for Postfix.  People are struggling to get clear information on the LMDB 1.0 update per se, and https://git.openldap.org/openldap/openldap/-/blob/mdb.RE/1.0/libraries/liblmdb/upgrading.doc?ref_type=heads (which may be very slow) or its mirror at https://github.com/LMDB/lmdb/blob/mdb.master3/libraries/liblmdb/upgrading.doc is possibly a good starting point.

The plan for the end user is:

0. Check with upstream if $APPLICATION has a release for LMDB 1.0 at all, or needs updating.  (For bogofilter, no released version supports LMDB 1.0 yet. 1.3.0.rc1 is broken as reported.)
1. With $APPLICATION_WITH_LMDB_0.9, dump/export ALL your databases
2. Update to LMDB 1.0
3. With $APPLICATION_WITH_LMDB_1.0, load/import ALL databases from 1. again

I'll file a separate PR against lmdb for a proper coordination, and will reject the patch for now, although it is mine and technically correct.
Comment 3 Matthias Andree freebsd_committer freebsd_triage 2026-07-05 10:07:34 UTC
Comment on attachment 272506 [details]
Proposed patch

While the patch is technically correct for bogofilter in itself, we need to add a migration path for users first because of the on-disk database change.  So I'll temporarily reject this patch to buy us time for getting migration bits in place. See my earlier comment for a thorough explanation.
Comment 4 Craig Leres freebsd_committer freebsd_triage 2026-07-05 17:29:39 UTC
(In reply to Xin LI from comment #1)
I can confirm the patch allows me to build with lmdb >= 1.0.0.
Comment 5 Matthias Andree freebsd_committer freebsd_triage 2026-07-06 21:13:25 UTC
Created attachment 272552 [details]
[PATCH] mail/bogofilter*: Prepare LMDB 1.0 transition; add bogofilter-lmdb1 port for half a year

Let's please use this instead. It incorporates delphij@'s patch 272506 (thanks for the preparation and the lmdb0 port!).

Here's what we do now:
- mail/bogofilter currently depends on lmdb0, a 0.9.X port, so
  it continues to work as before
- add a new mail/bogofilter-lmdb1 port that depends on lmdb, the 1.0
  version of the LMDB database.
- add the upstream patch for bogofilter 1.3.0.rc1 that I made and
  that delphij@ prepared for integration into the ports tree - thanks!

Here's the plan for end of 2026:
- AT THE SAME TIME:
   - upgrade mail/bogofilter to depend on LMDB 1.0
   - portrm mail/bogofilter-lmdb1
   - add a MOVED entry to redirect from bogofilter-lmdb1 to bogofilter.

PR:             296519
---
 UPDATING                                      | 21 +++++++++++++++++
 mail/Makefile                                 |  1 +
 mail/bogofilter-kc/Makefile                   |  2 +-
 mail/bogofilter-lmdb1/Makefile                | 23 +++++++++++++++++++
 mail/bogofilter-sqlite/Makefile               |  2 +-
 mail/bogofilter/Makefile                      | 12 ++++++----
 .../files/patch-src_datastore__lmdb.c         | 20 ++++++++++++++++
 7 files changed, 75 insertions(+), 6 deletions(-)
 create mode 100644 mail/bogofilter-lmdb1/Makefile
 create mode 100644 mail/bogofilter/files/patch-src_datastore__lmdb.c
Comment 6 Matthias Andree freebsd_committer freebsd_triage 2026-07-06 21:20:51 UTC
Whoever finds this easier to commit, use the merge request https://github.com/freebsd/freebsd-ports/pull/551
Comment 7 Mathieu Arnold freebsd_committer freebsd_triage 2026-07-07 05:40:55 UTC
(In reply to Matthias Andree from comment #2)
I have removed core@ from the CC.
Ports matters are handled by portmgr@, and core@ does not need to be involved unless portmgr@ itself decides to escalate.
Please keep that in mind for future PRs.
Comment 8 Matthias Andree freebsd_committer freebsd_triage 2026-07-07 18:04:13 UTC
(In reply to Mathieu Arnold from comment #7)
You could do something useful and commit the stuff that's on the table. 
So let me ask you: Are you interested in outcome or control?
Comment 9 Matthias Andree freebsd_committer freebsd_triage 2026-07-07 18:12:19 UTC
Comment on attachment 272552 [details]
[PATCH] mail/bogofilter*: Prepare LMDB 1.0 transition; add bogofilter-lmdb1 port for half a year

arrowd@ reviewed my Gitlab pull request and had useful comments. 
Refined patch coming up shortly.
Comment 10 Matthias Andree freebsd_committer freebsd_triage 2026-07-07 21:26:16 UTC
Created attachment 272578 [details]
[PATCH v2] mail/bogofilter*: Prepare LMDB 1.0 transition; add bogofilter-lmdb1 port for half a year

Revised and cleaned up, thanks arrowd@ for the review. Github PR updated, matching commit is e26fcadf1556c184e98e3cf46a26a1e327f07563
Comment 11 Craig Leres freebsd_committer freebsd_triage 2026-07-07 22:03:34 UTC
(In reply to Matthias Andree from comment #10)
UPDATING looks good! But I still have two comments:
 
 - I've found it's best to start by turn off inbound mail otherwise 
   a new incoming message will create a new, minimal wordlist.lmdb.
 
 - Given that bogoutil -d lets you specify the path to the data 
   file, I would move it to a "safe place" before dumping.
Comment 12 Matthias Andree freebsd_committer freebsd_triage 2026-07-07 22:16:13 UTC
(In reply to Craig Leres from comment #11)

Craig, whether we need to shut down the mail system depends on if it's the entity executing bogofilter.  If bogofilter runs underneath some GUI mail application, closing the application is all that's needed.

The second suggestion is about people making backups. The extreme would be "no backup, no sympathy".

How about this.  Else, please make concrete suggestions:

------------
20260707: 
  AFFECTS: users of mail/bogofilter 
  AUTHOR: mandree@FreeBSD.org 
 
  Due to the LMDB 0.9/1.0 incompatibility (see 20260702 entry for 
  databases/lmdb), mail/bogofilter currently depends on lmdb0 so it can 
  continue using existing wordlist.lmdb databases. 
 
  For users who wish to transition to LMDB 1.0.0, please: 
  - if bogofilter is part of your mail system, stop the latter 
  - identify all wordlist.lmdb files, and 
  - for each of them, run: 
    bogoutil -d /path/to/wordlist.ldmb >/safe/place/wordlist.txt 
  - THEN you can install mail/bogofilter-lmdb1, replacing bogofilter 
  - for each of the wordlist.lmdb files, move them to another safe 
    place as backup, and recreate them from the text file, i.e. 
    for each, run: 
    bogoutil -l /path/to/wordlist.lmdb </safe/place/wordlist.txt 
 
  At a later point, we will upgrade mail/bogofilter to refer 
  to lmdb 1.0 and remove the bogofilter-lmdb1 port.
Comment 13 Matthias Andree freebsd_committer freebsd_triage 2026-07-07 22:17:12 UTC
Of course we should also add:

  - if you stopped the mail system, restart it
Comment 14 Matthias Andree freebsd_committer freebsd_triage 2026-07-08 22:04:42 UTC
Now that this went unmerged in due time, we need to MFH this to 2026Q3.
Comment 15 commit-hook freebsd_committer freebsd_triage 2026-07-09 19:28:11 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=9d0203aad0254f4adb0e81c242ad6ec8717647a5

commit 9d0203aad0254f4adb0e81c242ad6ec8717647a5
Author:     Matthias Andree <mandree@FreeBSD.org>
AuthorDate: 2026-07-07 21:23:29 +0000
Commit:     Gleb Popov <arrowd@FreeBSD.org>
CommitDate: 2026-07-09 19:26:36 +0000

    mail/bogofilter*: Prepare LMDB 1.0 transition

    Here's what we do now:
    - mail/bogofilter currently depends on lmdb0, a 0.9.X port, so
      it continues to work as before
    - add a new mail/bogofilter-lmdb1 port that depends on lmdb, the 1.0
      version of the LMDB database.
    - add the upstream patch for bogofilter 1.3.0.rc1 that I made and
      that delphij@ prepared for integration into the ports tree - thanks!

    Here's the plan for end of 2026:
    - AT THE SAME TIME:
       - upgrade mail/bogofilter to depend on LMDB 1.0
       - portrm mail/bogofilter-lmdb1
       - add a MOVED entry to redirect from bogofilter-lmdb1 to bogofilter.

    While here, clean up and rearrange the Makefiles.

    PR:             296519

    Reviewed by:    arrowd
    Pull Request:   https://github.com/freebsd/freebsd-ports/pull/551

 UPDATING                                           | 21 ++++++++++
 mail/Makefile                                      |  1 +
 mail/bogofilter-kc/Makefile                        | 15 +++----
 mail/bogofilter-lmdb1/Makefile (new)               | 23 +++++++++++
 mail/bogofilter-sqlite/Makefile                    | 13 +++---
 mail/bogofilter/Makefile                           | 46 ++++++++++++----------
 mail/bogofilter/files/patch-contrib__trainbogo.sh  |  6 +--
 .../files/patch-src_datastore__lmdb.c (new)        | 26 ++++++++++++
 8 files changed, 115 insertions(+), 36 deletions(-)