Bug 265098 - mail/exim: update to 4.96
Summary: mail/exim: update to 4.96
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Kurt Jaeger
URL: https://lists.exim.org/lurker/message...
Keywords:
: 271233 (view as bug list)
Depends on:
Blocks: 258413
  Show dependency treegraph
 
Reported: 2022-07-08 21:20 UTC by Vladimir Druzenko
Modified: 2023-06-10 09:01 UTC (History)
8 users (show)

See Also:
fluffy: maintainer-feedback+


Attachments
patch (81.23 KB, patch)
2023-04-09 07:59 UTC, Kurt Jaeger
no flags Details | Diff
patch-v2 (81.50 KB, patch)
2023-04-09 08:30 UTC, Kurt Jaeger
no flags Details | Diff
patch-v3 (170.12 KB, patch)
2023-04-10 11:35 UTC, Kurt Jaeger
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Vladimir Druzenko freebsd_committer freebsd_triage 2022-07-08 21:20:01 UTC
Author: Jeremy Harris
Date: 2022-06-25 17:18 +300
To: exim-announce, exim users
Subject: [exim] Exim 4.96 released

New stuff we've added since 4.95:

  - A new ACL condition: seen. Records/tests a timestamp against a key.
  - A variant of the "mask" expansion operator to give normalised IPv6.
  - UTC output option for exim_dumpdb, exim_fixdb.
  - An event for failing TLS connects to the daemon.
  - The ACL "debug" control gains options "stop", "pretrigger" and "trigger".
  - Query-style lookups are now checked for quoting, if the query string is
     built using untrusted data ("tainted").  For now lack of quoting is merely
     logged; a future release will upgrade this to an error.
  - The expansion conditions match_<list-type> and inlist now set $value for
     the expansion of the "true" result of the ${if}.  With a static list, this
     can be used for de-tainting.

Notable removals since 4.95:

   - the "allow_insecure_tainted_data" main config option and the
     "taint" log_selector.  These were deprecated in the 4.95 release.
Comment 1 doctor 2022-09-16 17:55:29 UTC
When are we going to get an update?

I have to manual use exim-M for e-mail stuck in a queue.

Additional my mail lists are not working

Temporarily rejected RCPT: failed to expand "/usr/home/majordomo/lists/$local_data"
Comment 2 Vladimir Druzenko freebsd_committer freebsd_triage 2022-10-22 06:56:06 UTC
ping
Comment 3 Kurt Jaeger freebsd_committer freebsd_triage 2023-04-09 07:59:22 UTC
Created attachment 241365 [details]
patch

This patch seems to build. More tests @work
Comment 4 Kurt Jaeger freebsd_committer freebsd_triage 2023-04-09 08:30:10 UTC
Created attachment 241368 [details]
patch-v2

this builds on 14.0, 14.0i, 13.2, 13.1, 12.4. Run-test on 13.1 looks ok, but my config is not yet completly untainted 8-(
Comment 5 David Siebörger 2023-04-09 20:02:05 UTC
(In reply to Kurt Jaeger from comment #4)

I've a few comments on the patch:

* The change from USE_DB to USE_NDBM means that any existing configuration like ${lookup{$needle}dbm{haystack.db}} will need to change to ${lookup{$needle}dbm{haystack}}.  Perhaps this needs to be mentioned in UPGRADING?

* I'd recommend removing files/tls/patch-tls1.  The change that it intends to make was already included in Exim 4.96.  Now the patch applies with fuzz and changes something different.

* Since XX_STRIPCMD_XX was removed from files/patch-OS__Makefile-FreeBSD, I'd suggest removing the sed from Makefile, lines 602 and 603.

* I'd suggest removing files/patch-src-dmarc.c in favour of the patch that's been committed after 4.96 was released.  Debian have a patch here: https://salsa.debian.org/exim-team/exim4/-/blob/master/debian/patches/75_18-Fix-Build-with-libopendmarc-1.4.x-fixes-2728.patch

I haven't run this in production yet, but I have my testing with exim -bh has worked so far.
Comment 6 Kurt Jaeger freebsd_committer freebsd_triage 2023-04-10 08:06:15 UTC
(In reply to David Siebörger from comment #5)
Thanks for the very good recommendations. I'll adapt my patch.
And thanks for the pointer to https://salsa.debian.org/exim-team/exim4/-/blob/master/debian/patches/, I think we can use some of those other patches as well.
Comment 7 Kurt Jaeger freebsd_committer freebsd_triage 2023-04-10 11:35:07 UTC
Created attachment 241399 [details]
patch-v3

This adds the recommendations from David. Testbuild is fine. running on two of my testboxes now.
Comment 8 odhiambo@gmail.com 2023-04-10 22:50:09 UTC
(In reply to Kurt Jaeger from comment #7)
Hopefully we get an entry in /usr/ports/UPDATING??

In the meantime, how do I test the patches on my system?
Comment 9 Dima Panov freebsd_committer freebsd_triage 2023-04-11 11:49:47 UTC
Kurt, feel free to update port if you are ready.
Sorry, my current job give me not so much free time to play with exim
Comment 10 David Siebörger 2023-04-11 13:25:59 UTC
I've just installed a package built with Kurt's patch-v3 on a live server and found that the DB -> NDBM .db filename issue also affects Exim's retry, wait, etc databases.  It causes a confusing log entry like this:

2023-04-11 14:52:36.528 [74117] 1pmDUB-000JHM-17 failed to open DB file /var/spool/exim/db/wait-smtp: File exists

This fixed the issue for me:

service exim stop
find /var/spool/exim/db/* -not -name \*.lockfile -exec mv -i {} {}.db \;
service exim start

This is touched on the Exim documentation [1].  Anyone who's already converted all their databases to TDB or GDBM oughtn't be affected.

[1] https://exim.org/exim-html-current/doc/html/spec_html/ch-building_and_installing_exim.html#SECTdb
Comment 11 j2465 2023-04-11 14:05:00 UTC
In general, when changing DB types you're probably going to have to lose the
stored info.  For the hints databases (which are purely that) it is safe to
just remove the files.
Comment 12 Kurt Jaeger freebsd_committer freebsd_triage 2023-04-11 18:48:22 UTC
(In reply to David Siebörger from comment #10)
I'll add a note to UPDATING about this.
Comment 13 David Siebörger 2023-05-23 07:51:29 UTC
I've been using exim packages built with patch-v3 in production for more than a month now.  IMHO, this is ready to be committed.
Comment 14 Kurt Jaeger freebsd_committer freebsd_triage 2023-05-26 13:40:20 UTC
*** Bug 271233 has been marked as a duplicate of this bug. ***
Comment 15 commit-hook freebsd_committer freebsd_triage 2023-05-26 14:18:12 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=985788b37e14798739af343e2f5a7dfdf7d59b62

commit 985788b37e14798739af343e2f5a7dfdf7d59b62
Author:     Kurt Jaeger <pi@FreeBSD.org>
AuthorDate: 2023-05-26 14:14:00 +0000
Commit:     Kurt Jaeger <pi@FreeBSD.org>
CommitDate: 2023-05-26 14:14:00 +0000

    mail/exim: update 4.95 -> 4.96

    PR:             265098
    Obtained-from:  https://salsa.debian.org/exim-team/exim4/-/blob/master/debian/patches/
    Reported-by:    vvd
    Approved-by:    fluffy
    Reviewed-by:    drs-freebsd@sieborger.nom.za
    Tested-by:      pi, drs-freebsd@sieborger.nom.za
    Changes:        https://lists.exim.org/lurker/message/20220625.141825.d6de6074.en.html

 UPDATING                                           |  12 +
 mail/exim/Makefile                                 |  46 +-
 mail/exim/distinfo                                 |   6 +-
 ...imeofday-select-per-char-for-cmdli.patch (gone) | 616 --------------
 .../75_32-Fix-PAM-auth.-Bug-2813.patch (gone)      |  25 -
 ...iqgrep-check-arg-parsing.-Bug-2821.patch (gone) |  42 -
 ...l-uses-of-select-to-poll.-Bug-2831.patch (gone) | 931 ---------------------
 ...sic-memory-use-for-SPARC.-Bug-2838.patch (gone) | 140 ----
 .../75_43-BSD-fix-resource-leak.patch (gone)       |  61 --
 ...bogus-error-message-copy.-Bug-2857.patch (gone) |  38 -
 ...tory-in-redirect-routers.-Bug-2715.patch (gone) |  62 --
 ...55-Specific-check-for-null-pointer.patch (gone) |  67 --
 mail/exim/files/patch-OS__Makefile-Default (gone)  |  10 -
 mail/exim/files/patch-OS__Makefile-FreeBSD         |  20 +-
 mail/exim/files/patch-OS_os.c-FreeBSD (gone)       |  15 -
 mail/exim/files/patch-src-dmarc.c (gone)           |  19 -
 mail/exim/files/tls/patch-tls1 (gone)              |  43 -
 mail/exim/files/tls/patch-tls2 (gone)              | 174 ----
 18 files changed, 53 insertions(+), 2274 deletions(-)
Comment 16 commit-hook freebsd_committer freebsd_triage 2023-05-26 18:34:49 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=003482675264a732124cbd65b3a76cb67badd49a

commit 003482675264a732124cbd65b3a76cb67badd49a
Author:     Kurt Jaeger <pi@FreeBSD.org>
AuthorDate: 2023-05-26 18:33:29 +0000
Commit:     Kurt Jaeger <pi@FreeBSD.org>
CommitDate: 2023-05-26 18:33:29 +0000

    mail/exim: fix update to 4.96 by adding missing patches

    PR:             265098
    Reported by:    David Siebuerger <drs-freebsd@sieborger.nom.za>

 ...pt-to-rewrite-a-malformed-address.-.patch (new) |  57 +++++
 ...ix-memory-accounting-for-error-case.patch (new) |  25 +++
 ...ix-regex-n-use-after-free.-Bug-2915.patch (new) | 193 +++++++++++++++++
 ..._09-Fix-non-WITH_CONTENT_SCAN-build.patch (new) |  58 ++++++
 ...0-Fix-non-WITH_CONTENT_SCAN-build-2.patch (new) | 135 ++++++++++++
 ...1-Fix-non-WITH_CONTENT_SCAN-build-3.patch (new) |  45 ++++
 ...-clients-offering-no-TLS-extensions.patch (new) | 114 ++++++++++
 ...-with-libopendmarc-1.4.x-fixes-2728.patch (new) |  88 ++++++++
 ...-use-after-free-in-dmarc_dns_lookup.patch (new) |  39 ++++
 .../75_22-Fix-daemon-startup.-Bug-2930.patch (new) |  68 ++++++
 ...x-reccipients-after-run.-.-Bug-2929.patch (new) |  45 ++++
 ...ing-capture-variables-for-null-matc.patch (new) |  79 +++++++
 ...ng-capture-variables-for-null-match.patch (new) |  94 +++++++++
 ...string-capture-commentary.-Bug-2933.patch (new) |  48 +++++
 ...oading-creds-do-the-server-certs-be.patch (new) | 232 +++++++++++++++++++++
 ...e-expansion-of-tls_verify_certifica.patch (new) | 217 +++++++++++++++++++
 ...50-Fix-logging-of-max-size-log-line.patch (new) |  82 ++++++++
 ...-dns_again_means_nonexist.-Bug-2911.patch (new) |  76 +++++++
 ...-socket-explicitly-on-connect-ACL-d.patch (new) |  50 +++++
 ...ccurve-setting-explicit-curve-group.patch (new) | 184 ++++++++++++++++
 ...ccurve-on-earlier-versions-than-3.0.patch (new) |  42 ++++
 ...-rejected-for-bad-ALPN-with-the-off.patch (new) |  99 +++++++++
 ...-dns_again_means_nonexist-for-TLSA-.patch (new) |  96 +++++++++
 .../75_66-Fix-crash-in-expansions.patch (new)      |  84 ++++++++
 24 files changed, 2250 insertions(+)
Comment 17 odhiambo@gmail.com 2023-05-27 14:39:08 UTC
I have done 'make config' and enabled what I want, EXCEPT SA-Exim, but:

root@gw:/usr/ports/mail/exim # make
Exim now drops privileges when alternate configuration
files are used.  You can set make variable TRUSTED_CONFIG_LIST
to specify the list of configuration files for which
root privileges will be retained.

You can whitelist some macros using the make variable
WHITELIST_D_MACROS.  This is useful if you are running
with DISABLE_D_OPT set, but macros whitelisting will be
removed in some future Exim release, so it is better
to use TRUSTED_CONFIG_LIST to set the list of trusted
configuration files.

Included extra patches:

===>  License GPLv2 accepted by the user
===>   exim-4.96 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by exim-4.96 for building
===>  Extracting for exim-4.96
=> SHA256 Checksum OK for exim/exim-4.96.tar.bz2.
=> SHA256 Checksum OK for exim/sa-exim-4.2.1.tar.gz.
===>  Patching for exim-4.96
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_01-Fix-exit-on-attempt-to-rewrite-a-malformed-address.-.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_05-SPF-fix-memory-accounting-for-error-case.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_08-Fix-regex-n-use-after-free.-Bug-2915.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_09-Fix-non-WITH_CONTENT_SCAN-build.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_10-Fix-non-WITH_CONTENT_SCAN-build-2.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_11-Fix-non-WITH_CONTENT_SCAN-build-3.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_16-GnuTLS-fix-for-clients-offering-no-TLS-extensions.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_18-Fix-Build-with-libopendmarc-1.4.x-fixes-2728.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_19-DMARC-fix-use-after-free-in-dmarc_dns_lookup.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_22-Fix-daemon-startup.-Bug-2930.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_23-Fix-reccipients-after-run.-.-Bug-2929.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_31-Fix-regext-substring-capture-variables-for-null-matc.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_32-Fix-regex-substring-capture-variables-for-null-match.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_34-Fix-regex-substring-capture-commentary.-Bug-2933.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_37-OpenSSL-when-preloading-creds-do-the-server-certs-be.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_38-OpenSSL-fix-double-expansion-of-tls_verify_certifica.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_50-Fix-logging-of-max-size-log-line.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_55-Fix-recursion-on-dns_again_means_nonexist.-Bug-2911.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_58-Close-server-smtp-socket-explicitly-on-connect-ACL-d.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_60-OpenSSL-fix-tls_eccurve-setting-explicit-curve-group.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_62-OpenSSL-Fix-tls_eccurve-on-earlier-versions-than-3.0.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_63-OpenSSL-log-conns-rejected-for-bad-ALPN-with-the-off.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_64-DANE-do-not-check-dns_again_means_nonexist-for-TLSA-.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/debian/75_66-Fix-crash-in-expansions.patch with -p1
===>  Applying extra patch /usr/ports/mail/exim/files/extra-patch-Local-sa-exim.c
No file to patch.  Skipping...
13 out of 13 hunks ignored--saving rejects to Local/sa-exim.c.rej
===>  FAILED Applying extra patch /usr/ports/mail/exim/files/extra-patch-Local-sa-exim.c
*** Error code 1

Stop.
make: stopped in /usr/ports/mail/exim
Comment 18 Dani I. 2023-06-06 13:42:10 UTC
Hey guys. Exim 4.95 has been marked broken with OpenSSL 3.0 / 3.1. I've tried building Exim 4.96 with OpenSSL, which worked without any problem. Is it possible that this has been forgotten to remove?
Comment 19 Muhammad Moinur Rahman freebsd_committer freebsd_triage 2023-06-06 14:20:30 UTC
(In reply to Dani I. from comment #18)
I believe this was a mid air collision when I marked it BROKEN and the MAINTAINER was updating the port. I will fix it but by any chance can you confirm whether if it works with OpenSSL 3.1 or not ?
Comment 20 Dani I. 2023-06-06 14:47:24 UTC
Sadly i currently neither have an environment to build and test with OpenSSL 3.1 :(.
Comment 21 commit-hook freebsd_committer freebsd_triage 2023-06-07 07:25:45 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=842fc2718bed1dc7dbdc9effa6b5226be460733b

commit 842fc2718bed1dc7dbdc9effa6b5226be460733b
Author:     Muhammad Moinur Rahman <bofh@FreeBSD.org>
AuthorDate: 2023-06-06 20:53:18 +0000
Commit:     Muhammad Moinur Rahman <bofh@FreeBSD.org>
CommitDate: 2023-06-07 07:25:07 +0000

    mail/exim: Unmark BROKEN for openssl30

    New version 4.96 comes with support for OpenSSL 3.0.X which has been
    tested however support with 3.1.X has not yet been tested.

    It maybe the case that while I was marking ports BROKEN the MAINTAINER
    was updating the port and missed this snippet causing a mid air
    collission.

    PR:             265098
    Tested by:      i.dani@outlook.com

 mail/exim/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 22 odhiambo@gmail.com 2023-06-10 09:01:21 UTC
Why is it that sa-exim must be enabled as an option for a successful patching process and compile? Or is my system not clean?