Bug 282444 - mail/fetchmail*: doesn't build after update to 6.5.0
Summary: mail/fetchmail*: doesn't build after update to 6.5.0
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Corey Halpin
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-10-31 10:33 UTC by Helge Oldach
Modified: 2024-11-05 20:25 UTC (History)
5 users (show)

See Also:
mandree: maintainer-feedback+
mandree: merge-quarterly-


Attachments
fix OPIE support and make it optional (1.45 KB, patch)
2024-10-31 11:08 UTC, Helge Oldach
no flags Details | Diff
fix OPIE support and make it optional (1.47 KB, patch)
2024-10-31 14:30 UTC, Helge Oldach
mandree: maintainer-approval-
Details | Diff
config.log with make failure on FreeBSD-14.2-PRERELEASE-amd64-20241031-25ad37c9532b-269367 snapshot (107.39 KB, text/plain)
2024-11-01 13:56 UTC, Helge Oldach
no flags Details
Proposed Patch (2.46 KB, patch)
2024-11-05 02:05 UTC, Corey Halpin
chalpin: maintainer-approval+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Helge Oldach 2024-10-31 10:33:22 UTC
yesterday's update to 6.5.0 (bug #282413) committed as 205eb9b92486da1ce7495c8ce4cbd4b66f5e4a8b breaks upon OPIE support.

security/opie gets installed on stable/14 but ./configure bails out:

....snip....
checking size of short... 2
checking size of int... 4
checking size of long... 8
checking size of long long... 8
checking for opie.h... no
configure: error: cannot find <opie.h>, which is required for OPIE support.
===>  Script "configure" failed unexpectedly.
Please report the problem to chalpin@cs.wisc.edu [maintainer] and attach the
"/usr/obj/usr/ports/mail/fetchmail/work/fetchmail-6.5.0/config.log" including
the output of the failure of your make command. Also, it might be a good idea
to provide an overview of all packages installed on your system (e.g. a
/usr/local/sbin/pkg-static info -g -Ea).
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/mail/fetchmail
*** Error code 1

Stop.
make: stopped in /usr/ports/mail/fetchmail
# make -V OSVERSION
1401503
# pkg files opie | fgrep opie.h
opie-1.20230501_1 /usr/local/include/opie.h
#

Generally speaking, OPIE support should perhaps be made optional?
Comment 1 Helge Oldach 2024-10-31 11:08:25 UTC
Created attachment 254791 [details]
fix OPIE support and make it optional

Note that OPIE_CONFIGURE_ENABLE=opie does not work due to misbehaviour of ./configure. Work around that by using OPIE_CONFIGURE_ON.

Probably RPA and SDPS should be made OPTIONal similarly.
Comment 2 Helge Oldach 2024-10-31 14:30:01 UTC
Created attachment 254807 [details]
fix OPIE support and make it optional

Oops - added the actual OPIE dependency fix.
Comment 3 Corey Halpin 2024-11-01 12:45:55 UTC
mail/fetchmail* builds correctly in poudriere on 14.1, and indeed this was tested before the update was committed. It's not at all clear to me why it would fail on 14/stable. Also, as far as I can tell, the CFLAGS addition here *should* already be part of what the ports framework is already doing. I'm not sure why it would make a difference.

Can you provide a bit more information on your environment? I'd like to figure out what is causing this failure.

If we do make OPIE optional, it should be on by default because it was enabled in previous versions of the port/package. I'd rather not be dropping features out from under users.

However, I have to say that I'm not terribly excited about making all of OPIE, RPA, and SDPS optional because it makes the port more complex and harder for me to test. If any of these options pulled in a large tree of dependencies or caused licensing concerns, then I could see the motivation. However, security/opie is quite small and does not transitively pull in anything else.
Comment 4 Helge Oldach 2024-11-01 13:52:49 UTC
(In reply to Corey Halpin from comment #3)
On 14 (which became STABLE almost a year ago), previous fetchmail had always been without OPIE support, simply because 14 doesn't have OPIE (and until now there had been no security/opie dependency). As OPIE is legacy cruft anyway, I would suggest to make it default for 12/13 only.

Avoiding influence from my environment, I just built a fresh FreeBSD-14.2-PRERELEASE-amd64-20241031-25ad37c9532b-269367-disc1.iso machine and ran into the same error with building fetchmail.

Options settings:
OPTIONS_FILE_SET+=DOCS
OPTIONS_FILE_UNSET+=NLS
OPTIONS_FILE_UNSET+=GSSAPI_BASE
OPTIONS_FILE_UNSET+=GSSAPI_HEIMDAL
OPTIONS_FILE_UNSET+=GSSAPI_MIT
OPTIONS_FILE_SET+=GSSAPI_NONE
OPTIONS_FILE_SET+=OPENSSL
OPTIONS_FILE_UNSET+=WOLFSSL
Comment 5 Helge Oldach 2024-11-01 13:56:50 UTC
Created attachment 254846 [details]
config.log with make failure on FreeBSD-14.2-PRERELEASE-amd64-20241031-25ad37c9532b-269367 snapshot
Comment 6 Helge Oldach 2024-11-01 14:27:50 UTC
(In reply to Helge Oldach from comment #5)
I tried to reproduce with various option settings and it turns out that NLS is crucial: The port builds fine with NLS enabled, but bails out if not. Comparing the config.log's, with NLS enabled, -I/usr/local/include gets added to the conftest.c command lines.
Comment 7 Matthias Andree freebsd_committer freebsd_triage 2024-11-03 14:38:33 UTC
Sorry, it was I who proposed the OPIE change and I had not tested all option combinations. 

I would add the -I only to CPPFLAGS not to CFLAGS (l. 109 of Helge's patched version), we do not need that at link time, so CPPFLAGS is good enough.

Corey also asked for my opinion on making OPIE optional for 14 only, or more precisely, the exact OSVERSION where OPIE moved from base to a port, and that would go, building on Helge's patch, like this (I did not only rework the .if but also changed to CPPFLAGS) -- the .else OPTIONS_DEFAULT is meant to look at $OSVERSION.

.if ${MASTERDIR} == ${.CURDIR} && ${OPSYS} == FreeBSD
. if ${OSVERSION} >= 1400072
.  if !empty(PORT_OPTIONS:MOPIE)
LIB_DEPENDS+=   libopie.so:security/opie        # moved to port (from base in 13.X) 
CPPFLAGS+=               -I${LOCALBASE}/include 
.  endif 
. else 
OPTIONS_DEFAULT+=       OPIE
. endif
.endif

But Corey's argument against the combinatorial explosion is also valid, and I also like to avoid options on cheap dependencies in my own ports. This would then be just the one added CPPFLAGS+= line as above.
Comment 8 Corey Halpin 2024-11-03 14:44:18 UTC
Adding to OPTIONS_DEFAULT in that block makes portlint angry:

`FATAL: Makefile: [101]: OPTIONS_DEFAULT is set after including bsd.port.pre.mk.`
Comment 9 Matthias Andree freebsd_committer freebsd_triage 2024-11-03 14:55:21 UTC
Oh, then change the bsd.port.pre.mk to  bsd.port.options.mk and the bsd.port.post.mk to bsd.port.mk, too.
Comment 10 Corey Halpin 2024-11-03 15:05:00 UTC
(Sorry, that last comment was half-baked. I should have finished it..)

Moving to `bsd.port.options.mk` as in #9 still gets `FATAL: Makefile: [101]: OPTIONS_DEFAULT is set after including bsd.port.options.mk.`


But the more I think about it, the less I like the idea of having the packages for 13.x and 14.x provide slightly different configurations. I'd prefer that they be as similar as possible. So I think my vote here would be to *not* make OPIE optional and to just add the `CPPFLAGS` needed to have it build correctly without NLS.

I will be adding a `noNLS` situation to the poudriere tests I do for updates to the port so that this doesn't recur.

IMO, this problem is evidence that the current set of options is already pretty difficult to test effectively. Adding more different ways the port can be configured doesn't make that better.
Comment 11 Matthias Andree freebsd_committer freebsd_triage 2024-11-03 15:23:29 UTC
On FreeBSD 14.1 amd64:

   text   data   bss     dec      hex   filename
  25593   9800   418   35811   0x8be3   /usr/local/lib/libopie.so.8

The entire package is reported as flat size 186 kB, but that includes documentation, tools, everything, but it used to be part of FreeBSD so personally I find it better to err on the safe side that it might still be in use, for lack of proper usage statistics.  I (as fetchmail upstream maintainer) have no idea to what extend of these more exotic or older authentication schemes are still in use, and RPA as a variant of DIGEST-MD5 for instance, probably should not be.  Whether SDPS as a provider-specific protocol extension for Demon.co.uk is still in use anywhere, I don't know either.  Wikipedia suggests that it was sold an merged/demerged several times and Vodafone absorbed remaining 15,000 users.  

Still compatibility lets me not remove major features from the upstream version unless they are unfixably seriously dangerous, and unless I bump the version to 7 so everyone's aware major change is coming, and I have much sympathy for Corey's tendency to make OPIE unconditionally enabled.
Comment 12 Helge Oldach 2024-11-04 08:46:36 UTC
Again, OPIE is deprecated since years, and has been rightfully removed from base some 2 years ago (commit 0aa2700123e22c2b0a977375e087dc2759b8e980). Therefore, it was gone from recent fetchmail pkg's built under 14 for the same reason.

Likewise, RPA (which, to make it worse, has security implications) and SDPS are legacy and weren't in the fetchmail port in the past.

IMHO including - let alone enforcing - historical stuff back again is a weird proposition. Even if it's small, that it's still legacy, little maintained code. It's fine to keep such legacy optional through ports, but what is gone should remain gone and not get re-enforced.

However, at the end of the day I don't care too much. Adding a Makefile.local is fine for my needs: CONFIGURE_ARGS:=${CONFIGURE_ARGS:N--enable-opie:N--enable-RPA:N--enable-SDPS}.

The important thing is that the port does not build unless NLS. That should be fixed.
Comment 13 Matthias Andree freebsd_committer freebsd_triage 2024-11-04 19:33:06 UTC
Deprecated does not mean anything, and neither the commit nor the reviews discussion at https://reviews.freebsd.org/D36592 shed any light on the *why*. Except that it fell out of love somehow and nobody objected to its removal...

And that is exactly disastrous communication, because to convince me it is/was bad, it would have sufficed to put *ANYWHERE* a note that we only supported OTP-MD4 and OTP-MD5.  This is something you need to dig up from deep somewhere, and takes quite a bit of research.

That being written, we should have communicated that we don't trust OPIE or S/Key these days, and we are sunsetting it because it uses weak crypto. That's a few words that belong in the reviews discussion, in the commit message, and in the release notes.  We really fs*cked up our communication here. https://www.freebsd.org/releases/14.0R/relnotes/ just mentioned it's removed and security/opie can be used if desired. No word of weak crypto. No reference to http://www.feiri.de/skey/ or relevant papers.

Now what?

I propose to kill OPIE and RPA from all versions of the port in some future due date, and we can have that by committing it to the HEAD of ports, so that it will propagate with 2025Q1. Until then we need to communicate - possibly through a pkg-message, UPDATING, and later maybe a vuln.xml entry - why the option is about to be or will have been removed.  @Corey?
Comment 14 Matthias Andree freebsd_committer freebsd_triage 2024-11-04 20:07:45 UTC
Regarding SDPS, it only triggers if the domain starts with demon, and if so and using POP3, it will use a MSP-specific extension to fetch the mail envelope. This has nothing to do with crypto and should be safe - but it's questionable whether we use it at all.
Comment 15 Corey Halpin 2024-11-05 02:05:11 UTC
Created attachment 254962 [details]
Proposed Patch

I was also not aware that OPIE had been dropped from base because it has significant security problems. X should be dropped because it "is legacy cruft anyway" is VERY different from X should be dropped because it has security problems.


Regarding "Likewise, RPA (which, to make it worse, has security implications) and SDPS are legacy and weren't in the fetchmail port in the past."

- RPA has been part of the fetchmail port since 1998 (https://cgit.freebsd.org/ports/commit/mail/fetchmail/Makefile?id=0bf0a805ecf27ba9872cc61f695f753fb1cd6c93).

- SPDS has been part of the fetchmail port since 1999 (https://cgit.freebsd.org/ports/commit/mail/fetchmail/Makefile?id=f3cc3263158c0a4b3239920f3b506aca78dcd9e8)


As for how to go forward, since OPIE has been absent on 14.x until this point I'd argue for making it optional and disabling it by default now. Users on 13.x who still want this support can rebuild to get it.

In the attached patch I've optionalized RPA but left it enabled for now and added an entry to UPDATING describing the situation.
Comment 16 Helge Oldach 2024-11-05 07:40:05 UTC
Apologies for my sloppy wording; reference was indeed the known security concerns.

I'm fine with this way forward.
Comment 17 commit-hook freebsd_committer freebsd_triage 2024-11-05 20:24:55 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=6f60247b9b290857f7312e1ce3df08bc547df789

commit 6f60247b9b290857f7312e1ce3df08bc547df789
Author:     Corey Halpin <chalpin@cs.wisc.edu>
AuthorDate: 2024-11-05 20:19:00 +0000
Commit:     Matthias Andree <mandree@FreeBSD.org>
CommitDate: 2024-11-05 20:24:07 +0000

    mail/fetchmail: make OPIE and RPA optional, UPDATING info added

    Fix a build failure on FreeBSD 14+ with OPIE (now in ports) enabled and
    NLS disabled, by adding ${LOCALBASE}/include to the compiler's path.

    OPIE and RPA have been made optional because both have significant security
    problems. OPIE support was not provided by the port on 14.x until the update
    to 6.5.0, when it was briefly re-activated. Because of its security flaws, it
    has now been disabled by default on both 13.x and 14.x.

    RPA is currently enabled by default, but this will change in a future update.

    PR:             282444
    Reported by:    Helge Oldach

 UPDATING                | 11 +++++++++++
 mail/fetchmail/Makefile | 19 ++++++++++++-------
 2 files changed, 23 insertions(+), 7 deletions(-)
Comment 18 Matthias Andree freebsd_committer freebsd_triage 2024-11-05 20:25:40 UTC
Then we'll consider this case closed, I have committed Corey's patch with part my message, part the UPDATING part copied, and bumped PORTREVISION to flush out the OPIE option according to the new default.

Thanks everyone!