Bug 249714

Summary: mail/mailman: depends on python 2.x yet is a crucial piece of FreeBSD infrastructure
Product: Ports & Packages Reporter: Steve Wills <swills>
Component: Individual Port(s)Assignee: Matthias Andree <mandree>
Status: Closed Not A Bug    
Severity: Affects Only Me CC: adamw, core, de-freebsd, dvl, george, meta, philip, pi, portmgr, postmaster, rene, saper
Priority: --- Flags: mandree: maintainer-feedback+
Version: Latest   
Hardware: Any   
OS: Any   
Bug Depends on:    
Bug Blocks: 249337    

Description Steve Wills freebsd_committer freebsd_triage 2020-09-24 16:30:43 UTC
This port will be removed on 2020-12-31 if not fixed
Comment 1 Matthias Andree freebsd_committer freebsd_triage 2020-09-24 16:44:12 UTC
"Only over my and the postmaster's dead corpses."

Please arrange for a Tauthon port so we can move mailman onto it, until that is available, I will not consent to a removal of Python or mailman.
Comment 2 Matthias Andree freebsd_committer freebsd_triage 2020-09-24 16:49:19 UTC
Note that FreeBSD mailing list rely on mailman 2.x, and consequently on python 2.x. 

Mailman 3.x is a very different beast without migration path from 2.x, different setup, with a different archiver concept and whatnot. Archive migration is impossible, old archives would need to be frozen and new archives begun.  Mailman 3.x is not complete in terms of 2.x features, bounce processing, for instance, was only an afterthought in Mailman 3.x.

This was discussed before.

Just as someone threatens to remove this port or its requisites, I'll threaten in return to revert removals of Mailman and its requisites.
Comment 3 Adam Weinberger freebsd_committer freebsd_triage 2020-09-24 22:43:58 UTC
(In reply to Matthias Andree from comment #2)

> Just as someone threatens to remove this port or its requisites,
> I'll threaten in return to revert removals of Mailman and its requisites.

Please simmer down, Matthias. If you're under the impression that aggressive comments like that somehow make people more willing to work with you, you are quite wrong.

This conversation has been going on for a while now, so you jumping in in the middle and casting threats serves only to destabilize it. Please, just stop.
Comment 4 Philip Paeps freebsd_committer freebsd_triage 2020-09-25 02:15:06 UTC
Note that work is ongoing to decouple the archives from the mail processing.  bapt@ has a working prototype for us already.

Having said that: postmaster would be deeply unhappy if mailman2/python2 disappears from the ports tree as scheduled on 2020-12-31.

A mailman3 port was not even available in the ports tree until a couple of days ago.  Moving a critical piece of FreeBSD Project infrastructure in ~3 months is not a fun prospect.
Comment 5 Dan Langille freebsd_committer freebsd_triage 2020-11-26 19:02:16 UTC
recent information:

mail/mailman3 is not in working order. Some dependencies are missing, some are not present in the ports tree, including zope5.

Related tickets:

* zope: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250362
* run-tests needed: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225543#c14

What I've found missing so far:

* https://dan.langille.org/2020/11/25/missing-dependencies-from-mail-mailman3/

Updates will continue in the zope ticket above.
Comment 6 Garrett Wollman freebsd_committer freebsd_triage 2020-12-04 21:14:55 UTC
This is a pretty big deal for a lot of us who run small-scale mailing-lists. mailman2 will be supported by Ubuntu until 2025, presumably by RHEL for a similar time period. Should FreeBSD really be the first major OS to kick it to the curb? It would feel more like a kick in the teeth for a lot of people who have a setup that is *working* when the alternatives require a substantial investment in time and are not feature-complete.
Comment 7 Mathieu Arnold freebsd_committer freebsd_triage 2021-01-12 09:05:30 UTC
(In reply to Garrett Wollman from comment #6)

Ok, so, Python 3 was released 15 years ago. (2006)
Python 2 was deemed to be dead and being suported 5 years after 2.7 was released, that was 12-13 years ago. (circa 2008)
7 or 8 years ago, Python 2's EOL was explicitely set to 2015 (but as 2.7.0 was released in 2010, it always was 2015).
6 or 7 years ago, it was pushed to 2020.

Any software that has been sticking their head in the sand for the past 15 years can be considered as dead.

So, mailman is dead, and it is really not the FreeBSD's ports tree policy to support dead software.
Comment 8 Kurt Jaeger freebsd_committer freebsd_triage 2021-01-12 09:13:51 UTC
Maybe it's time to change the policy, if critical infrastructure depends on it.
It's not only for the project itself, but many users depend on mailman2 as well.

Use common sense, do not value policy over common sense.
Comment 9 Mathieu Arnold freebsd_committer freebsd_triage 2021-01-12 10:23:11 UTC
It does not make any sense to keep dead software.

Removing mailman from the ports tree does not mean it will be removed for people who installed it.  People can simply pkg lock it and voilĂ , no need to make so much fuss.
Comment 10 Rene Ladan freebsd_committer freebsd_triage 2021-01-12 10:49:06 UTC
Well, you could have complained to upstream for a long time...

Indeed, if a port gets removed it does not mean that the package gets uninstalled, I had libtrue installed until this weekend which was removed in 2019. So people with mailman 2 installed keep their mailman 2 and new installations will use something else.
Comment 11 Kurt Jaeger freebsd_committer freebsd_triage 2021-01-12 10:59:16 UTC
I know that installed packages are not getting removed if the port is being
removed. But I know which signal it sends to users of the package.

In this particular case, the signal will be "Hey, FreeBSD's loosing
another important package, let's move to some other OS".

I have not yet seen info that shows that this is being reflected in that decision.
Comment 12 Matthias Andree freebsd_committer freebsd_triage 2021-01-12 11:55:57 UTC
So let me waste my lunch break to remind everyone and in particular portmgr@ and core@ of several things that have long been known but that mat@ appears to have been ignoring or pushing to the side.

- upstream did not stick their head in the sand, but made a decision to rewrite from scratch with a different concept, lost features along the way (automated bounce mail processing was retrofitted a very long time after many releases only), so mailman 3 is available but not a replacement.  I personally call that decision misguided.

- mailman 2 is not "dead", whenever important bugs were found (critical to operation, or security), they were fixed in a very short time frame. It's feature-frozen, nothing else. 

So please stop defaming the project, it makes us and you in particular look stupid. 

- mailman 2 to 3 migration tools are barely existent and far from feature complete

- several solutions were proposed, such as rebasing onto 1. tauthon or 2. pypy. 

-> Tauthon is in ports, but lacks infrastructure support (a USES=tauthon analogon to USES=python is missing, or better integrating tauthon as nondefault last-resort option into Uses/python.mk.) so we have a hard time building requisites.  Rumors have it that pypy "might" be doable and then again not, so I personally would try Tauthon first.

- I actually committed tauthon and called Python@ for help with the integration above. No reply. 

I propose to escalate this integration to portmgr@ and obtain the necessary support from core@ or foundation or developers@ to get tauthon first-class.

- I laid out the issues repeatedly, and also applied to portmgr@ and core@ to decide to postpone the Python EOL, no reply.  One core@ member chose to bitch at me for putting up a due date well before Python 2.x's EOL, one portmgr@ now derides the upstream project as seen in previous comments.

So I propose the way forward is:

1 - collect a list of projects affected like this. One might think "all open bugs in 249337" is a good starting point, let's sift and sieve that.

2 - for these few ports where upstream maintainers have passed similar decisions as for mailman, where a Python 3 compatible version incurs a critical deviation, let's get tauthon first-class, so we can port - for instance mailman 2 - onto Tauthon, and test drive things for three months. Or Pypy. Whatever works.  

I can't integrate Tauthon without support from python-in-ports gurus, else it would probably have happened.

3 - the after-next quarterly branch after all these ports have migrated gets branched off without Python 2.7. Example: Suppose we finish item #2 in May, we'll ditch Python for 2021Q4 on Oct 1st.
Comment 13 Matthias Andree freebsd_committer freebsd_triage 2021-01-12 12:01:00 UTC
oh and of course we can pass AND MORE IMPORTANTLY PROPERLY COMMUNICATE 
a portmgr@ decision to ban "new py27-* ports" to be "not permitted unless they are a crucial requisite to another port we've had before py27 was marked deprecated and expiring end 2020".   For instance, if py27-cryptoA is discontinued but py27-cryptoB would fill its shoes, then I would permit py27-cryptoB as a rare exception.

This ban can actually happen right now already and I believe would be the course of action anyways even without formal stipulation.
Comment 14 George Mitchell 2021-01-12 13:41:52 UTC
Until I read this comment, I had never heard of Tauthon.  Thanks for bringing it to my attention.  I'll go educate myself on it now, and perhaps I'll be able to help in some way with getting Mailman to work with it (though I'm one of those people who tends to overpromise in my non-professional life).
Comment 15 Kurt Jaeger freebsd_committer freebsd_triage 2021-01-14 08:26:57 UTC
meta@ just mentioned that wiki.freebsd.org is based on www/moinmoin,
which is also py2 dependent...
Comment 16 Koichiro Iwao freebsd_committer freebsd_triage 2022-02-19 07:57:53 UTC
See also this Debian discussion.

The future of wiki.debian.org with MoinMoin 1.9.x
https://salsa.debian.org/debian/grow-your-ideas/-/issues/2