Bug 250362 - mail/mailman3: zope.interface>=5.0 is required
Summary: mail/mailman3: zope.interface>=5.0 is required
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Many People
Assignee: freebsd-ports-bugs (Nobody)
Depends on:
Reported: 2020-10-15 09:02 UTC by Julien Cigar
Modified: 2021-01-18 11:56 UTC (History)
11 users (show)

See Also:

Add missing dependencies declared in setup.py (1.74 KB, patch)
2020-11-26 23:49 UTC, Dan Langille
no flags Details | Diff
/usr/local/mailman/postorius/wsgi.py (532 bytes, text/plain)
2020-12-14 15:24 UTC, Henri Hennebert
no flags Details
The settings (8.33 KB, text/plain)
2020-12-26 12:29 UTC, Henri Hennebert
no flags Details
patch to mail/mailman3/files/patch-src_mailman_core_initialize.py (668 bytes, patch)
2020-12-28 14:01 UTC, Henri Hennebert
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Julien Cigar 2020-10-15 09:02:59 UTC
root@mailman:/ # pkg install py37-mailman
Message from py37-mailman-3.3.1:

To initialise Mailman's runtime directories, log files and database
for the first time, run

  service mailman info

You can then start Mailman by running:

  service mailman start
root@mailman:/ # service mailman info
Cannot 'info' mailman. Set mailman_enable to YES in /etc/rc.conf or use 'oneinfo' instead of 'info'.
root@mailman:/ # sysrc mailman_enable=YES
mailman_enable:  -> YES
root@mailman:/ # service mailman info
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (zope.interface 4.6.0 (/usr/local/lib/python3.7/site-packages), Requirement.parse('zope.interface>=5.0'), {'mailman'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/mailman", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3251, in <module>
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 585, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'zope.interface>=5.0' distribution was not found and is required by mailman
Comment 1 Bugzilla Automation freebsd_committer 2020-10-15 09:02:59 UTC
Maintainer informed via mail
Comment 2 crest 2020-10-27 12:57:29 UTC
I just ran into the same problem.
Comment 3 Dan Langille freebsd_committer 2020-11-25 22:08:59 UTC
I just hit there.  What's up?
Comment 4 Dan Langille freebsd_committer 2020-11-25 22:17:43 UTC
The original PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225543
Comment 5 Dan Langille freebsd_committer 2020-11-25 22:25:24 UTC
The zope5 dependency arrived with this commit 8 months ago in March 20202

Comment 6 Dan Langille freebsd_committer 2020-11-25 23:13:51 UTC
I managed to get mailman running by:

installing these:

* py37-importlib-resources
* py37-authres
* py37-authheaders
* py37-publicsuffix2
* py37-dkimpy

Modifying this line in /usr/local/etc/rc.d/mailman:


Running this to update /etc/rc.conf:

sudo sysrc mailman_config=/usr/local/lib/python3.7/site-packages/mailman/config/mailman.cfg

Modifying this file: /usr/local/lib/python3.7/site-packages/mailman-3.3.1-py3.7.egg-info/requires.txt

changing this line: zope.interface>=5.0

to: zope.interface>=4.0
Comment 7 Dan Langille freebsd_committer 2020-11-25 23:35:07 UTC
Documentation of how I got to the above:

Comment 8 Julien Cigar 2020-11-26 13:29:24 UTC
changing the zope.interface requirement from >= 5.0 to >= 4.0 should only be a temporary hack, there is a good reason, I guess, that it should be >= 5.0 (and you'll hit import errors or undefined errors or ...)

maybe we could create two separate ports: one for zope.interface 4.x and one for zope.interface 5.x (I haven't checked if there are backward incompatibilities between the two versions), and fix dependencies

now, I'm wondering if this port has ever been tested as there are other missing dependencies as well (gunicorn, ...) ...

(check https://gitlab.com/mailman/mailman/-/blob/c753dba53224cc6aacc426897577d9b098feae8d/setup.py#L106-128 for dependencies)
Comment 9 Dan Langille freebsd_committer 2020-11-26 13:33:01 UTC
(In reply to Julien Cigar from comment #8)

That is the list of dependencies I have been following. I have created a new list and imported the corresponding list from Mailman 2. That list has not been tested.

I don't see a reason to use zope.interface=4 because Mailman wants 5.

I don't see zope 5 in ports, but I don't know enough about zope to know what I'm looking for.
Comment 10 Kurt Jaeger freebsd_committer 2020-11-26 13:44:14 UTC
(In reply to Julien Cigar from comment #8)
Well, there's a reason that I had a "needs run-tests" in the original PR:

Comment 11 Dan Langille freebsd_committer 2020-11-26 14:01:08 UTC
I am in the process of updating an existing Mailman2 installation via https://docs.mailman3.org/en/latest/migration.html, we shall see. zfs snapshots have been taken. We shall soon see.

If others can work on the zope and run-test stuff. I'll work on this side.
Comment 12 Dan Langille freebsd_committer 2020-11-26 23:35:33 UTC
Four ports created:


via: https://svnweb.freebsd.org/ports?view=revision&revision=556421
Comment 13 Dan Langille freebsd_committer 2020-11-26 23:49:22 UTC
Created attachment 220017 [details]
Add missing dependencies declared in setup.py

Add some dependencies I know are missing.
Comment 14 Einar Bjarni Halldórsson 2020-11-27 08:15:19 UTC
Great work. I got it running on a test machine by installing needed dependencies with pip as they popped up. I still haven't tried sending mail, but  hopefully today.
Comment 15 Dan Langille freebsd_committer 2020-11-27 13:12:51 UTC
To get new #FreeBSD port www/py-hyperkitty to build:

* update devel/py-flufl.lock to 4.0
* update these ports to use www/py-django31 not www/py-django22:

And create these new #FreeBSD ports:

* www/py-django-q
* www/py-robot-detection
Comment 16 Dan Langille freebsd_committer 2020-11-27 13:46:21 UTC
These have not been commited to the FreeBSD ports tree because I'm not sure we
are heading down the HyperKitty path.

https://www.freshports.org/www/hypermail may be the new way.

Comment 17 crest 2020-11-30 15:59:02 UTC
Is there a plan towards a working mailman3 setup without resorting to `pip install --user`?
Comment 18 Dan Langille freebsd_committer 2020-12-01 00:26:39 UTC
For me, now, I'm stopped.

Speaking of pip, here's one solution: https://pads.c3w.at/code/#/2/code/edit/4doDChOdxO6OpgsohywJ-akf/
Comment 19 Henri Hennebert 2020-12-13 14:40:29 UTC
After (In reply to Dan Langille from comment #6)

After installing mail/mailman3 and complementing ports as by Dan, here is my route:

mv /usr/local/mailman /usr/local/mailman2

to be safe:

cd /usr/local
tar cvzf mailman2 mailman2.tgz
chflags schg mailman2.tgz


[root@tignes local]# mkdir mailman
[root@tignes local]# chown mailman:mailman mailman
[root@tignes local]# su -l mailman
[mailman@tignes ~]$ pwd
[mailman@tignes ~]$ ls -a
.	..
[mailman@tignes ~]$ mailman info
GNU Mailman 3.3.1 (Tom Sawyer)
Python 3.7.9 (default, Oct 26 2020, 11:47:54) 
[Clang 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611a
config file: None
db url: sqlite:////usr/local/mailman/data/mailman.db
devmode: DISABLED
REST root url: http://localhost:8001/3.1/
REST credentials: restadmin:restpass
[mailman@tignes ~]$ ls -l 
total 45
drwxr-xr-x   2 mailman  mailman   2 Dec 13 14:36 archives
drwxr-xr-x   2 mailman  mailman   2 Dec 13 14:36 cache
drwxr-xr-x   2 mailman  mailman   3 Dec 13 14:36 data
drwxr-xr-x   2 mailman  mailman   3 Dec 13 14:36 etc
drwxr-xr-x   2 mailman  mailman   2 Dec 13 14:36 lists
drwxr-xr-x   2 mailman  mailman   2 Dec 13 14:36 locks
drwxr-xr-x   2 mailman  mailman   7 Dec 13 14:36 logs
drwxr-xr-x   2 mailman  mailman   2 Dec 13 14:36 messages
drwxr-xr-x  14 mailman  mailman  14 Dec 13 14:36 queue
drwxr-xr-x   2 mailman  mailman   2 Dec 13 14:36 templates
[mailman@tignes ~]$ cat etc/mailman.cfg 
# This is your GNU Mailman 3 configuration file.  You can edit this file to
# configure Mailman to your needs, and Mailman will never overwrite it.
# Additional configuration information is available here:
# https://mailman.readthedocs.io/en/latest/src/mailman/config/docs/config.html
# For example, uncomment the following lines to run Mailman in developer mode.
# [devmode]
# enabled: yes
# recipient: your.address@your.domain
[mailman@tignes ~]$ cat <<EOF >>etc/mailman.cfg 
> [webservice]
> hostname:
> port: 8001
> use_https: no
> admin_user: restadmin
> admin_pass: restpass
> api_version: 3.1

--- I replace localhost by because I am running with
--- ip6addrctl_policy="ipv6_prefer" in /etc/rc.conf.local and
--- mailman seems to listen on ipv4 only and if i want to
--- be able to user mailmanclient without undue delay...

mailman@tignes ~]$ exit
[root@tignes local]# service mailman start
Starting Mailman's master runner
Generating MTA alias maps

Then I install mail/py-mailmanclient, www/py-django, devel/py-pip.

And then, to avoid modifying the /usr/local/lib/python3.7:

[root@tignes local]# su -l mailman
[mailman@tignes ~]$ pip install postorius
Defaulting to user installation because normal site-packages is not writeable
Collecting postorius
  Downloading postorius-1.3.3.tar.gz (2.7 MB)
     |################################| 2.7 MB 4.9 MB/s 
Collecting django-mailman3>=1.3.3
  Downloading django-mailman3-1.3.4.tar.gz (390 kB)


Successfully built postorius django-mailman3 django-allauth cmarkgfm
Installing collected packages: defusedxml, python3-openid, oauthlib, requests-oauthlib, pyjwt, django-allauth, django-gravatar2, django-mailman3, webencodings, bleach, cmarkgfm, readme-renderer, postorius
  WARNING: The script pyjwt is installed in '/usr/local/mailman/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed bleach-3.2.1 cmarkgfm-0.5.0 defusedxml-0.6.0 django-allauth-0.44.0 django-gravatar2-1.4.4 django-mailman3-1.3.4 oauthlib-3.1.0 postorius-1.3.3 pyjwt-1.7.1 python3-openid-3.2.0 readme-renderer-28.0 requests-oauthlib-1.3.0 webencodings-0.5.1

[mailman@tignes ~]$ fetch https://gitlab.com/mailman/postorius/-/archive/master/postorius-master.tar.gz?path=example_project
[mailman@tignes ~]$ tar xvzf postorius-master.tar.gz\?path\=example_project 
x postorius-master-example_project/
x postorius-master-example_project/example_project/
x postorius-master-example_project/example_project/.gitignore
x postorius-master-example_project/example_project/README.rst
x postorius-master-example_project/example_project/__init__.py
x postorius-master-example_project/example_project/logs/
x postorius-master-example_project/example_project/logs/.keep
x postorius-master-example_project/example_project/mailman.cfg
x postorius-master-example_project/example_project/manage.py
x postorius-master-example_project/example_project/settings.py
x postorius-master-example_project/example_project/test_settings.py
x postorius-master-example_project/example_project/urls.py
x postorius-master-example_project/example_project/wsgi.py
[mailman@tignes ~]$ mv postorius-master-example_project/example_project postorius
[mailman@tignes ~]$ cd postorius
[mailman@tignes postorius]$ 

--- In settings.py I set
--- same reason as above.

[mailman@tignes postorius]$ python manage.py migrate
Operations to perform:
  Apply all migrations: account, admin, auth, contenttypes, django_mailman3, openid, postorius, sessions, sites, socialaccount
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying account.0001_initial... OK
  Applying account.0002_email_max_length... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK


  Applying sites.0002_alter_domain_unique... OK
  Applying socialaccount.0001_initial... OK
  Applying socialaccount.0002_token_max_lengths... OK
  Applying socialaccount.0003_extra_data_default_dict... OK

[mailman@tignes postorius]$ python manage.py createsuperuser
Username (leave blank to use 'mailman'): 
Email address: postmaster@restart.be
Password (again): 
This password is too short. It must contain at least 8 characters.
This password is too common.
Bypass password validation and create user anyway? [y/N]: y
Superuser created successfully.

[mailman@tignes postorius]$ python manage.py runserver
INFO: Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
December 13, 2020 - 14:19:12
Django version 2.2.17, using settings 'settings'
Starting development server at
Quit the server with CONTROL-C.

And now I have access to the web interface:

Then I create a list and then convert the mailman2 config with:

[mailman@tignes postorius]$ cd
[mailman@tignes ~]$ mailman import21 xxxx@example.com /usr/local/mailman2/lists/xxxx

And now I can wait for the official ports with mail mailman running...

Comment 20 Dan Langille freebsd_committer 2020-12-13 14:46:11 UTC
(In reply to Henri Hennebert from comment #19)

Did you see https://www.freshports.org/www/py-postorius ?

I mention it because I see you using pip to install it.  Was that a required step?

Thank you for that work. It is very useful and appreciated.
Comment 21 Henri Hennebert 2020-12-13 15:04:42 UTC
(In reply to Dan Langille from comment #20)

I miss this port. I will give it a try.

I use pip under /usr/local/mailman  because I was not aware of the postorius port and was shy to "pollute" /usr/local/lib/python3.7.

Thank you for the hint!
Comment 22 Henri Hennebert 2020-12-13 15:24:47 UTC
(In reply to Dan Langille from comment #20)

I install www/py-postorius and then:

[mailman@tignes ~]$ mv .cache/ __cache
[mailman@tignes ~]$ mv .local/ __local
To avoid using local pip.

Then AFAIK the use of example_project is needed:

[mailman@tignes ~]$ cd postorius/
[mailman@tignes postorius]$ python manage.py runserver
INFO: Watching for file changes with StatReloader
Performing system checks...

/usr/local/lib/python3.7/site-packages/readme_renderer/markdown.py:44: UserWarning: Markdown renderers are not available. Install 'readme_renderer[md]' to enable Markdown rendering.
System check identified no issues (0 silenced).
December 13, 2020 - 15:15:44
Django version 2.2.17, using settings 'settings'
Starting development server at
Quit the server with CONTROL-C.

And all is running well.

Now, postorius must be installed to be available for internet connections.
Comment 23 Henri Hennebert 2020-12-14 15:20:40 UTC
(In reply to Henri Hennebert from comment #22)
I to access the postorius with apache24

I first install www/mod_wsgi4

In /usr/local/etc/apache24/Includes/lists.conf:

<VirtualHost x.x.x.x:80>
ServerName <server fqdn>
LogFormat combined
TransferLog "| /usr/local/sbin/cronolog /var/log/httpd/lists/%Y/%m/access-%d.log"
ErrorLog "| /usr/local/sbin/cronolog /var/log/httpd/lists/error-%Y-%m.log"
DocumentRoot "/usr/local/mailman/postorius"

Alias /static /usr/local/lib/python3.7/site-packages/postorius/static
<Directory "/usr/local/lib/python3.7/site-packages/postorius/static">
    Require all granted

WSGIScriptAlias / /usr/local/mailman/postorius/wsgi.py
<Directory "/usr/local/mailman/postorius">
    <Files wsgi.py>
    Require all granted

[root@tignes local]# chmod 775 mailman/postorius 
[root@tignes local]# chmod 664 mailman/postorius/postorius.db 
[root@tignes local]# chgrp www mailman/postorius mailman/postorius/postorius.db

in /usr/local/mailman/postorius/settings.py:

ALLOWED_HOSTS = ['server fqdn>']

and I replace /usr/local/mailman/postorius/wgsi.py with attached file
Comment 24 Henri Hennebert 2020-12-14 15:24:09 UTC
Created attachment 220549 [details]
Comment 25 C. Bern 2020-12-25 14:40:49 UTC
Has anyone heard anything back from the maintainer of mailman3 about the subopimal  installation or non-installation of his port?
I sent him an email 2 weeks ago, but not heard back.
Comment 26 Dan Langille freebsd_committer 2020-12-25 15:08:51 UTC
(In reply to C. Bern from comment #25)

Why do you ask?

I think keeping everything public and on a PR, such as this one, helps to document the process and keeps everyone informed.
Comment 27 C. Bern 2020-12-25 15:21:33 UTC
(In reply to Dan Langille from comment #26)

.. because it would be nice that your and H. Henniebert's nice work are integrated in a new revision of this port to make thing smoother for others.
Comment 28 Dan Langille freebsd_committer 2020-12-25 15:48:20 UTC
(In reply to C. Bern from comment #27)
I think that is going to come from us.
Comment 29 Henri Hennebert 2020-12-26 12:23:06 UTC
(In reply to Henri Hennebert from comment #23)

To mimic the the configuration at https://lists.mailman3.org/
I rename /usr/local/mailman/postorius to /usr/local/mailman/mailman3

my config of apache24 is now:

<VirtualHost x.x.x.x:80>
ServerName <server fqdn>
LogFormat combined
TransferLog "| /usr/local/sbin/cronolog /var/log/httpd/lists/%Y/%m/access-%d.log"
ErrorLog "| /usr/local/sbin/cronolog /var/log/httpd/lists/error-%Y-%m.log"
DocumentRoot "/usr/local/mailman/mailman3"

Alias /static /usr/local/mailman/mailman3/static
<Directory "/usr/local/mailman/mailman3/static">
    Require all granted

WSGIScriptAlias / /usr/local/mailman/mailman3/wsgi.py
<Directory "/usr/local/mailman/mailman3">
    <Files wsgi.py>
    Require all granted

Then I add textproc/rubygem-sass and textproc/py-cmarkgfm

Next I install hyperkitty (there is no ports so I use pip):

pip install hyperkitty

After some try and errors I have my settings.py updated (see attachment)

[root@tignes ~]# su -l mailman
[mailman@tignes ~]$ cd mailman3
[mailman@tignes mailman3]$ django-admin migrate --pythonpath /usr/local/mailman/mailman3 --settings settings
Operations to perform:
  Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, openid, postorius, sessions, sites, socialaccount
Running migrations:

Then I MUST install the /usr/local/mailman/mailman3/static directory to avoid problems with the css:

[mailman@tignes mailman3]$ python manage.py collectstatic

and then I restart apache24 and all is running well: archives are updated.

NOTE: I try to recollect everything from my last 2 weeks :-/
so maybe I forget a step or two...
Comment 30 Henri Hennebert 2020-12-26 12:29:34 UTC
Created attachment 220945 [details]
The settings
Comment 31 Henri Hennebert 2020-12-26 12:44:17 UTC
(In reply to Henri Hennebert from comment #29)
Houps I forget this update to mailman:

pip install mailman-hyperkitty

and some updates in /usr/local/mailman/etc:


base_url: http://<fqdn>/archives                                                                                          
api_key: SecretArchiverAPIKey

and mailman.cfg:

port: 8001                                                                                                                          
use_https: no                                                                                                                       
admin_user: restadmin                                                                                                               
admin_pass: restpass                                                                                                                
api_version: 3.1                                                                                                                    
enable: yes                                                                                                                         
class: mailman_hyperkitty.Archiver                                                                                                  
enable: yes                                                                                                                         
configuration: /usr/local/mailman/etc/mailman-hyperkitty.cfg                                                                        

Of course:

service mailman restart
Comment 32 Henri Hennebert 2020-12-28 13:59:29 UTC
(In reply to Henri Hennebert from comment #31)

It should be useful to add this section to /usr/local/mailman/etc/mailman.cfg:

# This address is the "site owner" address.  Certain messages which must be
# delivered to a human, but which can't be delivered to a list owner (e.g. a
# bounce from a list owner), will be sent to this address.  It should point to
# a human.
site_owner: postmaster@restart.be

and when running mailman from the command line it is useful to replace in the port /usr/ports/mail/mailman3/files/patch-src_mailman_core_initialize.py:




(see attachment)

and for now do it by hand in /usr/local/lib/python3.7/site-packages/mailman/core/initialize.py
Comment 33 Henri Hennebert 2020-12-28 14:01:48 UTC
Created attachment 221053 [details]
patch to mail/mailman3/files/patch-src_mailman_core_initialize.py

So that mailman called from the command line work properly in all circonstances
Comment 34 Einar Bjarni Halldórsson 2021-01-04 21:50:51 UTC
We just migrated our mailman install to a new server running mailman3. I appreciate the work you've done and if I can help with anything I will.

I installed everything from ports, but installed hyperkitty with pip.
Comment 35 Marko Cupać 2021-01-18 11:56:32 UTC
Hi everyone,

I have just made my first mailman3 (suite) installation on FreeBSD work. Took me some time and nerves :) but it works.

As for mailman itself, it almost works just by installing mail/mailman3. As Dan said on his blog, it is throwing some errors on `service mailman start`, which are resolved by installing additional packages which are undeclared dependencies. All the ports/packages are already in official ports tree. Latest devel/py-zope.interface 5.2.0 is trivial to compile from ports (poudriere), just needs latest revision in Makefile + make makesum.

As for web suite (postorius + hyperkitty), I solved it through pip in virtualenv. It also needed textproc/sassc globally installed with pkg, and also hyperkitty (or was it mailman-hyperkitty?) installed globally with pip as there is no port as of yet.

Current verdict: it works and it is possible to set up. But it is quite complicated from admin's point of view, as it needs compiling custom ports, mixing pip and pkg for installation of base python packages, and additional virtualenv for web suite. Also it takes at least 3 different config files.

From my (admin's) point of view, it would be best if mailman3 was ported to FreeBSD as three (meta) ports, similar to:
mailman (core)
mailman (web) 
mailman (full) - core + web

All should be installable with pkg, and all should be started from rc.conf.

I'd be glad to help if someone told me how.