Bug 223359 - security/py-fail2ban: after upgrade to 0.10.x, fail2ban no longer works with python 3
Summary: security/py-fail2ban: after upgrade to 0.10.x, fail2ban no longer works with ...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-ports-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-11-01 06:09 UTC by Ari Suutari
Modified: 2018-02-03 09:03 UTC (History)
4 users (show)

See Also:
theis: maintainer-feedback+


Attachments
patch (731 bytes, text/plain)
2018-01-24 15:35 UTC, theis
theis: maintainer-approval+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ari Suutari 2017-11-01 06:09:43 UTC
Fail2ban port doesn't produce a working installation when using python 3.6.
Attempt to start it results in following errors:

Traceback (most recent call last):
  File "/usr/local/bin/fail2ban-client", line 34, in <module>
    from fail2ban.client.fail2banclient import exec_command_line, sys
  File "/usr/local/lib/python3.6/site-packages/fail2ban/client/fail2banclient.py", line 39, in <module>
    from ..server.utils import Utils
  File "/usr/local/lib/python3.6/site-packages/fail2ban/server/utils.py", line 54, in <module>
    for name, num in signal.__dict__.iteritems() if name.startswith("SIG"))
AttributeError: 'dict' object has no attribute 'iteritems'


Older port using 0.9.x used to work just fine.

There seems to be a fail2ban-2to3 script in fail2ban github, the port
should probably run it if building with python 3. At least running it manually over installed files makes things work again.
Comment 1 theis 2017-11-01 08:34:30 UTC
Yes, there are differences between Python 2 and 3, among them the method iteritems for dictionaries was dropped in favour of items. Python includes a script called 2to3 which takes care of such changes, fail2ban-2to3 is a wrapper around it. I see that I don't include that script in the installation though it did exist in 0.9.x as well.  
I will update the port later to include that script.
Comment 2 theis 2018-01-19 15:58:32 UTC
Addressed in patch for upgrade to 0.10.2 (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225317)
Comment 3 David O'Rourke 2018-01-24 00:33:22 UTC
With 0.10.2 the build fails in the patch phase. It looks like the python${FLAVOUR} package isn't installed in time.

=======================<phase: patch          >============================
===>  Patching for py36-fail2ban-0.10.2
===>  Applying FreeBSD patches for py36-fail2ban-0.10.2
(cd /wrkdirs/usr/ports/security/py-fail2ban/work-py36/fail2ban-0.10.2/ && /usr/local/bin/2to3-3.6 --no-diffs --write --nobackups --fix=all bin/* fail2ban)
/bin/sh: /usr/local/bin/2to3-3.6: not found
*** Error code 127

Stop.
make: stopped in /usr/ports/security/py-fail2ban
Comment 4 theis 2018-01-24 10:11:17 UTC
I don't understand how this could happen and thus how to deal with it:
With USES=python python should be a prerequisite of fail2ban and should have been installed by the make system before attempting to build fail2ban.
Comment 5 David O'Rourke 2018-01-24 11:18:20 UTC
Looking at the Poudriere output, it looks like there's a `patch-depends` phase where the necessary Python should be installed in this case, while the regular `USES` that are set are installed after the `patch` phase in the `build-depends` phase.

It looks like this is controlled by the `PATCH_DEPENDS` setting in the port's Makefile (see `Mk/bsd.port.mk`). Hopefully setting Python as a dependency here will fix the problem.
Comment 6 theis 2018-01-24 12:06:23 UTC
Thank you for the explanation. Instead of adding a PATCH_DEPENDS I'll move the 2to3 step to post-build: it's as good as post-fetch and I won't need extra dependencies.

I cleaned my system from all python. After the patch step build will start making devel/py-setuptools@py36, but that fails because it is missing /usr/local/bin/python3.6 and python3 is not installed automatically. I couldn't even install it manually. 

In /etc/make.conf I have
DEFAULT_VERSIONS+= python=3.6

Without that line in /etc/make.conf the build will proceed to install py-setuptools@py27 and, as a dependency, python2.

Is there something wrong with bootstrapping python3?
Comment 7 theis 2018-01-24 15:35:57 UTC
Created attachment 190033 [details]
patch

To solve the dependency issue I moved the 2to3 stuff to post-build.
A patch is attached.
Comment 8 Ken 2018-01-24 21:22:58 UTC
I'm having the exact same issue.I hope this patch gets added to the ports system soon.
Comment 9 theis 2018-01-25 18:27:17 UTC
@mayhem: If you have python3 already installed the current version in the ports shall help you. This latest bugfix is for the situation if no python installation is present, which I guess is quite uncommon for fail2ban users.
Comment 10 Tobias Kortkamp freebsd_committer freebsd_triage 2018-01-25 18:42:28 UTC
This should be fixed in ports r459924.
Comment 11 Tobias Kortkamp freebsd_committer freebsd_triage 2018-02-02 07:04:04 UTC
The silence is deafening.

I'd really appreciate it if someone could confirm that this is indeed fixed
after ports r459924 and that applying this patch is no longer necessary.
Comment 12 theis 2018-02-02 07:53:33 UTC
With the current version of the Makefile I can install and start fail2ban on a bare system with python 3. I can't say if everything is working, my production system is python 2.
Comment 13 Ken 2018-02-02 08:25:34 UTC
(In reply to theis from comment #9)

I'm not sure if it was this patch or the previous one, but everything works fine here with Python 3.6
Comment 14 theis 2018-02-02 08:46:05 UTC
@mayhem: The previous patch, the upgrade to version 0.10.2, applied also the 2to3 stuff. The latest patch fixed the build issue when no python was yet installed.
Comment 15 Ken 2018-02-02 08:51:06 UTC
Ok, then I'm not sure if this patch works. I uninstalled Python 2 and then manually installed Python 3 before installing fail2ban, postfix-policyd, etc.
Comment 16 David O'Rourke 2018-02-02 09:26:25 UTC
Sorry about failing to reply here. I'm doing another poudriere run now with my Python preference set to 3.6. Will let you know shortly.
Comment 17 David O'Rourke 2018-02-02 10:16:39 UTC
I can confirm that (for me at least) fail2ban now builds properly inside poudriere while using Python 3. Thanks.
Comment 18 Tobias Kortkamp freebsd_committer freebsd_triage 2018-02-03 09:03:01 UTC
Thank you.  It looks like there is nothing left to do here then but
closing this bug.