Bug 243610

Summary: sysutils/py-salt failed to build with python 3.8
Product: Ports & Packages Reporter: Tommy P <tommyhp2>
Component: Individual Port(s)Assignee: Ben Woods <woodsb02>
Status: Closed FIXED    
Severity: Affects Some People CC: christer.edwards, freebsd, koobs, python, tommyhp2, w.schwarzenfeld, woodsb02
Priority: --- Flags: w.schwarzenfeld: maintainer-feedback? (christer.edwards)
koobs: merge-quarterly?
Version: Latest   
Hardware: Any   
OS: Any   
See Also: https://github.com/saltstack/salt/issues/55029
Description Flags
patch for devel/py-ioflo to support sysutils/py-salt with python 3.x
patch for net/py-raet to support sysutils/py-salt with python 3.x none

Description Tommy P 2020-01-26 11:11:08 UTC
Salt fails to build with python 3.8.  I've inquired with upstream regarding possibility of fix for FreeBSD.

===>  Configuring for py38-salt-2019.2.2_1
Traceback (most recent call last):
  File "/wrkdirs/usr/ports/sysutils/py-salt/work-py38/salt-2019.2.2/salt/version.py", line 15, in <module>
    from platform import linux_distribution as _deprecated_linux_distribution
ImportError: cannot import name 'linux_distribution' from 'platform' (/usr/local/lib/python3.8/platform.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "setup.py", line 137, in <module>
    exec(compile(open(SALT_VERSION).read(), SALT_VERSION, 'exec'))
  File "/wrkdirs/usr/ports/sysutils/py-salt/work-py38/salt-2019.2.2/salt/version.py", line 22, in <module>
    from distro import linux_distribution
ModuleNotFoundError: No module named 'distro'
*** Error code 1

make: stopped in /usr/ports/sysutils/py-salt


Also, devel/py-ioflo and net/py-raet fail to build when RAET option selected:

===> The following configuration options are available for py38-salt-2019.2.2_1:
     AWS=on: Install dependencies required for Amazon Web Services
     RAET=on: Install dependencies required for RAET transport
     TCP=on: Install dependencies required for TCP transport
     ZEROMQ=on: Install dependencies required for ZeroMQ transport
===> Use 'make config' to modify these settings
Comment 1 Tommy P 2020-01-26 11:12:11 UTC
Forgot to include link to upstream bug report:

Comment 2 Tommy P 2020-01-26 11:13:26 UTC
Created attachment 211056 [details]
patch for devel/py-ioflo to support sysutils/py-salt with python 3.x
Comment 3 Tommy P 2020-01-26 11:14:25 UTC
Created attachment 211057 [details]
patch for net/py-raet to support sysutils/py-salt with python 3.x
Comment 4 Tommy P 2020-01-27 00:47:13 UTC
Perhaps mark port broken temporarily for python 3.8 since there are other issues too?

Comment 5 Kubilay Kocak freebsd_committer freebsd_triage 2020-01-28 03:27:42 UTC
(In reply to Tommy P from comment #4)

The more precise fix is to limit USES=python:<version-spec> to only list Python versions upstream explicitly supports/tests against. Since upstream only just added Python 3.8 to their QA process and there are this and other Python 3.8 issues, the more appropriate interim change is to set USES=python:<version-spec> to 2.7-3.7

@Christer Are you OK with this change in the meantime until an upstream fix/resolution is obtained that we can use?

If Salt on the other hand does expect/claim 3.8 support for the current port version, then presumably there is a bug/change that we can make, such as porting and conditionally dependending on the the distro module?
Comment 6 Christer Edwards 2020-02-04 17:11:10 UTC
Let's pin between supported versions until upstream support improves.
Comment 7 Kubilay Kocak freebsd_committer freebsd_triage 2020-02-07 06:47:31 UTC
We need a single patch that fixes py-salt (to correctly/accurately declare Python support) and any consumers that need to be fixed.

There is no explanation for py-ioflo py-raet changes and I have not verified whether or not (those or any other) consumers (depending on py-salt) need their <version-spec> changed, fixed or otherwise altered, or what versions of Python they support. 

This needs to be verified and the rationale described in comments.
Comment 8 Tommy P 2020-02-07 07:19:24 UTC
(In reply to Kubilay Kocak from comment #7)
When I tried to build sysutils/py-salt with python 3.7 and the selected option:

RAET=on: Install dependencies required for RAET transport

It failed because the two dependent ports net/py-raet and devel/py-ioflo as seen sysutils/py-salt/Makefile:

RAET_RUN_DEPENDS=       ${PYTHON_PKGNAMEPREFIX}libnacl>=1.0.0:security/py-libnacl@${PY_FLAVOR} \
                        ${PYTHON_PKGNAMEPREFIX}ioflo>=1.1.7:devel/py-ioflo@${PY_FLAVOR} \

were configured for python 2.7 in the Makefiles at the time of this report.  The only way I could get successful build was make the changes as seen in the patches.  As for other ports using devel/py-ioflo and/or net/py-raet, I've found only sysutils/py-salt in the entire ports tree:

# grep -Rn 'py-raet' * |grep Makefile
net/Makefile:1106:    SUBDIR += py-raet
net/py-raet/Makefile:2:# $FreeBSD: head/net/py-raet/Makefile 483068 2018-10-27 00:42:41Z swills $
net/py-raet/Makefile.ori:2:# $FreeBSD: head/net/py-raet/Makefile 483068 2018-10-27 00:42:41Z swills $
sysutils/py-salt/Makefile:77:                   ${PYTHON_PKGNAMEPREFIX}raet>=0.6.0:net/py-raet@${PY_FLAVOR}

# grep -Rn 'py-ioflo' *|grep Makefile
devel/Makefile:4504:    SUBDIR += py-ioflo
devel/py-ioflo/Makefile:2:# $FreeBSD: head/devel/py-ioflo/Makefile 483079 2018-10-27 00:55:31Z swills $
devel/py-ioflo/Makefile.ori:2:# $FreeBSD: head/devel/py-ioflo/Makefile 483079 2018-10-27 00:55:31Z swills $
sysutils/py-salt/Makefile:76:                   ${PYTHON_PKGNAMEPREFIX}ioflo>=1.1.7:devel/py-ioflo@${PY_FLAVOR} \

Currently, sysutils/py-salt is configured to build all versions of python (except 3.8 as I've found out and upstream does not fully support 3.8 yet) except with option RAET (as I've also found out).
Comment 9 Kubilay Kocak freebsd_committer freebsd_triage 2020-02-07 10:39:23 UTC
(In reply to Tommy P from comment #8)

Thank you for the explanation Tommy
Comment 10 Peter 2020-02-20 16:06:02 UTC
fyi - even latest version [3000] does not support 3.8:

Comment 11 commit-hook freebsd_committer 2020-03-07 03:45:08 UTC
A commit references this bug:

Author: woodsb02
Date: Sat Mar  7 03:44:24 UTC 2020
New revision: 527918
URL: https://svnweb.freebsd.org/changeset/ports/527918

  sysutils/py-salt: python3.8 is not supported yet


  PR:		243610
  Reported by:	Tommy P <tommyhp2@gmail.com>
  Approved by:	Christer Edwards <christer.edwards@gmail.com> (maintainer)

Comment 12 Ben Woods freebsd_committer 2020-03-07 04:13:15 UTC
Closing this bug, as the sysutils/py-salt port has been updated to correctly declare that python 3.8 is not supported.

Separate bugs have been opened to address python support for devel/py-ioflo and net/py-raet
Comment 13 Kubilay Kocak freebsd_committer freebsd_triage 2020-03-07 13:53:44 UTC

 - Assign to committer that resolved.
 - Re-open requesting MFH (quarterly is assumed affected also?)
Comment 14 commit-hook freebsd_committer 2020-03-08 23:17:21 UTC
A commit references this bug:

Author: woodsb02
Date: Sun Mar  8 23:16:22 UTC 2020
New revision: 528084
URL: https://svnweb.freebsd.org/changeset/ports/528084

  net/py-raet: Update USES=python to include python3

  Python3 support was added in raet version 0.5.0:

  Exactly which python 3 versions are supported / known to work will be
  discussed here:

  We know at least python 3.5 and 3.6 are supported:

  PR:		244653, 243610
  Submitted by:	Tommy P <tommyhp2@gmail.com>
  Approved by:	portmgr (blanket: dependency fix, ports (python) compliance)
  MFH:		2020Q1

Comment 15 Ben Woods freebsd_committer 2020-05-16 06:00:34 UTC
Committed, and MFH not necessary since 2020Q2 ports quarterly branch now exists - thanks for your input everyone