Bug 209649 - archivers/py-borgbackup: py34-borgbackup-1.0.2 installs py27 dependencies
Summary: archivers/py-borgbackup: py34-borgbackup-1.0.2 installs py27 dependencies
Status: Closed Overcome By Events
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Mark Felder
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2016-05-19 19:02 UTC by aeuii
Modified: 2018-03-16 04:49 UTC (History)
8 users (show)

See Also:
jjuanino: maintainer-feedback+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description aeuii 2016-05-19 19:02:38 UTC
When I install py-borgbackup using portmaster it installs py27- packages/ports, however install itself as a py34- package.  When I try to run it, it throws an exception:

$ borg
Traceback (most recent call last):
  File "/usr/local/bin/borg", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python3.4/site-packages/pkg_resources/__init__.py", line 3141, in <module>
    @_call_aside
  File "/usr/local/lib/python3.4/site-packages/pkg_resources/__init__.py", line 3127, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python3.4/site-packages/pkg_resources/__init__.py", line 3154, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python3.4/site-packages/pkg_resources/__init__.py", line 640, in _build_master
    ws.require(__requires__)
  File "/usr/local/lib/python3.4/site-packages/pkg_resources/__init__.py", line 941, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python3.4/site-packages/pkg_resources/__init__.py", line 828, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'msgpack-python>=0.4.6' distribution was not found and is required by borgbackup



freebsd-version: 10.3-RELEASE-p3 & up-to-date port tree
Comment 1 Jose G. Juanino 2016-05-20 12:16:58 UTC
Show the output of


pkg info -d py34-borgbackup-1.0.2




In my system:


# pkg info -d py34-borgbackup-1.0.2
py34-borgbackup-1.0.2:
        liblz4-131
        py34-setuptools34-20.0
        py34-msgpack-python-0.4.6
        libressl-2.2.6
        python34-3.4.4_1
Comment 2 Jose G. Juanino 2016-05-20 12:19:58 UTC
It sounds to me you had previously installed some 27 version of 
devel/py-setuptools_scm or devel/py-msgpack-python ...
Comment 3 aeuii 2016-05-24 20:44:09 UTC
(In reply to jjuanino from comment #1)

pkg info -d py34-borgbackup-1.0.2
py34-borgbackup-1.0.2:
	liblz4-131
	openssl-1.0.2_12
	py34-setuptools34-20.0
	python34-3.4.4_2
Comment 4 aeuii 2016-05-24 20:48:32 UTC
(In reply to jjuanino from comment #2)

pkg info | grep setuptools
py27-setuptools27-20.0         Python packages installer
py34-setuptools34-20.0         Python packages installer
py34-setuptools_scm-1.10.1     Setuptools plugin to manage your versions by scm tags


pkg info | grep msgpack-python
py27-msgpack-python-0.4.7      MessagePack (de)serializer for Python



devel/py-msgpack-python is pulled by portmaster:

===>>> The following actions were performed:
	Installation of devel/py-msgpack-python (py27-msgpack-python-0.4.7)
	Installation of archivers/py-borgbackup (py34-borgbackup-1.0.2)


Interestingly, there's no package py34-borgbackup-1.0.2, so I'm not able to `pkg install py34-borgbackukp' and have to use portmaster.
Comment 5 Jose G. Juanino 2016-05-24 21:16:01 UTC
Hi,

I think that portmaster is doing something wrong. I have reproduced the issue, getting the following:


# portmaster archivers/py-borgbackup

===>>> Port directory: /usr/ports/archivers/py-borgbackup

===>>> Gathering distinfo list for installed ports

===>>> Launching 'make checksum' for archivers/py-borgbackup in background
===>>> Gathering dependency list for archivers/py-borgbackup from ports
===>>> Starting dependency check
===>>> Checking dependency: devel/py-setuptools_scm
===>>> Launching child to install devel/py-setuptools_scm

===>>> archivers/py-borgbackup >> devel/py-setuptools_scm (1/1)

===>>> Port directory: /usr/ports/devel/py-setuptools_scm

===>>> Launching 'make checksum' for devel/py-setuptools_scm in background
===>>> Gathering dependency list for devel/py-setuptools_scm from ports
===>>> Starting dependency check
===>>> Checking dependency: devel/py-setuptools27  <--- WRONG!


borgbackup requires devel/py-setuptools_scm, but I think that portmaster is unable to detect the right python version. Instead, it assumes that the default ports infraestructure python version is the right one, but in this corner case is not.

Workaround: install py-borbackup by mean of "make install clean" and let me know if works. Thanks for tell about this issue, I will ask in ports@ mailing list.
Comment 6 Jose G. Juanino 2016-05-25 10:01:09 UTC
Hi, please take a look at:

https://lists.freebsd.org/pipermail/freebsd-ports/2016-May/103308.html

It is a known issue. I close the bug report. Regard.
Comment 7 aeuii 2016-05-25 14:36:15 UTC
(In reply to jjuanino from comment #6)

Thanks a lot for the prompt reaction.

The `make install clean' workaround works fine.
Comment 8 freebsd 2017-03-26 21:10:29 UTC
I struggled with this as of py35-borgbackup-1.0.10. Spinning up a fresh jail and requesting a build of devel/py-borgbackup results in a proper installation using only python3.5 components.

However, on a server that's been used for while, a request to build devel/py-borgbackup results in a mix of python2.7 and python3.5 packages being installed (note that no python-related packages are installed at the time that the build is requested). Borg then fails at runtime because py27-msgpack-python is installed, not py35-msgpack-python. However, adding the following to /etc/make.conf prior to the build causes it to build properly.

PYTHON_VERSION=		3.5
PYTHON_DEFAULT_VERSION=	3.5
Comment 9 freebsd 2017-03-26 21:18:29 UTC
I have experienced this on one of my systems as of py35-borgbackup-1.0.10--with a mix of py27 anbd py35 packages being installed when building the py-borgbackup port (and the "borg" command then failing at runtime because py27-msgpack-python was installed rather than py35-msgpack-python). However, adding:

PYTHON_VERSION=		3.5
PYTHON_DEFAULT_VERSION=	3.5

to /etc/make.conf causes the correct packages to be installed.

Furthermore, building on a fresh jail results in a proper install, with only py35 packages being installed.
Comment 10 Zach Leslie freebsd_committer freebsd_triage 2017-07-04 18:45:36 UTC
Now that python3.6 is the default python3 version, I've removed the DEFAULT_VERSIONS from my make.conf for python and am running into a similar version related build problem in poudriere.  If the port requires python3, I would expect that it just use python3 instead of looking at the default version of python, which is still 2.7, but that doesn't seem to be the case.

Can an adjustment to the port be made now that 3.6 is the default python3 version?
Comment 11 Jose G. Juanino 2017-07-04 20:17:43 UTC
Hi, this issue seems to be related to poudriere, as I am unable to reproduce it in my environment:

# cat /etc/make.conf
DEFAULT_VERSIONS+=ssl=libressl

# pkg info -x msgpack
py27-msgpack-python-0.4.7
py35-msgpack-python-0.4.7
py36-msgpack-python-0.4.7

# pwd
/usr/ports/archivers/py-borgbackup

# make install clean
....
(builds fine)

# borg --version
borg 1.0.10

Sorry, I am unable to address your issue, unless some poudriere skilled guy helps.
Comment 12 Zach Leslie freebsd_committer freebsd_triage 2017-07-05 14:26:16 UTC
The issue I'm seeing could be related to poudriere, but from what I can tell, it would only be due to respecting the 'IGNORE' line in the Makefile.  It seems that I am matching that condition since PYTHON_DEFAULT is 2.7 without adjustment.

The 'make install' you've executed might skip this 'IGNORE' line?  I'm unsure, but it still seems that the python detection or usage could use a slight improvement.
Comment 13 Mark Felder freebsd_committer freebsd_triage 2017-07-23 14:20:09 UTC
assigning to maintainer
Comment 14 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-24 01:54:49 UTC
The root cause contributing conditions are:

- archivers/py-borgbackup USES=python:3.x+ (the 3.x+ bit)
- archivers/py-borgbackup incorrectly *_DEPENDS on py-* (not py3-*) versions of dependencies.
- poudriere has a DEPENDS_ARGS issue[1][2]

Notes:

- Existing py3-* ports are workarounds until ports variants/flavors (ugly name) support is added to the tree.

- A workaround for having to create py3-* ports was added to a recent poudriere{-devel} version to automatically build py3-* variants when necessary. 

The official freebsd package repositories have not been updated (yet) to this poudriere version (pending exp-runs, i dont believe there's a tracking bug yet).

This is fundamentally a python framework/poudriere issue, manifest in a certain class of python ports/packages in certain conditions.

archivers/py-borgbackup can be fixed by *_DEPENDS'ing on py3- versions of its dependencies (and their dependencies).

[1] https://github.com/freebsd/poudriere/issues/259
[2] https://github.com/freebsd/poudriere/pull/483
Comment 15 Kubilay Kocak freebsd_committer freebsd_triage 2017-07-24 01:56:41 UTC
Note also this does not preclude additions/other issues/bugs in portmaster
Comment 16 Zach Leslie freebsd_committer freebsd_triage 2017-10-08 04:47:00 UTC
As a user of borgbackup, I'd like to request the suggested updates to the dependency to the port.  Would the maintainer be willing to make these adjustments please?
Comment 17 Tobias Kortkamp freebsd_committer freebsd_triage 2018-03-14 12:28:24 UTC
Is this still a problem now? I'd think that Python flavors would solve this.
Comment 18 aeuii 2018-03-14 13:21:32 UTC
(In reply to Tobias Kortkamp from comment #17)

not a problem any more