Bug 241913 - net-mgmt/netbox: conflict with py-salt: py-pycrypto conflicts with py-pycryptodome
Summary: net-mgmt/netbox: conflict with py-salt: py-pycrypto conflicts with py-pycrypt...
Status: Closed Not A Bug
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Kai Knoblich
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-11-12 10:58 UTC by O. Hartmann
Modified: 2019-12-03 04:46 UTC (History)
5 users (show)

See Also:


Attachments
netbox-switch-to-py-pycryptodomex.patch (5.85 KB, patch)
2019-11-13 10:23 UTC, Kai Knoblich
kai: maintainer-approval+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description O. Hartmann 2019-11-12 10:58:30 UTC
It seems that in the vain of several py36-packages a conflict arises between net-mgmt/netbox and sysutils/py-salt. Having installed sysutils/py-salt@py36 on all to maintain systems, the installation of net-mgmt/netbox requires to to delete sysutils/py36-salt and security/py36-pycrypto (in our case), obviously triggered by the conflict between the packages

security/py-pycrypto and
security/py-pycryptodome

(FLAVOR set to py36 by default):

[...]
All repositories are up to date.
Checking integrity... done (1 conflicting)
  - py36-pycryptodome-3.9.0 conflicts with py36-pycrypto-2.6.1_3 on /usr/local/lib/python3.6/site-packages/Crypto/Cipher/AES.py
Checking integrity... done (0 conflicting)
The following 65 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
        py36-pycrypto-2.6.1_3
        py36-salt-2019.2.2_1

The packages were build via a poudriere-based builder on local site.
Comment 1 Kai Knoblich freebsd_committer freebsd_triage 2019-11-13 08:20:05 UTC
(In reply to O. Hartmann from comment #0)

Thank you for the report. It's indeed an interesting case of a Python package collision.

Here are some quick facts about the related Python packages:

security/py-pycrypto:
~~~~~~~~~~~~~~~~~~~~~
The project seems to be dead as the last update was in 2013. (see Github issue #173)

security/py-pycryptodome: 
~~~~~~~~~~~~~~~~~~~~~~~~~
This a fork of security/py-pycrypto which is actively maintained and can be used as a drop-in replacement for security/py-pycrypto.

The trade off: security/py-pycrypto and security/py-cryptodome cannot coexist because both use the same package name (= "Crypto").

security/py-pycryptodomex: 
~~~~~~~~~~~~~~~~~~~~~~~~~~
Like security/py-cryptodome (= same upstream) but uses a different package name (= "Cryptodome" instead "Crypto") thus it can coexist with security/py-pycrypto.


Let's go over to NetBox and Salt:

net-mgmt/netbox:
~~~~~~~~~~~~~~~~
Upstream switched from security/py-pycrypto to security/py-pycryptodome in 2017. (see GitHub issue #1527)

sysutils/py-salt:
~~~~~~~~~~~~~~~~~
The requirement for security/py-pycrypto is pulled in via the ZEROMQ or TCP options.

The ZEROMQ option is also set as default one and reflects the actual requirements by upstream as given in sysutils/py-salt's 'requirements/zeromq.txt':

> # PyCrypto has issues on Windows, while pycryptodomex does not
> pycrypto>=2.6.1; sys.platform != 'win32'
> pycryptodomex; sys.platform == 'win32'

There was a PR (see Github pull request #45971) in 2018 for the ZeroMQ dependency to use security/py-cryptodome in favor of security/py-pycrypto.

But that PR was then closed after some discussion because upstream had tried it a while ago which led to some problems due a bug with a specific version of security/py-pycryptodome.


Conclusion:
~~~~~~~~~~~
Because sysutils/py-salt is IMHO a pretty complex and frequently used port that need good care a patching of its 'requirements/zeromq.txt' is not trivial.

I did also some quick comparison of the code:

net-mgmt/netbox:
~~~~~~~~~~~~~~~~
> $ grep -r -e Crypto ./netbox-2.6.7/* |wc -l
> 	8

sysutils/py-salt:
~~~~~~~~~~~~~~~~~
> $ grep -r -e Crypto ./salt-2019.2.2/ |wc -l
>     244

So I'll create a PR for NetBox that switches from security/py-cryptodome to security/py-cryptodomex as it seems the best (and quickest) option to solve the issue.
Comment 2 Kai Knoblich freebsd_committer freebsd_triage 2019-11-13 10:23:06 UTC
Created attachment 209127 [details]
netbox-switch-to-py-pycryptodomex.patch

Attached is a patch that switches all relevant code to security/py-pycryptodomex .

My tests were successful so far:

- Login/Logoff -> OK
- Unlock/Lock secrets via private key -> OK
- Create new user key pair -> OK
- Active new user key pair -> OK
- Unlock/Lock secrets with new user key pair -> OK

I'll upstream that patch with some additional changes (related to documentation) as soon as possible.
Comment 3 Kai Knoblich freebsd_committer freebsd_triage 2019-11-17 13:10:23 UTC
Upstream of Netbox closed my PR without merging it. I asked again if the decision could be reconsidered but I guess the odds aren't very high.

I'll use then the set of patches attached in this PR for a while to get the issue with the Python package collision resolved. Even if it's means additional QA work in the future with every update of Netbox.

From a long-term perspective the developers of Salt should really try again to switch fully from security/py-pycrypto to security/py-cryptodome or security/py-cryptodomex.

(In reply to O. Hartmann from comment #0)

Did you already have the opportunity to test the attached patch? As already mentioned in comment #2 there shouldn't be any problems with switching from `pycryptodome` to `pycryptodomex`.
Comment 4 Kai Knoblich freebsd_committer freebsd_triage 2019-11-19 08:51:07 UTC
Comment on attachment 209127 [details]
netbox-switch-to-py-pycryptodomex.patch

Pull my proposed patch for net-mgmt/netbox back as it wasn't accepted by upstream and I won't have always the time to do full QA against an unsupported version of net-mgmt/netbox.
Comment 5 Kai Knoblich freebsd_committer freebsd_triage 2019-11-19 08:54:24 UTC
After some thinking and internal discussions I close this bug as "not a bug" because there's little we can do to resolve the issue from a Ports related perspective.

There's still some confusion (in the Ports and Python world) about the existence of both security/py-pycryptodome and security/py-pycryptodomex ports.

While security/py-pycryptodome is meant as a drop-in replacement for security/py-pycrypto that seems to be no longer 100% true due some API incompatibilities (see also issue #89).

And security/py-pycryptodomex seems to be hardly used in the Python world at all.

To keep it short:

- The problem can only be resolved at upstream level.
- Upstream of security/py-pycryptodome should start using it's own namespace (= "Cryptodome" instead "Crypto") and make the security/py-pycryptodomex package obsolete.

OR

- Upstream of sysutils/py-salt tries again to fully abandon security/py-pycrypto (remove 'pycrypto' from the requirements for ZeroMQ).
Comment 6 Trix Farrar 2019-11-19 19:26:40 UTC
According to ##52674 (https://github.com/saltstack/salt/issues/52674 - "PyCryptodome as replacement for PyCrypto", Salt _can_ use security/py-pycryptodome, but doesn't do to pycrypto being mentioned in the requirements.txt file for ZeroMQ (even though it may or may not use it directly).  From the issue text:

... and there is a picking[sic] order as to which package is used as follows:

PyCrypto - basic level
PyCryptodomx - preferred over PyCrypto if installed
M2Crypto - preferred over PyCryptodome and PyCrypto if installed

Although 52674 is closed, David Murphy (dmurphy18) goes on to talk about how this will likely be revisited once Python2 has reached end-of-life (in a little under a month and a half at this writing) -- around the Neon release (probably in 2020).