Bug 267040 - devel/py-nbformat and devel/py-black incorrectly install files belonging to the test infrastructure
Summary: devel/py-nbformat and devel/py-black incorrectly install files belonging to t...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Kai Knoblich
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-10-14 08:46 UTC by Martin Birgmeier
Modified: 2022-11-05 18:40 UTC (History)
5 users (show)

See Also:
koobs: maintainer-feedback? (sunpoet)
kai: maintainer-feedback+
kai: merge-quarterly-


Attachments
py-black-fix-packaging-issues.patch (835 bytes, patch)
2022-10-20 21:05 UTC, Kai Knoblich
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Birgmeier 2022-10-14 08:46:32 UTC
Scenario:
- FreeBSD 12.3 latest patches
- latest ports
- upgrading using portmaster
- graphics/qgis is installed; this needs devel/py-nbformat which is also installed
- devel/py-black is installed

Result:
- Upgrading py-black from 22.8.0 to 22.10.0 fails with a conflict during install:

Installing py39-black-22.10.0...
pkg-static: py39-black-22.10.0 conflicts with py39-nbformat-5.6.1_1 (installs files into the same place).  Problematic file: /usr/local/lib/python3.9/site-packages/tests/__init__.py
*** Error code 1

Expected result:
- The install should succeed without conflicting files.

Analysis and suggested fix:
- It seems that devel/py-nbformat and devel/py-black both install files from the generic python test infrastructure.
- Most likely the staging part for python ports must be changed to exclude such files.

See also bug 266944.

-- Martin
Comment 1 Rainer Hurling freebsd_committer freebsd_triage 2022-10-14 08:54:01 UTC
See also bug #266944, comment #2.
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2022-10-14 09:41:43 UTC
Needs to be fixed upstream (patched locally via exclusion in setup.py, until then is fine)

This is another example of bug 262759, setting
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2022-10-14 09:43:19 UTC
Request feedback from black maintainer (also on Python)
Comment 4 Martin Birgmeier 2022-10-16 08:06:04 UTC
Thanks to commit 76670f6a4f8400c18f480f8b92dbf25cdc07eeea, py-black conflicting with py-nbformat is resolved. However, it seems that still files are installed which do not strictly belong to py-black:

[0]# pkg query %Fp py39-black-22.10.0_1 | egrep -v 'black|blib2to3'
/usr/local/lib/python3.9/site-packages/scripts/__init__.py
/usr/local/lib/python3.9/site-packages/scripts/__pycache__/__init__.cpython-39.opt-1.pyc
/usr/local/lib/python3.9/site-packages/scripts/__pycache__/__init__.cpython-39.pyc
/usr/local/lib/python3.9/site-packages/scripts/__pycache__/check_pre_commit_rev_in_example.cpython-39.opt-1.pyc
/usr/local/lib/python3.9/site-packages/scripts/__pycache__/check_pre_commit_rev_in_example.cpython-39.pyc
/usr/local/lib/python3.9/site-packages/scripts/__pycache__/check_version_in_basics_example.cpython-39.opt-1.pyc
/usr/local/lib/python3.9/site-packages/scripts/__pycache__/check_version_in_basics_example.cpython-39.pyc
/usr/local/lib/python3.9/site-packages/scripts/__pycache__/diff_shades_gha_helper.cpython-39.opt-1.pyc
/usr/local/lib/python3.9/site-packages/scripts/__pycache__/diff_shades_gha_helper.cpython-39.pyc
/usr/local/lib/python3.9/site-packages/scripts/__pycache__/fuzz.cpython-39.opt-1.pyc
/usr/local/lib/python3.9/site-packages/scripts/__pycache__/fuzz.cpython-39.pyc
/usr/local/lib/python3.9/site-packages/scripts/check_pre_commit_rev_in_example.py
/usr/local/lib/python3.9/site-packages/scripts/check_version_in_basics_example.py
/usr/local/lib/python3.9/site-packages/scripts/diff_shades_gha_helper.py
/usr/local/lib/python3.9/site-packages/scripts/fuzz.py
[0]# 

The same holds true for py-nbformat, which was already improved by commit d8495b59b7b816819070d2c8cb3acbe5b815ad7b. Again, there seem to be extraneous files:

[0]# pkg query %Fp py39-nbformat-5.6.1_2 | grep -v nbformat
/usr/local/bin/jupyter-trust
/usr/local/bin/jupyter-trust-3.9
[0]# 

-- Martin
Comment 5 Kai Knoblich freebsd_committer freebsd_triage 2022-10-20 21:05:41 UTC
Created attachment 237489 [details]
py-black-fix-packaging-issues.patch

The current version of devel/py-black is still broken as it has packaging issues. The games/anki port, which requires devel/py-black for build, also fails to build due this regression.

For instance if one tries to do a

> $ python3.9 -m black
> /usr/local/bin/python3.9: No module named black

or even try to run the script "black"

> $ black
> Traceback (most recent call last):
>   File "/usr/local/bin/black", line 33, in <module>
>     sys.exit(load_entry_point('black==22.10.0', 'console_scripts', 'black')())
>   File "/usr/local/bin/black", line 25, in importlib_load_entry_point
>     return next(matches).load()
>   File "/usr/local/lib/python3.9/importlib/metadata.py", line 86, in load
>     module = import_module(match.group('module'))
>   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in > import_module
>     return _bootstrap._gcd_import(name[level:], package, level)
>   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
>   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
> ModuleNotFoundError: No module named 'black'

Attached is a patch that should remedy the packaging issues. (At least games/anki builds fine so far and I haven't had the time to do a bulk run against all consumers of devel/py-black, yet).
Comment 6 commit-hook freebsd_committer freebsd_triage 2022-10-22 09:33:15 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=bfff400638f87b7be4144b56728d54e271048d13

commit bfff400638f87b7be4144b56728d54e271048d13
Author:     Kai Knoblich <kai@FreeBSD.org>
AuthorDate: 2022-10-22 09:26:51 +0000
Commit:     Kai Knoblich <kai@FreeBSD.org>
CommitDate: 2022-10-22 09:26:51 +0000

    devel/py-black: Really fix packaging issues

    * When the port was updated via dea032cba8b3, it still wouldn't build,
      which was then fixed with 25b2be6023fe.

      Although the port can now be built, the content of the generated
      package is not correct at the moment as every Python module is placed
      into an additional "src/" directory.

      There was an attempt to fix the problem via 76670f6a4f84, but the
      root cause has not yet been eliminated because the Python module is
      still not usable:

      $ python3.9 -m black
      /usr/local/bin/python3.9: No module named black

    * Remedy these issues by adjusting the backported "setup.py" to generate
      the correct package content.

    * Bump PORTREVISION due package change.

    PR:             267040 [1]
    Reported by:    Martin Birgmeier [1]
                    pkg-fallout (for games/anki which requires py-black)
    Approved by:    portmgr (blanket, fix packaging issues)
    Fixes:  25b2be6023fe devel/py-black: fix build

 devel/py-black/Makefile       | 2 +-
 devel/py-black/files/setup.py | 9 ++-------
 2 files changed, 3 insertions(+), 8 deletions(-)
Comment 7 Kai Knoblich freebsd_committer freebsd_triage 2022-10-22 09:38:11 UTC
(In reply to Martin Birgmeier from comment #4)

Thank you Martin, for the report and feedbacks. The packaging issues should be resolved now.

I'll leave this bug for reference a few days open.


(In reply to Kubilay Kocak from comment #3)

Hi Koobs, no MFH is required as devel/py-black in 2022Q4 (= 22.8.0) doesn't have these issues.
Comment 8 Martin Birgmeier 2022-10-23 10:32:04 UTC
Thank you, it is working nicely.

-- Martin
Comment 9 Kai Knoblich freebsd_committer freebsd_triage 2022-11-05 18:40:16 UTC
(In reply to Martin Birgmeier from comment #8)

Thank you for your feedback and confirmation.  I'm therefore now closing this bug because there have been no further issues.