===> Registering installation for py39-shell-gpt-0.8.1 before /usr/local/sbin/pkg-static register for py39-shell-gpt-0.8.1, FLAVOR: py39 Installing py39-shell-gpt-0.8.1... pkg-static: py39-shell-gpt-0.8.1 conflicts with py39-notus-scanner-22.4.1 (installs files into the same place). Problematic file: /usr/local/lib/python3.9/site-packages/tests/__init__.py *** Error code 1 Both of these ports install files in lib/python3.9/site/packages/tests/__init__.py That seems like that is remarkably presumptuous of these packages both to own a generic location like <sitepkgs>/tests. Is this perhaps unintended fallout of the recent python infrastructure refactoring? Neither of these packages has USE_PYTHON=pep517, and I have not done any research at all to substantiate that thought. So I am assuming recent python infrastructure changes are NOT related, but it did pop in my head. Also is it necessary that both of these packages install to tests/ ? Are those tests/* files required for normal operation? Again, just a question here - I have not done digging to try to determine that myself (I don't know much about these packages yet).
Add py-shell-gpt maintainer to CC.
So after briefly looking at the setup.py / pyproject.toml for both of these projects (py-shell-gpt & py-notus-scanner), they include 'tests' with no option that I saw to disable that. They both stick the tests/* files (including its own __init__.py for each project) in the generic location <site-packages>/tests thus presuming sole ownership of that generically named directory. So that does seem like an upstream problem to me at this time. I have not looked to see if, as a downstream packager, we could do something different, like: - Move the tests/* elsewhere - such as sgpt/tests and notus/tests - Skip packaging tests/* altogether for either or both py-shell-gpt or py-notus-scanner. Are those files necessary for normal run-time operation? Fedora, Debian, and Ubuntu don't have packages for either that I saw. So there's no "how does another distro handle this" information that I found. Neither upstream github repo documents and bug / issue relating to a problematic policy of installing files in tests. That might be a reasonable next step (to open a discussion upstream).
(In reply to John Hein from comment #2) Hi John, Thanks for pointing this out. py-shell-gpt does not require the test files for normal operations. So I decided to not install the test files for avoiding the collision. I will update the port shortly.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=c881e82101784ab6b0ee63de2485eb90fd37570f commit c881e82101784ab6b0ee63de2485eb90fd37570f Author: Hiroki Tagato <tagattie@FreeBSD.org> AuthorDate: 2023-04-02 06:23:12 +0000 Commit: Hiroki Tagato <tagattie@FreeBSD.org> CommitDate: 2023-04-02 06:25:08 +0000 misc/py-shell-gpt: don't install test files py-shell-gpt and py-notus-scanner both try to install test files to a generic location (/usr/local/lib/python3.9/site-packages/tests). py-shell-gtp does not require test files for normal operations, so avoid collision by not installing test files. PR: 270542 Reported by: John Hein <jcfyecrayz@liamekaens.com> misc/py-shell-gpt/Makefile | 9 ++++++++- .../files/patch-shell__gpt.egg-info_SOURCES.txt (new) | 10 ++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-)
Created attachment 241273 [details] [patch] explicitly exclude 'tests' as a package (In reply to commit-hook from comment #4) Thanks for taking a look and confirming that the tests are not needed at all by the run-time installation. I had a different change that excludes 'tests' from the packages in setup.py. I think either this change or removing tests/__init__.py as you did should probably be upstreamed. I suspect they do not really intend to have tests installed as part of a distribution package (I could see that maybe it should go in an sdist package). The attached patch includes the change and does not need the extra post-extract & post-patch steps.
I added an issue upstream to start the conversation: https://github.com/TheR1D/shell_gpt/issues/135
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=0b5bc765936618a1c79da7273d98b1530f7dc01a commit 0b5bc765936618a1c79da7273d98b1530f7dc01a Author: Jose Alonso Cardenas Marquez <acm@FreeBSD.org> AuthorDate: 2023-04-02 22:41:55 +0000 Commit: Jose Alonso Cardenas Marquez <acm@FreeBSD.org> CommitDate: 2023-04-02 22:41:55 +0000 security/py-notus-scanner: Update to 22.4.5 - Fix issues with tests files installation path[1] ChangeLog at: https://github.com/greenbone/notus-scanner/releases/tag/v22.4.5 PR: 270542 Reported by: jcfyecrayz _at_ liamekaens.com [1] security/py-notus-scanner/Makefile | 6 +- security/py-notus-scanner/distinfo | 6 +- security/py-notus-scanner/pkg-plist | 148 ++++++++++++++++++------------------ 3 files changed, 82 insertions(+), 78 deletions(-)
Hello John. I have committed a new update of py-notus-scanner. It fixed issues test files installation. Thanks for you PR
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=52b0843d3594f81fb2f755c25529ef33e75aa7e0 commit 52b0843d3594f81fb2f755c25529ef33e75aa7e0 Author: John Hein <jcfyecrayz@liamekaens.com> AuthorDate: 2023-04-03 09:17:03 +0000 Commit: Hiroki Tagato <tagattie@FreeBSD.org> CommitDate: 2023-04-03 09:18:33 +0000 misc/py-shell-gpt: switch to better way to avoid installation of test files PR: 270542 Reported by: John Hein <jcfyecrayz@liamekaens.com> misc/py-shell-gpt/Makefile | 8 +------- misc/py-shell-gpt/files/patch-setup.py (new) | 11 +++++++++++ .../files/patch-shell__gpt.egg-info_SOURCES.txt (gone) | 10 ---------- 3 files changed, 12 insertions(+), 17 deletions(-)
(In reply to John Hein from comment #5) Hi, this is better way to avoid test files installation. Switched to your solution. Thanks for the patch!
(In reply to Hiroki Tagato from comment #10) Upstream shell_gpt removed __init__.py (and updated to 0.8.3), so find_packages() (with no exclude) should now not pull in 'tests' to the distribution. In the commit after the upstream issue was opened, tests/__init__.py was removed - https://github.com/TheR1D/shell_gpt/commit/096b690e6349b5e11ba3c1d8074e9c49acd6fe32 was specifically mentioned in response to the upstream issue 135. So when you update to 0.8.3 or later, the setup.py patch can be removed. As an effect of that upstream change, however, the distribution tar ball no longer has tests at all. If you want to get the tests, you will have to pull in the github tarball. I added a suggestion upstream (in the same issue 135) that they include tests in MANIFEST.in to have them in the sdist. Then you could change MASTER_SITES to the following to get the sdist: MASTER_SITES= https://files.pythonhosted.org/packages/source/s/${PORTNAME}/ We'll see what upstream has to say in the upstream issue.
(In reply to Jose Alonso Cardenas Marquez from comment #8) Jose, I tried to install your update to py-notus-scanner and hit a plist issue: ===> Registering installation for py39-notus-scanner-22.4.5 pkg-static: Unable to access file /wrkdirs/usr/ports/security/py-notus-scanner/work-py39/stage/usr/local/lib/python3.9/site-packages/notus/scanner/models/__pycache__/advisory.cpython-39.opt-1.pyc:No such file or directory pkg-static: Unable to access file /wrkdirs/usr/ports/security/py-notus-scanner/work-py39/stage/usr/local/lib/python3.9/site-packages/notus/scanner/models/__pycache__/advisory.cpython-39.pyc:No such file or directory pkg-static: Unable to access file /wrkdirs/usr/ports/security/py-notus-scanner/work-py39/stage/usr/local/lib/python3.9/site-packages/notus/scanner/models/advisory.py:No such file or directory pkg-static: Unable to access file /wrkdirs/usr/ports/security/py-notus-scanner/work-py39/stage/usr/local/lib/python3.9/site-packages/notus/tests/models/__pycache__/test_advisory.cpython-39.opt-1.pyc:No such file or directory pkg-static: Unable to access file /wrkdirs/usr/ports/security/py-notus-scanner/work-py39/stage/usr/local/lib/python3.9/site-packages/notus/tests/models/__pycache__/test_advisory.cpython-39.pyc:No such file or directory pkg-static: Unable to access file /wrkdirs/usr/ports/security/py-notus-scanner/work-py39/stage/usr/local/lib/python3.9/site-packages/notus/tests/models/test_advisory.py:No such file or directory
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=d8153a49cb9a2c77a16459a675edca62caa23100 commit d8153a49cb9a2c77a16459a675edca62caa23100 Author: Hiroki Tagato <tagattie@FreeBSD.org> AuthorDate: 2023-04-04 06:20:57 +0000 Commit: Hiroki Tagato <tagattie@FreeBSD.org> CommitDate: 2023-04-04 06:23:10 +0000 misc/py-shell-gpt: update to 0.8.3 Upstream removed the test files from the distribution. So temporarily disable test target. Changelog: - https://github.com/TheR1D/shell_gpt/releases/tag/0.8.2 - https://github.com/TheR1D/shell_gpt/releases/tag/0.8.3 PR: 270542 Reported by: John Hein <jcfyecrayz@liamekaens.com> misc/py-shell-gpt/Makefile | 11 ++++++----- misc/py-shell-gpt/distinfo | 6 +++--- misc/py-shell-gpt/files/patch-setup.py (gone) | 11 ----------- 3 files changed, 9 insertions(+), 19 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=b0955319f764587697ae7b5092e5df6261e49245 commit b0955319f764587697ae7b5092e5df6261e49245 Author: Jose Alonso Cardenas Marquez <acm@FreeBSD.org> AuthorDate: 2023-04-04 15:06:11 +0000 Commit: Jose Alonso Cardenas Marquez <acm@FreeBSD.org> CommitDate: 2023-04-04 15:06:11 +0000 security/py-notus-scanner: Fix pkg-plist PR: 270542 Reported by: jcfyecrayz _at_ liamekaens.com security/py-notus-scanner/pkg-plist | 6 ------ 1 file changed, 6 deletions(-)
(In reply to John Hein from comment #12) Hello John, I think it is fixed right now in my latest commit