Bug 285997 - pkg-static: py311-wheel-0.45.1 conflicts with py311-wheel044-0.44.0 (installs files into the same place). Problematic file: /usr/local/bin/wheel-3.11
Summary: pkg-static: py311-wheel-0.45.1 conflicts with py311-wheel044-0.44.0 (installs...
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Charlie Li
URL:
Keywords:
: 286261 (view as bug list)
Depends on:
Blocks:
 
Reported: 2025-04-10 06:07 UTC by bagas
Modified: 2025-04-27 13:39 UTC (History)
13 users (show)

See Also:


Attachments
Patch (1.74 KB, patch)
2025-04-14 07:04 UTC, Gleb Popov
no flags Details | Diff
Reproducible bug in clean environment (82.39 KB, text/plain)
2025-04-15 16:34 UTC, Andrea Cocito
no flags Details
Problem reproduction log (82.39 KB, text/plain)
2025-04-15 16:36 UTC, Andrea Cocito
no flags Details
setuptools/_vendor (apply with git-am) (2.52 KB, patch)
2025-04-17 20:08 UTC, Charlie Li
no flags Details | Diff
Build log for devel/meson (72.36 KB, text/plain)
2025-04-19 05:58 UTC, Andrea Cocito
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description bagas 2025-04-10 06:07:08 UTC
Hello.
My system FreeBSD 14.2-RELEASE-p2 amd64

My /etc/make.conf
CPUTYPE?=nocona
MAKE_JOBS_NUMBER=6
NO_GAMES=true
NO_INET6=true
NO_BLUETOOTH=true
NO_SHAREDOCS=true
OPTIONS_UNSET=DOCS X11 IPV6 BLUETOOTH GAMES SMB CUPS VULKAN FFMPEG GITWEB EXAMPLES

How to fix?
pkg-static: py311-wheel-0.45.1 conflicts with py311-wheel044-0.44.0 (installs files into the same place). Problematic file: /usr/local/bin/wheel-3.11

# pkg version -v | egrep 'py'
py311-build-1.2.2_2 = up-to-date with index
py311-flit-core-3.11.0 = up-to-date with index
py311-installer-0.7.0 = up-to-date with index
py311-packaging-24.2 = up-to-date with index
py311-pyproject-hooks-1.2.0 = up-to-date with index
py311-setuptools-63.1.0_3 = up-to-date with index
py311-wheel044-0.44.0 = up-to-date with index
python311-3.11.11 = up-to-date with index

installation log
portupgrade -arR
...
...
...
=> SHA256 Checksum OK for wheel-0.45.1.tar.gz.
===> Patching for py311-wheel-0.45.1
===> py311-wheel-0.45.1 depends on package: py311-flit-core>=3.8 - found
===> py311-wheel-0.45.1 depends on file: /usr/local/bin/python3.11 - found
===> py311-wheel-0.45.1 depends on package: py311-build>=0 - found
===> py311-wheel-0.45.1 depends on package: py311-installer>=0 - found
===> Configuring for py311-wheel-0.45.1
===> Building for py311-wheel-0.45.1
* Getting build dependencies for wheel...
* Building wheel...
Successfully built wheel-0.45.1-py3-none-any.whl
===> Staging for py311-wheel-0.45.1
===> py311-wheel-0.45.1 depends on file: /usr/local/bin/python3.11 - found
===> Generating temporary packing list
===> Creating unique files: Move MAN files needing SUFFIX
===> Creating unique files: Move files needing SUFFIX
Move: bin/wheel --> bin/wheel-3.11
Link: @bin/wheel --> bin/wheel-3.11
====> Compressing man pages (compress-man)
===> Installing for py311-wheel-0.45.1
===> Checking if py311-wheel is already installed
===> Registering installation for py311-wheel-0.45.1 as automatic
[sites25] Installing py311-wheel-0.45.1...
pkg-static: py311-wheel-0.45.1 conflicts with py311-wheel044-0.44.0 (installs files into the same place). Problematic file: /usr/local/bin/wheel-3.11
*** Error code 1

Stop.
make[8]: stopped in /usr/ports/devel/py-wheel
*** Error code 1

Stop.
make[7]: stopped in /usr/ports/devel/meson
*** Error code 1

Stop.
make[6]: stopped in /usr/ports/devel/jsoncpp
*** Error code 1

Stop.
make[5]: stopped in /usr/ports/devel/cmake-core
*** Error code 1

Stop.
make[4]: stopped in /usr/ports/devel/cmake-core
*** Error code 1

Stop.
make[3]: stopped in /usr/ports/devel/re2c
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/devel/re2c
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/lang/php82
*** Error code 1

Stop.
Comment 1 Yuri Victorovich freebsd_committer freebsd_triage 2025-04-10 06:28:40 UTC
Why don't you use 'pkg upgrade -f'?

Just remove py311-wheel044, it's only a build dependency.

There is no bug here that needs to be fixed.
Comment 2 Andrea Cocito 2025-04-12 08:11:33 UTC
Hello,

it really looks like a bug to me, as a matter fo fact if you start with a clean machine and simply install portmaster and do "portmaster meson" you cannot even get meson working: following different dependency chains it tries to install both ports.

I am sure it breaks other ports besides meson, too.

The situation got worse after https://cgit.freebsd.org/ports/commit/devel/py-setuptools/Makefile?id=8b5ae17d2f43388fade30e266615a9e34bf06abd : two days ago one could work around this physically removing the "temporarily” added devel/py-wheel044, but now as it is an hardcoded build dependency of py-setuptools (which, again, is required to build meson) the workaround dows not work anymore.

Thanks,

A.
Comment 3 jSML4ThWwBID69YC 2025-04-12 14:23:17 UTC
(In reply to Yuri Victorovich from comment #1)

I'm seeing the same issue across a dozen different systems. Ports are being built with portmaster. 

OS: 14.2-release
/etc/make.conf
DEFAULT_VERSIONS=perl5=5.38 ssl=openssl python=3.11 python3=3.11 pgsql=16

Here's an example to trigger it. 
--------------
portmaster -d security/py-bcrypt
        Install security/py-bcrypt
        Install devel/py-build@py311
        Install devel/py-flit-core@py311
        Install devel/py-installer@py311
        Install devel/py-packaging@py311
        Install devel/py-pyproject-hooks@py311
        Install devel/py-cffi@py311
        Install devel/py-pycparser@py311
        Install devel/py-setuptools@py311
        Install devel/py-wheel044@py311
        Install devel/py-wheel@py311
        Install lang/cython@py311
<snip>
===>>> security/py-bcrypt >> devel/py-cffi@py311 >> devel/py-wheel@py311 (9/11)

===>  Installing for py311-wheel-0.45.1
===>  Checking if py311-wheel is already installed
===>   Registering installation for py311-wheel-0.45.1 as automatic
[uwsgi16-01.test] Installing py311-wheel-0.45.1...
pkg-static: py311-wheel-0.45.1 conflicts with py311-wheel044-0.44.0 (installs files into the same place).  Problematic file: /usr/local/bin/wheel-3.11
*** Error code 1

Stop.
make: stopped in /usr/ports/devel/py-wheel

===>>> Installation of py311-wheel-0.45.1 (devel/py-wheel@py311) failed
===>>> Aborting update
----------------

Can this bug be reopened?
Comment 4 Andrea Cocito 2025-04-12 14:30:01 UTC
(In reply to jSML4ThWwBID69YC from comment #3)
Hi,
in my opinion the fact that one can take a fresh machine, "pkg add portmaster" and then "portmaster <onlyoneport>" is the expected behavior.
The "temporary" addition of py311-wheel044 broke this behaviour for dozens of ports, including very critical ones.
I cannot see how this should not be considered a bug.
Please, please, please: revert this and the change on py-setuptools, or fix it.
Thanks!
A.
Comment 5 Yuri Victorovich freebsd_committer freebsd_triage 2025-04-12 19:23:46 UTC
Assign to vishwin because he added the devel/py-wheel044 port.
Comment 6 Yuri Victorovich freebsd_committer freebsd_triage 2025-04-12 19:25:48 UTC
This change broke portmaster and other local build tools, as others complain.
poudriere doesn't seem to be affected.
Comment 7 Max Brazhnikov freebsd_committer freebsd_triage 2025-04-12 21:12:00 UTC
(In reply to jSML4ThWwBID69YC from comment #3)
build devel/py-setuptools, remove devel/py-wheel044, proceed with you update.
Comment 8 Yuri Victorovich freebsd_committer freebsd_triage 2025-04-13 02:44:47 UTC
Or you can always use poudriere.
poudriere is a lot more reliable.
Comment 9 Chris Hutchinson 2025-04-13 08:54:01 UTC
(In reply to Yuri Victorovich from comment #8)
FWIW a me too here.
I just unpacked releng-13.5:
13.5-RELENG FreeBSD 15.0-CURRENT #1 main-n270118-94b09d388b81-dirty
to create a ports-dev jail. An attempt to build devel/git within that
resulted in the same error reported here. Because I didn't
have the time to investigate. I simply performed
pkg delete py311-wheel044
and continued my build session.

This is a bug.
Comment 10 jSML4ThWwBID69YC 2025-04-13 15:11:26 UTC
(In reply to Max Brazhnikov from comment #7)

That functions as a workaround. However, the issue breaks every single automated build that has py-wheel in it. At least if you're using portmaster.
Comment 11 Max Brazhnikov freebsd_committer freebsd_triage 2025-04-13 16:25:21 UTC
To portmaster users: you are welcome to come up with a patch to fix this problem. I'm pretty sure it will be committed.
Comment 12 commit-hook freebsd_committer freebsd_triage 2025-04-13 16:34:42 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=d29fc7825c6e4d5a3e04355fbc72f6b957d15b22

commit d29fc7825c6e4d5a3e04355fbc72f6b957d15b22
Author:     Yuri Victorovich <yuri@FreeBSD.org>
AuthorDate: 2025-04-13 16:31:15 +0000
Commit:     Yuri Victorovich <yuri@FreeBSD.org>
CommitDate: 2025-04-13 16:34:06 +0000

    devel/py-wheel{,044}: Add CONFLICTS_INSTALL

    PR:             285997

 devel/py-wheel/Makefile    | 2 ++
 devel/py-wheel044/Makefile | 2 ++
 2 files changed, 4 insertions(+)
Comment 13 Yuri Victorovich freebsd_committer freebsd_triage 2025-04-13 16:35:16 UTC
The fix might be to remove conflicting build-time dependencies before building any port.
This can be achieved by using the CONFLICTS_INSTALL lines.
I just added these lines to ports in question.
Comment 14 Chris Hutchinson 2025-04-13 17:05:33 UTC
(In reply to Yuri Victorovich from comment #13)
Thanks Yuri. That seems reasonable. Fingers crossed. :)
Comment 15 Gleb Popov freebsd_committer freebsd_triage 2025-04-14 07:04:22 UTC
Created attachment 259541 [details]
Patch

The following patch seems to fix the issue for me.
Comment 16 Andrea Cocito 2025-04-15 16:34:48 UTC
Created attachment 259590 [details]
Reproducible bug in clean environment
Comment 17 Andrea Cocito 2025-04-15 16:36:33 UTC
Created attachment 259591 [details]
Problem reproduction log
Comment 18 Andrea Cocito 2025-04-15 16:38:09 UTC
(In reply to Yuri Victorovich from comment #13)

Hello Yuri and thanks for the time you are dedicating to his.

Unfortunately it seems that this did not solve the problem, and no, it is not a portmaster issue.

Please find attached the console log on a "brand new and fresh machine" (a VM in which I simply installed a clean releng-14.2 base system and did not touch anything besides checking out /usr/ports from the current tree): a simple "make install", with no other ports installed, in /usr/ports/devel/meson fails due to this conflict.

I am completely ignorant about python so I cannot contribute, what I can do overnight is to check if the patch proposed by Gleb Popov fixes it.

Yes, it is a bug, it is breaking the build chain for several users (basically anyone who uses something that depends on python...), and IMHO it should not be closed.

Thank you for your help,

A.
Comment 19 Colin Percival freebsd_committer freebsd_triage 2025-04-15 18:31:33 UTC
In addition to other complaints, this breaks FreeBSD release building.

Gleb, can you get your patch committed in the next 24 hours so this week's snapshot builds can happen?
Comment 20 Charlie Li freebsd_committer freebsd_triage 2025-04-15 18:34:07 UTC
(In reply to Colin Percival from comment #19)
Can't guarantee 24 hours. Still reviewing this, but I don't like this workaround. Opens up a can of worms that I will explain in subsequent comments.
Comment 21 Colin Percival freebsd_committer freebsd_triage 2025-04-16 15:39:10 UTC
(In reply to Charlie Li from comment #20)
Can we commit Gleb's patch as a temporary fix even if the final solution ends up being different?  This week's snapshot builds start in about 8 hours and I'd really like to have them succeed.
Comment 22 Charlie Li freebsd_committer freebsd_triage 2025-04-16 22:07:28 UTC
I'm opposed to committing the patch as it stands at all. In general, the whole PYTHONPATH override just because a dependency has to be pinned is not something to be encouraged.

Unfortunately dependency conflicts stemming from version pinning and bounding are a relatively common occurrence in the Python package world, for which (isolated) virtual environments have become essential to managing these between consumers. [0][1] This phenomenon is not limited to Python packaging. As a result, the whole ports mantra of running `make -C /usr/ports/foo/bar all install`, ie recursively building your whole dependency tree, and expecting it to Just Work is evaporating; it only works when the communities of everything we port actually respects this mantra.

That said, I'm testing something where the whole of devel/py-wheel044 is moved under the setuptools vendor directory, to reflect the target consumer. This is only possible because setuptools (version currently in tree) has circular dependencies, and wheel is not already vendored as it is only needed when building under USE_PYTHON=pep517.

Again, this port only lasts until bug 270358 can be committed. However, at some point we cannot continue to advertise recursively building an entire dependency tree without isolated environments (like poudriere, but certainly many other ways to do this), because the rest of the world has moved on and is not something we can necessarily control.

[0] https://peps.python.org/pep-0704/ (itself withdrawn but context applies)
[1] https://discuss.python.org/t/22846
Comment 23 Gleb Popov freebsd_committer freebsd_triage 2025-04-17 05:48:09 UTC
(In reply to Charlie Li from comment #22)
I agree in general, but isn't this specific case sort of special, for which we may make an exception? It is basically the same circular dep problem we saw with glib/gobject-intro, so we won't see these very often.

Right now a lot of people are hit by this issue including re@, I feel bad about making them wait even more.
Comment 24 Charlie Li freebsd_committer freebsd_triage 2025-04-17 19:28:15 UTC
(In reply to Gleb Popov from comment #23)
That's the problem, there are enough special cases in the wild that can eventually creep into our tree to the point where nothing is special. Making exceptions is simply not sustainable given where the rest of the open source world (and even commercial industry) is going and chooses to support. `make -C /usr/ports/foo/bar all install`, ie recursively building the whole dependency tree in the same runtime environment, simply does not make sense in today's world as it currently stands.

Stand by for attachment.
Comment 25 Charlie Li freebsd_committer freebsd_triage 2025-04-17 20:08:55 UTC
Created attachment 259648 [details]
setuptools/_vendor (apply with git-am)

This moves the installed binary distribution under the setuptools vendor directory rather than ${PYTHON_SITELIBDIR}, with a .pth so the setuptools build can find it. Note that while this avoids the file conflict stemming from being in the same hierarchy, anything in ${PYTHON_SITELIBDIR}, including an existing devel/py-wheel, always takes precedence.
Comment 26 Gleb Popov freebsd_committer freebsd_triage 2025-04-18 07:14:13 UTC
(In reply to Charlie Li from comment #25)
It worked for my with my synthetic test, should also work for Colin.
Comment 27 Andrea Cocito 2025-04-19 05:58:43 UTC
Created attachment 259690 [details]
Build log for devel/meson

The last patch does not solve the problem.
See attached log for a clean machine (14.2R) on which all that was done is:
root@test:/usr/ports/devel/meson # history
    1 pkg install git
    2 cd /usr/
    3 git clone --depth 1 https://git.freebsd.org/ports.git
    4 cd ports/devel/meson
    5 make
root@test:/usr/ports/devel/meson #
Comment 28 Gleb Popov freebsd_committer freebsd_triage 2025-04-19 06:01:32 UTC
(In reply to Andrea Cocito from comment #27)
You log says

py311-wheel044-0.44.0

while it should say

py311-wheel044-0.44.0_1

according to Charlie's patch.
Comment 29 Andrea Cocito 2025-04-19 06:25:35 UTC
(In reply to Gleb Popov from comment #28)

My fault, I assumed it was on GIT already, after patching it compiles properly, thanks.
Comment 30 Charlie Li freebsd_committer freebsd_triage 2025-04-23 17:25:03 UTC
*** Bug 286261 has been marked as a duplicate of this bug. ***
Comment 31 Gleb Popov freebsd_committer freebsd_triage 2025-04-27 13:39:06 UTC
(In reply to Charlie Li from comment #25)
Charlie are you going to commit this?