Bug 237795 - devel/gobject-introspection: "needs Python 3.x at least, but 2.7 was specified"
Summary: devel/gobject-introspection: "needs Python 3.x at least, but 2.7 was specified"
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Kubilay Kocak
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2019-05-08 12:31 UTC by Marco Beishuizen
Modified: 2019-09-10 08:08 UTC (History)
12 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marco Beishuizen 2019-05-08 12:31:58 UTC
When trying to portupgrade www/firefox to 66.0.5, it tries to build the dependency devel/gobject-introspection. The default Python version is 3.6 on the system, but building gobject-introspection still fails:

...
===>  Patching for libnotify-0.7.7_1
===>   libnotify-0.7.7_1 depends on executable: gmake - found
===>   libnotify-0.7.7_1 depends on package: pkgconf>=1.3.0_1 - found
===>   libnotify-0.7.7_1 depends on executable: g-ir-scanner - not found
===>  gobject-introspection-1.56.1,1 needs Python 3.4 at least, but 2.7 was
specified.
*** Error code 1

Stop.
make[3]: stopped in /usr/ports/devel/gobject-introspection
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/devel/libnotify
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/www/firefox
*** Error code 1
...
Comment 1 Marco Beishuizen 2019-05-08 18:05:50 UTC
Turns out that it's probably a portupgrade/flavors bug. I have a patch installed for portupgrade that should make portupgrade flavor-capable. But somehow this doesn't work well in case of gobject-introspection. Upgrading with portmaster went ok.

I'll close this one for now since it's not a gobject-introspection bug.
Comment 2 Marco Beishuizen 2019-05-10 10:37:49 UTC
Reopen this bug because it doesn't seem to be a portupgrade bug. Tried to "make install clean" math/gnumeric in ports, but stops at devel/gobject-introspection with the same error.
Comment 3 Mikhail Teterin freebsd_committer 2019-06-05 02:07:41 UTC
If portmgr actually built ports from source -- rather than relying on Podrierre to install all dependencies from pre-built packages -- they would've encountered this bug much earlier. It affects ports needing one version of Python, but depending on ports requiring another.

For example:

...
===>   gstreamer1-plugins-gl-1.14.4_2 depends on shared library: libgraphene-1.0.so - not found
===>  graphene-1.8.2_1 needs Python 3.4 at least, but 2.7 was specified.
*** Error code 1

Installing the dependencies (like graphene in my example) first -- directly from command line -- is the workaround...
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2019-06-05 09:18:29 UTC
(In reply to Marco Beishuizen from comment #0)

Can you please provide more information, including:

- Exact FreeBSD version (uname -a output)
- pkg version -v output (as an attachment)
- /etc/make.conf contents (as an attachment)
- Full build log, including portupgrade/portmaster command (as an attachment)

^Triage: Leave this as an individual port issue until (if) isolated to a framework issue.
Comment 5 Marco Beishuizen 2019-06-15 18:46:07 UTC
I need closing this one again because the bug seems to have disappeared.

I did notice that the bug doesn't appear when using portmaster. Portmaster is builing devel/bison before building gobject-introspection, and then the bug doesn't happen. Using portupgrade or doing a make install directly was triggering the bug because devel/bison wasn't build first for some reason.

Trying to reproduce the bug today failed: bison does build before gobject-introspection now, and the bug disappears.

My uname -a:
FreeBSD yokozuna 12.0-STABLE FreeBSD 12.0-STABLE r343805 YOKOZUNA  amd64

My make.conf:
CPUTYPE?=nocona
KERNCONF=YOKOZUNA
DISTDIR=/var/ports/distfiles
PACKAGES=/var/ports/packages
OPTIONS_UNSET=GNOME GCONF DCONF GSETTINGS CUPS AVAHI HAL ALSA PULSEAUDIO WAYLAND COLORD
OPTIONS_SET=LPR NVIDIA NVTHREADS SNDIO OSS ZTS
PKG_NOCOMPRESS=1
WITH_MPM=event
#DEVELOPER=yes                                                                        
#BATCH=yes                                                                            
#DISABLE_VULNERABILITIES=yes                                                          
#FORCE_PKG_REGISTER=yes                                                               
DEFAULT_VERSIONS+= mysql=8.0 pgsql=11 emacs=nox lua=5.3
MESA_LLVM_VER=80
.if ${.CURDIR:M*/editors/emacs*}
FLAVOR?=nox
.endif
Comment 6 Mikhail Teterin freebsd_committer 2019-06-16 10:04:45 UTC
Gentlemen, what's happening? The problem's been analyzed... When building a port (P0), that -- for whatever reason -- wants Python-N, the Python-version is propagated to all dependencies.

When one of the dependencies (Pk) is incompatible with Python-N, you get the  error...

In Marco's case, for example, P0 is math/gnumeric, which insists on Python-2.7 -- and Pk is devel/gobject-introspection, which wants 3.4+ or higher.

This is difficult to reproduce, because, once you have the dependency (such as gir) installed -- either through a direct "make install" or via a pre-built package -- the dependents will build just fine... The problem strikes only, when the dependency is built automatically -- with the requested Python-version passed from the dependent.

The bug is not with devel/gobject-introspection itself. This is a portmgr (or python@ ?) issue -- there needs to be a way to distinguish between the two cases:

 1. I'm going to use Python-N, and so must you, or else we'd be incompatible at run-time.
 2. I need to use Python-N, but I don't care, what you're using.
Comment 7 Kubilay Kocak freebsd_committer freebsd_triage 2019-06-16 10:50:13 UTC
I have no idea how the situation ended up resolving itself, but being provided that information, or an opportunity to obtain it may have been enlightening.

As far as I can see, the gnumeric case is a case of wrong dependencies.

gnumeric cannot logically *only* support 2.7, and have a set of dependencies that *don't also* support 2.7.

Either gnumeric is incorrectly constraining itself to 2.7 when it supports 3.x, or gobject-introspection is incorrectly constrained to 3.4+, when it also supports 2.7, or some dependencies are spurious.

math/gnumeric (2.7)
`-- devel/pygobject3-common (no python version-spec, any version)
    `-- devel/gobject-introspection (3.4+)

And I don't see how 'installing it manually' can fix *this* issue.

You cant install py36-gobject-introspection and have it be a satisfactory dependency for py27-pygobject3-common

You also cant install py36-pygobject3-common to get py36-gobject-introspection and have either of those satisfy math/gnumeric which only ever wants/supports (at present) 2.7.

If I'm misunderstanding what you meant, please elaborate with an example of 'installing it manually, not as a automatic, depends', that fixes *this* issue, which is:

 a) a port requiring one specific version (eg: only 2.7)
 b) cannot be built when one of its dependencies (eg: gobject-introspection)
 c) requires a version that does not overlap with its own (eg: 3.4+)

But I can't see how that's possible, as making (a)(b)(c) work is precluded fundamentally just by the the Python ecosystem works.
Comment 8 Mikhail Teterin freebsd_committer 2019-06-16 11:08:21 UTC
(In reply to Kubilay Kocak from comment #7)
> please elaborate with an example of 'installing it manually, not as a
> automatic, depends', that fixes *this* issue, which is:
>
> a) a port requiring one specific version (eg: only 2.7)
> b) cannot be built when one of its dependencies (eg: gobject-introspection)
> c) requires a version that does not overlap with its own (eg: 3.4+)

The version of Python, that's used by a dependency, may not actually matter, when the dependency is not itself a Python-package -- that is, is not used by the dependent as "import foo". Or, when Python is merely a BUILD-dependency for the dependency... (Both of these examples fall under item 2. in my Comment #6.)

This is why, after you install gir manually -- allowing it to use whatever version it wants -- build works.

Things break, when the build of gir is invoked automatically by the build of a depending port, because in this case the Python-version is EXPLICITLY PRESCRIBED (by adding PYTHON_VERSION to the environment, I think)...
Comment 9 Kubilay Kocak freebsd_committer freebsd_triage 2019-06-16 11:59:19 UTC
(In reply to Mikhail Teterin from comment #8)

Sure, got it. Thanks for helping me get there. And the dependency is satisfied after manual installation because it's a binary/filename *_DEPENDS, not a package type.

So for case (2) of your two types, this type, one would need to be able to, minimally at least, based on what we know now:

a) more finely declare the dependency kind (so as not to propagate for it)
b) declare it on a per-dependency basis (have it not apply to any other things where it may not apply)

Independent to the feasibility, something like that is going to require some very clear and precise "spec'ing" out before it goes to portmgr, as I suspect it will need to. 

This is because I don't see this as being a bug, but rather a feature to more finely control (perhaps more precisely: ability to constrain) python version propagation, to take into account how a dependency is used.

It might be worth us chewing the fat on IRC (#freebsd-python) and coming up with an unambiguous and lightweight "PEP" for it. Who knows, we may even be able to come up with a hack^W workaround in the short term, or other alternatives
Comment 10 Kubilay Kocak freebsd_committer freebsd_triage 2019-06-16 12:03:35 UTC
I'll take this for now, and will re-summarise/classify accordingly.
Comment 11 takeda 2019-06-20 04:41:58 UTC
Similar thing is happening with meson when used as a build dependency for deluge. We will have more of these occurrences as 2.7 is becoming EOL and packages are dropping support for it.
Comment 12 Alfredo Dal'Ava Júnior 2019-07-04 16:25:11 UTC
I reproduce the same issue on powerpc64 using a clean FreeBSD 13/current:

# cd /usr/ports/www/firefox && make

It crashed with "needs Python 3.4 at least, but 2.7 was specified." on 'devel/gobject-introspection'

# cd /usr/ports/devel/gobject-introspection && make

It crashed with "needs Python 3.4 at least, but 2.7 was specified." now on 'devel/meson'


I did workaround this by doing:


# cd /usr/ports/devel/meson && make && make install
# cd /usr/ports/devel/gobject-introspection && make && make install
# cd /usr/ports/www/firefox && make
Comment 13 Walter Schwarzenfeld freebsd_triage 2019-08-19 10:50:40 UTC
/usr/ports/MOVED:
lang/python34|lang/python36|2018-12-18|Has expired: Fails to build with recent OpenSSL
Comment 14 Ting-Wei Lan 2019-08-19 11:34:09 UTC
(In reply to Walter Schwarzenfeld from comment #13)
What did you mean by posting this line? It doesn't look relevant to the issue.
Comment 15 Mikhail Teterin freebsd_committer 2019-08-23 04:03:09 UTC
(In reply to Kubilay Kocak from comment #10)
Right now this is breaking firefox build -- firefox wants both python-2.7 and llvm80. The latter wants python-3.6 (why?!)

So, if llvm80 is not already installed, the build of firefox tries to build it -- which breaks, because firefox passes 2.7 as the requirement, which llvm80 rejects.

The work-around is to build llvm80 first. It should not be necessary.
Comment 16 Bryan Drewery freebsd_committer 2019-08-28 16:13:31 UTC
(In reply to Mikhail Teterin from comment #3)

This is dumb. Portmgr uses Poudriere which builds everything from source.

Portupgrade doesn't support FLAVORs. Move on.
Comment 17 Mikhail Teterin freebsd_committer 2019-08-28 21:44:31 UTC
(In reply to Bryan Drewery from comment #16)
> This is dumb. Portmgr uses Poudriere which builds everything from source.

This is dumb -- because it does not, not quite. Poudriere performs (the equivalent of) the following:

   1. make -C /usr/ports/depend/ency install clean
   2. make -C /usr/ports/dep/endent install clean

This still works. For example, llvm (the dependency) builds against Python-3.6, while Firefox then uses llvm to build against Python-2.7. And those using Podriere exclusively are unaware of the problem -- hence my frustration with portmgr@ (overly) relying on the tool.

What happens, when one REALLY builds everything from source is:

   1. make -C /usr/ports/dep/endent install clean

In the above scenario, the dependent passes its preferred FLAVORs to the dependencies. Which breaks, if one of the dependencies disagrees on the flavor. This is a design flaw in the FLAVORs implementation, which needs to be corrected.

This was already explained, in detail, in comment #8 -- and accepted by koobs@ in comment #9. But you saw me "unfairly attacking" your team and rushed to defend it without even reading to the end, evidently...

You can reproduce the problem today -- by building Firefox on a machine without llvm80 already installed. FROM SOURCE.

> Portupgrade doesn't support FLAVORs. Move on.

I'll take this information, however irrelevant to the ticket, under advisement. Thank you.
Comment 18 Walter Schwarzenfeld freebsd_triage 2019-08-28 23:36:46 UTC
python.mk:
457 # Pass PYTHON_VERSION down the dependency chain. This ensures that
    458 # port A -> B -> C all will use the same python version and do not
    459 # try to find a different one, if the passed version fits into
    460 # the supported version range.
    461 PYTHON_VERSION?=        python${_PYTHON_VERSION}
    462 .if !defined(PYTHON_NO_DEPENDS) && \
    463     ${PYTHON_VERSION} != python${PYTHON_DEFAULT}
    464 DEPENDS_ARGS+=          PYTHON_VERSION=${PYTHON_VERSION}
    465 .endif


....that is the cause...