Bug 241463 - devel/scons: flavor scons for python3
Summary: devel/scons: flavor scons for python3
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Sunpoet Po-Chuan Hsieh
URL:
Keywords: feature, needs-qa
Depends on:
Blocks: 242002
  Show dependency treegraph
 
Reported: 2019-10-24 12:22 UTC by Ronald Klop
Modified: 2019-11-16 13:26 UTC (History)
4 users (show)

See Also:
koobs: maintainer-feedback? (sunpoet)


Attachments
Changes to Mk/Uses/scons.mk for suppport of the :py3 argument. (930 bytes, patch)
2019-10-24 12:22 UTC, Ronald Klop
no flags Details | Diff
Changes to devel/scons to support a slave port. (391 bytes, patch)
2019-10-24 12:23 UTC, Ronald Klop
no flags Details | Diff
Makefile for the new devel/scons-py3 port. (222 bytes, text/plain)
2019-10-24 12:24 UTC, Ronald Klop
no flags Details
Add flavor to scons port and Mk/Uses (943 bytes, patch)
2019-10-29 22:56 UTC, Ronald Klop
no flags Details | Diff
Add flavor to scons port and Mk/Uses (1.83 KB, patch)
2019-10-31 11:32 UTC, Ronald Klop
no flags Details | Diff
Add flavor to scons port and Mk/Uses (1.95 KB, patch)
2019-10-31 11:39 UTC, Ronald Klop
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ronald Klop 2019-10-24 12:22:32 UTC
Created attachment 208562 [details]
Changes to Mk/Uses/scons.mk for suppport of the :py3 argument.

There are some attempts on flavoring devel/scons to support python3. These PRs did not get through. I don't know the details why.

This is my attempt on creating a scons-py3 slave port and supporting it in Mk/Uses/scons.mk.
It works for me. I can create the mongodb42 port with USES=scons:py3.
Comment 1 Ronald Klop 2019-10-24 12:23:36 UTC
Created attachment 208563 [details]
Changes to devel/scons to support a slave port.
Comment 2 Ronald Klop 2019-10-24 12:24:30 UTC
Created attachment 208564 [details]
Makefile for the new devel/scons-py3 port.
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2019-10-25 07:23:45 UTC
Though I understand the motivation for this port proposal, we'd prefer not to add py2/3 specific ports, because:

- Python ports should always allow *any* Python version to be used for the build that the upstream supports. Any Python port unnecessarily restricting the declared version support is considered a bug.

- All else being equal, there ought to be no work required for 'specifically' or explicitly flavouring any Python port, and instead leverage the automatic flavouring that is produced solely by declaring the versions that the software *supports*. Perhaps scons is differerent, but in that case, we should understand where and how, and seek to aim our effort toward addressing those questions rather than creating a potentially very short lived port that has limited utility, and comes with downsides.

- It breaks with the conventions/mechanism that we have developed to allow arbitrary user-selection of their preferred Python version, while minimising conflicts between packages.

We should resolve any barriers/issues with the main scons port not allowing, or not being currently appropriate for building with Python 3.x.

Could you please add any relevant scons issue ID's to this bugs See Also field
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2019-10-25 08:25:46 UTC
@Sunpoet Let me know if/how I can help figure out the best way forward, available on IRC (freenode) any time to assist.
Comment 5 Sunpoet Po-Chuan Hsieh freebsd_committer 2019-10-29 15:41:31 UTC
(In reply to Ronald Klop from comment #1)

I think scons@py27 should be able to coexist with scons@py3x before python 2.7 EoL. In your proposal, it's scons and scons-py3.
Comment 6 Sunpoet Po-Chuan Hsieh freebsd_committer 2019-10-29 15:45:07 UTC
Regarding using scons with python 3.x, my initial plan was to change its dependency from python 2.7 to python 3.x and patch dependent ports. But that's a long list and I failed to fix some ports.

I think the better solution right now is as follows:
- Add flavored devel/scons by relaxing USES=python
- Change Mk/Uses/scons.mk to use flavored scons and add :py27 arg for scons@py27
- Let scons@py27 and scons@py3x coexist
- Change all dependent ports which are not python3-ready to scons:py27
Comment 7 Ronald Klop 2019-10-29 17:35:03 UTC
(In reply to Sunpoet Po-Chuan Hsieh from comment #6)
Hi, would it be possible to make scons@py27 the default? So that existing ports need not be updated at once, but new ports can start using scons@py3x?
To stay backwards compatible for now?
I have never 'flavoured' a port, so I hope I don't ask the obvious.
Comment 8 Ronald Klop 2019-10-29 22:56:07 UTC
Created attachment 208682 [details]
Add flavor to scons port and Mk/Uses

New patch which uses flavor to create a py27-scons and a py36-scons package.
This works for me with the existing mongodb36 ports which uses py27-scons without any changes to the mongodb36 port.
And my new mongodb42 port uses py36-scons because it declares "USES=python:3.5+,build scons".
These py*-scons packages conflict with each other. But that shouldn't be to big of a problem. If it is I'm willing to look into the uniquefiles Uses and see if that helps.
AFAIK this change keeps the ports tree backwards compatible. Please give feedback.
Comment 9 Kubilay Kocak freebsd_committer freebsd_triage 2019-10-30 01:33:17 UTC
(In reply to Sunpoet Po-Chuan Hsieh from comment #6)

This is the better option. The scons port should do nothing but declare the versions *it* supports, and scons consumers ought to declare specifically the version(s) they support, allowing them to move forward at their own pace, without 'imperative' selection by the framework.

Notes:

- It's worth considering having scons.mk support <version-specifier> in the exact same way that USES=python supports it, rather than it being an FLAVOR argument (scons:py27)

This allows, scons consuming ports to declaratively specify/declare version support, rather than 'imperatively' 'choosing' a framework specific implementation details (flavor)

This also enables / gives us the ability to factor out a generic <version-specifier> support across more areas of the framework, starting with Python / Scons, and improving it in the process ("!=X" support, "X,Y" support, etc), which has huge benefits for consistency and version derivation across our tree.
Comment 10 Kubilay Kocak freebsd_committer freebsd_triage 2019-10-30 01:35:51 UTC
(In reply to Ronald Klop from comment #7)

Consumers requiring a specific version should be update to correctly declare their version support, at the same time that the scons port is updated to declare correctly its supported versions (in this case: "relaxing python versions to allow 2/3")

It is not worth the benefit of punting the problem of updating ("fixing") consumers down the road
Comment 11 Kubilay Kocak freebsd_committer freebsd_triage 2019-10-30 01:37:16 UTC
(In reply to Ronald Klop from comment #8)

What are the conflicting files? If similar to other python packages/software, only a small proportion of expected files in non-python-version-specific locations will conflict (binaries in LOCALBASE, man pages, docs) and in that case, they ought to be resolved such that they are concurrently installable.
Comment 12 Ronald Klop 2019-10-31 11:32:16 UTC
Created attachment 208730 [details]
Add flavor to scons port and Mk/Uses

New version.

- No more conflicts because of --standard-lib and uniquefiles:dirs.
- No implicit flavor chosen. Explicit need to specify USES=scons:py36.
- USES=scons defaults to scons:py27 to stay backwards compatible.
- /usr/local/bin/scons -> scons-3.6 is only installed by scons:py36 (this was automaticly done by the python/uniquefiles framework).
- Easy to add more py* versions.

Is there something necessary to fix or improve on this patch?
Comment 13 Ronald Klop 2019-10-31 11:39:21 UTC
Created attachment 208731 [details]
Add flavor to scons port and Mk/Uses

Minor fix: USES=scons:py27 was not working, only USES=scons. Fixed, now supports both.
Comment 14 Ronald Klop 2019-11-06 09:06:02 UTC
(In reply to Kubilay Kocak from comment #9)
I'm looking for feedback on the latest patches. Does this version already provide the features you are talking about or did I misunderstand something?

In short: What is needed to make this ready for commit?