Bug 271673 - lang/python312: New port, update to 3.12.9
Summary: lang/python312: New port, update to 3.12.9
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: freebsd-python (Nobody)
URL:
Keywords:
Depends on: 268283 270510
Blocks:
  Show dependency treegraph
 
Reported: 2023-05-27 11:50 UTC by Wen Heping
Modified: 2025-04-08 10:00 UTC (History)
33 users (show)

See Also:


Attachments
- 3.12.0 beta 1 (41.12 KB, application/x-gzip)
2023-05-27 11:50 UTC, Wen Heping
no flags Details
python-3.12.0 beta2 (41.36 KB, application/x-gzip)
2023-06-08 02:11 UTC, Wen Heping
no flags Details
python-3.12.0 RC1 (41.42 KB, application/x-gzip)
2023-08-07 08:04 UTC, Wen Heping
no flags Details
python-3.12.0rc2 (41.08 KB, application/x-gzip)
2023-09-07 09:38 UTC, wen
no flags Details
Update to 3.12.0rc3 (41.37 KB, application/x-gzip)
2023-09-22 10:30 UTC, Wen Heping
no flags Details
Update to 3.12.0 Final (41.37 KB, application/x-gzip)
2023-10-03 11:27 UTC, Wen Heping
no flags Details
update to 3.12.1 (41.58 KB, application/x-gzip)
2023-12-09 07:57 UTC, Wen Heping
no flags Details
Update to 3.12.2 (41.56 KB, application/x-gzip)
2024-02-08 12:32 UTC, Wen Heping
no flags Details
Update to 3.12.3 (42.25 KB, application/x-gzip)
2024-04-12 09:17 UTC, Wen Heping
no flags Details
Update to 3.12.4 (42.27 KB, application/x-gzip)
2024-06-07 08:42 UTC, Wen Heping
no flags Details
Update to 3.12.5 (42.36 KB, application/x-gzip)
2024-08-08 01:21 UTC, Wen Heping
no flags Details
Update to 3.12.7 (42.41 KB, application/x-gzip)
2024-10-03 02:19 UTC, Wen Heping
no flags Details
Update to 3.12.8 (42.62 KB, application/x-gzip)
2024-12-05 13:14 UTC, Wen Heping
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Wen Heping freebsd_committer freebsd_triage 2023-05-27 11:50:26 UTC
Created attachment 242441 [details]
- 3.12.0 beta 1

New port, now is 3.12.0 beta 1.
Comment 1 Wen Heping freebsd_committer freebsd_triage 2023-05-27 11:53:30 UTC
Shall we import python-3.12.0 just for test, currently not hook it into Uses/python.mk ?
Comment 2 Wen Heping freebsd_committer freebsd_triage 2023-05-27 11:53:59 UTC
The test result:

== Tests result: FAILURE ==

434 tests OK.

10 slowest tests:
- test_tools: 5 min 34 sec
- test_smtpnet: 5 min
- test_signal: 3 min 56 sec
- test_concurrent_futures: 2 min 10 sec
- test_socket: 1 min 46 sec
- test_multiprocessing_spawn: 1 min 36 sec
- test_multiprocessing_forkserver: 1 min 17 sec
- test_multiprocessing_fork: 1 min 10 sec
- test_nntplib: 1 min 6 sec
- test_io: 35.4 sec

5 tests failed:
    test_dtrace test_posix test_shutil test_tempfile test_tools

1 test altered the execution environment:
    test_capi

27 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils test_dbm_gnu test_devpoll
    test_epoll test_gdb test_idle test_ioctl test_launcher test_msilib
    test_peg_generator test_perf_profiler test_perfmaps test_spwd
    test_sqlite3 test_startfile test_tcl test_tix test_tkinter
    test_ttk test_ttk_textonly test_turtle test_winconsoleio
    test_winreg test_winsound test_wmi test_zipfile64
0:18:51 load avg: 0.30
0:18:51 load avg: 0.30 Re-running failed tests in verbose mode
0:18:51 load avg: 0.30 Re-running test_tools in verbose mode (matching: test_freeze_simple_script)
test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script) ... creating the script to be frozen at /tmp/tmps_u2puux/app.py
copying the source tree into /tmp/tmps_u2puux/cpython...
configuring python in /tmp/tmps_u2puux/python-build...
CalledProcessError: Command '['/tmp/tmps_u2puux/cpython/python', '-c', 'import sysconfig; print(sysconfig.get_config_var("CONFIG_ARGS"))']' returned non-zero exit status 1.
--- STDOUT ---

--- STDERR ---
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/tmps_u2puux/cpython/Lib/sysconfig.py", line 740, in get_config_var
    return get_config_vars().get(name)
           ^^^^^^^^^^^^^^^^^
  File "/tmp/tmps_u2puux/cpython/Lib/sysconfig.py", line 723, in get_config_vars
    _init_config_vars()
  File "/tmp/tmps_u2puux/cpython/Lib/sysconfig.py", line 670, in _init_config_vars
    _init_posix(_CONFIG_VARS)
  File "/tmp/tmps_u2puux/cpython/Lib/sysconfig.py", line 536, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_sysconfigdata__freebsd14_'

---- END ---
Comment 3 Charlie Li freebsd_committer freebsd_triage 2023-05-27 12:08:29 UTC
Not a good idea to import until packaging/setuptools particularly is fixed up. Will cause too much distracting noise on mailing lists and such from those trying to test it with their packages.
Comment 4 Daniel Engberg freebsd_committer freebsd_triage 2023-05-27 12:48:19 UTC
A few core libs are also going away / being replaced
https://peps.python.org/pep-0594/
Comment 5 Wen Heping freebsd_committer freebsd_triage 2023-06-08 02:11:44 UTC
Created attachment 242677 [details]
python-3.12.0 beta2

Python-3.12.0 beta2
Comment 6 Wen Heping freebsd_committer freebsd_triage 2023-08-07 08:04:05 UTC
Created attachment 243920 [details]
python-3.12.0 RC1

python-3.12.0 RC1.

Result of `make test`:
439 tests OK.

10 slowest tests:
- test_tools: 6 min 1 sec
- test_socket: 4 min 15 sec
- test_smtpnet: 3 min 10 sec
- test_concurrent_futures: 2 min 7 sec
- test_multiprocessing_fork: 1 min 28 sec
- test_multiprocessing_spawn: 1 min 28 sec
- test_multiprocessing_forkserver: 1 min 17 sec
- test_ssl: 1 min 12 sec
- test_nntplib: 1 min 3 sec
- test_signal: 46.9 sec

2 tests failed:
    test_dtrace test_tools

26 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils test_dbm_gnu test_devpoll
    test_epoll test_gdb test_idle test_ioctl test_launcher test_msilib
    test_perf_profiler test_perfmaps test_spwd test_sqlite3
    test_startfile test_tcl test_tix test_tkinter test_ttk
    test_ttk_textonly test_turtle test_winconsoleio test_winreg
    test_winsound test_wmi test_zipfile64

2 re-run tests:
    test_dtrace test_tools

Total duration: 15 min 25 sec
Comment 7 wen 2023-09-07 07:05:37 UTC
python-3.12.0RC2:

453 tests OK.

10 slowest tests:
- test_smtpnet: 5 min
- test_math: 3 min 25 sec
- test_tarfile: 2 min 38 sec
- test_subprocess: 2 min 30 sec
- test.test_multiprocessing_spawn.test_processes: 2 min 10 sec
- test.test_multiprocessing_forkserver.test_processes: 1 min 49 sec
- test_signal: 1 min 48 sec
- test_tokenize: 1 min 36 sec
- test.test_multiprocessing_fork.test_processes: 1 min 32 sec
- test_zipfile: 1 min 25 sec

2 tests failed:
    test_dtrace test_tempfile

26 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils test_dbm_gnu test_devpoll
    test_epoll test_gdb test_idle test_ioctl test_launcher test_msilib
    test_perf_profiler test_perfmaps test_spwd test_sqlite3
    test_startfile test_tcl test_tix test_tkinter test_ttk
    test_ttk_textonly test_turtle test_winconsoleio test_winreg
    test_winsound test_wmi test_zipfile64

4 re-run tests:
    test_dtrace test_shutil test_tempfile test_tools

2 tests run no tests:
    test_shutil test_tools

Total duration: 30 min 7 sec
Total tests: run=40,183 failures=8 skipped=1,274
Total test files: success=453 failed=2 skipped=26 resource_denied=1 rerun=4 run_no_tests=2
Comment 8 wen 2023-09-07 09:38:30 UTC
Created attachment 244689 [details]
python-3.12.0rc2
Comment 9 Wen Heping freebsd_committer freebsd_triage 2023-09-22 10:30:26 UTC
Created attachment 245113 [details]
Update to 3.12.0rc3

Python 3.12.0rc3
Comment 10 Wen Heping freebsd_committer freebsd_triage 2023-10-03 11:27:20 UTC
Created attachment 245399 [details]
Update to 3.12.0 Final

- Update to 3.12.0 Final
Comment 11 Mohamed Akram 2023-11-24 19:45:18 UTC
Any update on this?
Comment 12 Wen Heping freebsd_committer freebsd_triage 2023-12-09 07:57:00 UTC
Created attachment 246924 [details]
update to 3.12.1

Update to 3..12.1
Comment 13 Henrich Hartzer 2023-12-30 01:46:48 UTC
I'm also interested in this. Could this be merged? Would be great to have it before 2023Q1!
Comment 14 Miroslav Lachman 2024-01-07 08:49:54 UTC
Is there some known problem that prevents lang/python312 from being committed to the ports tree?
Comment 15 Charlie Li freebsd_committer freebsd_triage 2024-01-07 12:23:26 UTC
Not until the setuptools-related PRs linked are completed. Otherwise you will not have many, if at all, Python package ports to use due to our assumption that distutils is available in the standard library, which is no longer the case in 3.12. The road to getting that done is delicate and must be in a specific order to minimise breakage and distractions.
Comment 16 Wen Heping freebsd_committer freebsd_triage 2024-02-08 12:32:55 UTC
Created attachment 248255 [details]
Update to 3.12.2

Update to 3.12.2
Comment 17 Wen Heping freebsd_committer freebsd_triage 2024-04-12 09:17:27 UTC
Created attachment 249927 [details]
Update to 3.12.3

Update to 3.12.3
Comment 18 Amar Takhar 2024-05-22 18:21:19 UTC
This fails to install for me:

===>  Installing for python312-3.12.3
===>  Checking if python312 is already installed
===>   Registering installation for python312-3.12.3
pkg-static: Unable to access file /usr/ports/lang/python312/work/stage/usr/local/man/man1/python3.12.1.gz:No such file or directory
Comment 19 Amar Takhar 2024-05-22 18:21:39 UTC
This fails to install for me:

===>  Installing for python312-3.12.3
===>  Checking if python312 is already installed
===>   Registering installation for python312-3.12.3
pkg-static: Unable to access file /usr/ports/lang/python312/work/stage/usr/local/man/man1/python3.12.1.gz:No such file or directory
Comment 20 Amar Takhar 2024-05-22 18:22:28 UTC
Sorry for the double comment Bugzilla was lagging didn't realise I pressed Save twice.
Comment 21 Wen Heping freebsd_committer freebsd_triage 2024-06-07 08:42:39 UTC
Created attachment 251264 [details]
Update to 3.12.4

Update to 3.12.4
Comment 22 takeda 2024-07-05 20:26:45 UTC
(In reply to Charlie Li from comment #15)

Python 3.12 was released 9 months ago and 3.13 will be soon released.

It looks like we are now having issues because we were clever and released core parts of python as a separate packages and now removal of distutils complicates this version.

Why don't we do the same thing we do with other packages and provide options to enable/disable them? There is much more of dependencies within python than just sqlite that are optional and can be disabled, so singling it out and making it a separate package only adds more work for us):

https://github.com/NixOS/nixpkgs/blob/bc50dd54e8caaa8e3718cc93d2cf32d2aefdc3c6/pkgs/development/interpreters/python/default.nix#L103-L130
Comment 23 Charlie Li freebsd_committer freebsd_triage 2024-07-05 20:37:50 UTC
(In reply to takeda from comment #22)
It doesn't work like that. What nix does is not really relevant to us here.

The distutils deprecation in 3.11 and subsequent removal in this version is a much bigger problem than just separating out sqlite et al from the base Python distribution. Fixing the separated components are relatively easy despite some caveats (namely PEP-517 is only intended for Python 3). They are only separate ports because building some of the libraries they bind would result in circular dependencies back to Python itself (esp if the library uses meson or ninja to build), and not because they are truly optional parts of the base Python distribution.
Comment 24 takeda 2024-07-05 22:34:48 UTC
(In reply to Charlie Li from comment #23)

I see, I was actually wondering why they had the Python3Minimal, was kind of specific and they didn't provide separate one for each python version. I guess that was their solution for this chicken and egg problem.
Comment 25 Wen Heping freebsd_committer freebsd_triage 2024-08-08 01:21:26 UTC
Created attachment 252600 [details]
Update to 3.12.5

Update to 3.12.5
Comment 26 Eren Türkay 2024-09-15 09:37:33 UTC
I've compiled and installed 3.12.5 and it seems to be working fine except sqlite3 module. I cannot get IPython working since it depends on sqlite3. Virtual environments work just fine.

Is there any update regarding to this issue?
Comment 27 Alexey Vyskubov 2024-09-15 11:07:05 UTC
JFYI: I have 3.12.6 installed with mise, and everything seems to work.

🌐 eldanna in ~
❯ ipython
Python 3.12.6 (main, Sep  8 2024, 13:55:00) [GCC 13.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.26.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: print((1+2)*3)
9

In [2]: import sqlite3

In [3]:
Do you really want to exit ([y]/n)?
🌐 eldanna in ~ took 47s
❯ freebsd-version
14.1-RELEASE-p4
Comment 28 Eren Türkay 2024-09-15 12:10:03 UTC
> JFYI: I have 3.12.6 installed with mise, and everything seems to work.

I believe this is due to the fact that mise includes sqlite3 module when compiling Python. I've compiled using the diff provided in this bug report and I have an issue with sqlite3 module. It's supposed to be provided apart from the package itself.

I think there is an ongoing discussion on what to do with those modules.
Comment 29 Charlie Li freebsd_committer freebsd_triage 2024-09-15 12:16:46 UTC
There is no discussion regarding the sqlite3 (and by extension others nominally in the base distribution) module; it will continue to be provided as separate port due to circular dependencies. Any pain point pertains to continued Python 2 (lang/python27) support while Python 3 marches on.

(In reply to Eren Türkay from comment #26)
All dependent bugs must be fixed (two in a specific order) before this can proceed. Committing this out of order will result in basically no Python package ports available for use with this version, along with endless amounts of build fallouts and bickering to the same.
Comment 30 Wen Heping freebsd_committer freebsd_triage 2024-10-03 02:19:41 UTC
Created attachment 253972 [details]
Update to 3.12.7

Update to 3.12.7
Comment 31 dan 2024-11-02 23:30:35 UTC
I'd have to argue just having the lang port without any module ports for 3.12/3.13 would be nice. pip should be main way to install new packages. What is frustrating is when you have modules for a python lang in pkg that can overwrite ones you had installed with pip.
Comment 32 dan 2024-11-02 23:35:27 UTC
personally I install pip-review and execute
pip-review --local --interactive

This keeps you up to date faster than using FreeBSD pkg.
Comment 33 Wen Heping freebsd_committer freebsd_triage 2024-12-05 13:14:20 UTC
Created attachment 255639 [details]
Update to 3.12.8

Update to 3.12.8
Comment 34 Saro 2024-12-17 19:34:37 UTC
(In reply to Wen Heping from comment #33)

I believe the pkg-plist needs to be updated as it is referencing the wrong pip version upon packaging:

===>  Building packages for python312-3.12.8
===>   Building python312-3.12.8
pkg-static: Unable to access file /wrkdirs/overlays/custom/lang/python312/work/stage/usr/local/lib/python3.12/ensurepip/_bundled/pip-24.2-py3-none-any.whl:No such file or directory
*** Error code 1


Change:
lib/python%%XYDOT%%/ensurepip/_bundled/pip-24.2-py3-none-any.whl

To:
lib/python%%XYDOT%%/ensurepip/_bundled/pip-24.3.1-py3-none-any.whl
Comment 35 Chad Jacob Milios 2024-12-21 07:52:25 UTC
(In reply to Saro from comment #34)

i did not run into that issue. attachment #255639 [details] worked well for me as-is, seems to include the very line you suggest.
Comment 36 dan 2024-12-21 18:30:33 UTC
For anyone doesn't want to wait 2-3 more years for recent python version, you can download tar ball off python site, and do the usual ./configure ; make ; make install.

No issues running python 3.12 and 3.13 and using pip to update modules.

You'll have to constantly do it this way however for upgrade of actual python version only downside.
Comment 37 Mohamed Akram 2025-02-23 16:50:06 UTC
(In reply to Charlie Li from comment #15)

I don't need the Python package ports, I just need a working Python so I can use `python3.13 -m venv` to create a virtual environment and install the dependencies for my application using the bundled pip there. It would be really useful to have 3.12 and 3.13 available in the meantime for this case.
Comment 38 Kurt Jaeger freebsd_committer freebsd_triage 2025-02-23 20:11:56 UTC
(In reply to Mohamed Akram from comment #37)
Thanks for describing this approach. So what is the counter-argument to this ?
Comment 39 Charlie Li freebsd_committer freebsd_triage 2025-02-24 00:48:03 UTC
(In reply to Mohamed Akram from comment #37)
This isn't about you, but rather for the ports tree and those who interact with it. Having a Python interpreter/distribution in the tree without its supporting cast, in this case, the ability to build Python packages from source (binary wheels are not acceptable), gives ports users the wrong impression (and ammo for complaints/threads/etc that distract from actually making progress).
Comment 40 Konstantin Belousov freebsd_committer freebsd_triage 2025-02-28 06:39:37 UTC
(In reply to Charlie Li from comment #39)
There is no (visible) progress with having more recent python versions in the
ports.  The suggested use case of using self-supported virtual environments is
completely valid.  It seems to constructive to block at least some needed uses
on another.

The concern of not working ports' py31{2-...} packages is valid as well, but can
be worked around by blocking and complaining the setting of the python default
version to anything that cannot be handled by ports ATM.

IMO.
Comment 41 Michael Osipov freebsd_committer freebsd_triage 2025-02-28 07:09:07 UTC
(In reply to Konstantin Belousov from comment #40)

Good point.
Comment 42 Charlie Li freebsd_committer freebsd_triage 2025-03-03 04:57:26 UTC
(In reply to Konstantin Belousov from comment #40)
Ironically this proves my point/argument of not making available newer Python distributions yet. From a maintainer-support angle, keeping the complaining and such contained here with this one subject is better than multiple threads on multiple subjects, explained henceforth.

Let's say we do commit/make available lang/python312 and lang/python313, but without the python.mk bits to register them as valid FLAVORs. From a user (generously) perspective:
- sees CPython > 3.11 available in ports/packages
- installs one or both
- changes ${PYTHON_DEFAULT} to that version

Ports/packages user:
- finds out no packages for the desired CPython available/built from ports
- starts thread about the aforementioned
- somebody posts a patch enabling the intentionally not registered FLAVOURs
- people test said patch locally
- "wtf hardly anything builds"
- chaos ensues over trying to get stuff to build

For the user who claims to only use specifically venv virtual environments, unfortunately you are still interacting with Python ports/packages as dependencies for other programs (Python or not) from ports/packages. But even with a venv, you will have to run ensurepip yourself in there to bootstrap a usable and update-able pip. We do not ship a bootstrapped pip in the Python distribution port build, nor should we, as it prevents/conflicts with pkg(8) managing ${PYTHON_SITELIBDIR}.

Bottom line, the Python ecosystem is confusing and fast-changing enough, even for maintainers and others who keep close tabs, let alone everyone else. We don't need to incite more confusion and debt.
Comment 43 Konstantin Belousov freebsd_committer freebsd_triage 2025-03-04 06:36:29 UTC
I do not follow your logic.  So somebody have to install the python port with
explicitly disabled use for default version, then somebody has to post a patch
to enable its use for flavor, then somebody should apply that patch, and then
the havoc is ensured.  Awful indeed.

On the other hand, the current situation, where the practical and desired use
of the system, albeit limited, is denied, is fine.
Comment 44 Matthias Andree freebsd_committer freebsd_triage 2025-03-04 20:59:25 UTC
We should add 3.12 and 3.13 ports *NOW* without making them default, and after 2025Q2 has been branched, flip the switch and mop up the fallout. 3.11 has gone on life support (security fixes only) already and 3.13 has been available since October.  

Make sure default stuff is part of the package, and stop tearing it to pieces before we commit.

We can't have it perfect, and we can't torture contributors such as Wen longer.
Comment 45 Antoine Brodin freebsd_committer freebsd_triage 2025-03-04 21:02:44 UTC
(In reply to Matthias Andree from comment #44)
You can't switch just like that, it has to pass an exp-run
Comment 46 Matthias Andree freebsd_committer freebsd_triage 2025-03-04 21:06:13 UTC
we're not switching anything yet, but as proposed, only adding now, and switching only after 2025Q2 branch - and what do we do will fallout when the respective maintainers are asleep at the helm? 

The usual recourse is fixing high-profile stuff, marking leaves and low-profile stuff BROKEN and deprecated and moving on.

If we don't start moving things, we'll still be staring at vishwin's obstructions three years from now when python 3.11 goes out of support.
Comment 47 Charlie Li freebsd_committer freebsd_triage 2025-03-04 21:06:36 UTC
(In reply to Matthias Andree from comment #44)
No course of action is perfect, this is a matter of keeping things orderly. Adding the other ports isn't orderly at this time, it only incites more mess and debt.

(In reply to Konstantin Belousov from comment #43)
Making the newer Python ports available without the framework support tacitly encourages those not privy to further context to post such a patch in a mob mentality fashion. Which then incites more confusion as to why things don't work as intended, when the proper way figured out long long ago but can only be executed with patience and care.
Comment 48 Matthias Andree freebsd_committer freebsd_triage 2025-03-04 21:08:06 UTC
(In reply to Charlie Li from comment #47)
Charlie, you've used up your "stop the world now" tokens for the next three years already. Either you get things moving or you shut up.  I will have no more "yes but" or other showstoppers from you.
Comment 49 Antoine Brodin freebsd_committer freebsd_triage 2025-03-04 21:08:13 UTC
(In reply to Matthias Andree from comment #46)
From my experience, poudriere bulk -a does not even start the 1st time you do a python new version exp-run
Comment 50 Charlie Li freebsd_committer freebsd_triage 2025-03-04 21:12:01 UTC
(In reply to Matthias Andree from comment #48)
You alone don't get to decide that, especially when it's not obvious to me that you have put *any* work into this or the supporting cast. Not everything is visible because of distractions like this.

For the record, I have had everything, at least stuff I use, working locally for quite some time, but at least another exp-run is needed on a different supporting cast member. Such cannot happen until a different commit is made, for which the missing piece is an UPDATING/CHANGES entry.
Comment 51 Matthias Andree freebsd_committer freebsd_triage 2025-03-04 21:13:46 UTC
(In reply to Charlie Li from comment #50)
That's what I am saying.  Disruptions by vishwin@ because there's an imperfection such as a missing documentation commit.  That doesn't prevent an exp-run, so when is the exp-run starting?
Comment 52 Charlie Li freebsd_committer freebsd_triage 2025-03-04 21:16:35 UTC
(In reply to Matthias Andree from comment #51)
Not an imperfection, but a required component to address questions as to why the change happened/is needed.

The exp-run on the other component is prevented because otherwise the premise for said run is wrong.

Continuing to make comments like this do not help me or others move things along, quite the opposite actually.
Comment 53 Matthias Andree freebsd_committer freebsd_triage 2025-03-04 21:19:13 UTC
You're free to set the ball rolling instead of replying what doesn't work. Instead, show what does work.
Comment 54 j.david.lists 2025-03-07 16:15:19 UTC
Some other languages, notably gcc, have some more volatile versions that can't be fully supported labelled -devel in the ports tree. E.g., lang/gcc14 vs. lang/gcc14-devel.

Maybe it would be worthwhile to add newer versions of Python to the tree as lang/python312-devel or (to be *extremely* clear) lang/python312-unsupported to indicate that they aren't a part of the full ports Uses/versioning scheme yet.

This would give a substantive leg up to users who only need the newer versions to bootstrap a venv.

As a test, I tried setting "DEFAULT_VERSIONS+= python=3.45 python3=3.45" and I got an error immediately. It looks like the same would happen for a suffixed version; the Makefile won't find it.

If that existing error isn't satisfactory, it seems like it would be possible to get Mk/Uses/python.mk to validate the version chosen and kick out a "please meditate on the meaning of the word 'unsupported'" message if a port tries to USES= or a builder tries to DEFAULT_VERSIONS= an unsupported version. It already has similar warnings about Python 2.7.

Could even reiterate that with appropriately dire warnings in the pkg-message for unsupported versions.

It may not be possible to get it to the point that no one will ever try "DEFAULT_VERSIONS+= python3=3.12-unsupported" in their make.conf and then complain that it's not supported. People are going to people. But hopefully it would be more of an occasional amusement and less of an ongoing hassle. And it might well happen less often than "How come Python 3.12 has been out this long and FreeBSD doesn't have it yet?"
Comment 55 Kamil Choudhury 2025-03-22 17:04:03 UTC
Is there any objection to someone just submitting a port for lang/python312-standalone version with a warnings that:

- ensurepip must be invoked to gain access to dependencies
- nothing is plugged into the rest of the python ports machinery
- the user is well and truly on their own

We can deprecate the standalone port when lang/python312 is done.
Comment 56 p5B2EA84B3 2025-03-31 14:54:36 UTC
Please provide the Python3.12 port as is now.
It is desperately needed for development!
Comment 57 p5B2EA84B3 2025-04-01 14:56:30 UTC
I used the the latest attachement to create the badly needed lang/python312 myself and run python3.12 -m ensurepip

So thanks for providing the attachement! This saved me and using FreeBSD at work.

Just a note on porting Python modules for FreeBSD:

I'm VERY unhappy with the situation of py3xx-ports. Therefore I do use Pip wherever possible an avoid using the ports. 
From my view Pip should be the general way installing Python modules on FreeBSD and py3xx ports should only provide those modules where patching is necessary and this in way NOT conflicting with Pip-installed modules.

Also note that Python dropped FreeBSD to lowest Tier 3 level in PEP 11.
https://peps.python.org/pep-0011/#tier-3
Comment 58 Baptiste Daroussin freebsd_committer freebsd_triage 2025-04-02 07:05:29 UTC
vishwin: is adding this version to the tree, with no possibility to enable it via as default python in collision with the part you are working on ? if no then we should proceed with the addition of the 3.12 version and only block adding the python.mk bits until your work has hit the tree

Another question is: where is you work located, and what is missing, my understanding from the discussion, is that some UPDATING/CHANGES bits are missing but, the work is ok pending some exp-run am I right in my understanding, if yes, just to the exp-run, and push your changes if suitable, documentation on UPDATING/CHANGES can be done in a second step or even by someone else. also if you work it public someone should be able to give a hand and write those bits while the exp-run is running.
Comment 59 p5B2EA84B3 2025-04-02 14:43:32 UTC
Please increase  Importance: 	--- Affects Only Me ---> Affects some people
Comment 60 Charlie Li freebsd_committer freebsd_triage 2025-04-05 16:44:06 UTC
(In reply to Baptiste Daroussin from comment #58)
From a technical standpoint, it may be possible to add a new CPython version to the tree without adjusting python.mk. From a political standpoint, it is not. The case of landing new CPython without the corresponding python.mk bits results in complaints over false advertising of supported Python 3.12+ pkg(8)s on every mailing list and random PRs, some of which may include patches to enable 3.12+ prematurely. That's more disorganised than containing everything here in this PR.

Generally I post my stuff on phab, sometimes as PR attachments depending on maintainer habit/preference, only when ready for public testing and feedback. In this case everything relevant has been publicly shared.

As a compromise since I found more tree-wide changes that need to happen before latest upstream setuptools can land (bug 270358 comment 78), I will offer this: convert the existing setuptools (without update) to USE_PYTHON=pep517 so that it stops using the standard library baseutils removed in Python 3.12 and later, then this, 3.13 and associated python.mk changes can land one after the other pretty much simultaneously.

(In reply to p5B2EA84B3 from comment #57)
site-packages under ${PREFIX}/${LOCALBASE} is the exclusive domain of pkg(8), ie externally-managed by pkg(8), so pip is never to be used there. PEP-668 [0][1] allows enforcing this restriction and will come independent of this. Many non-Python programs in our ports tree have Python dependencies so it is a bad idea to mix package management systems in the same hierarchy.

[0] https://peps.python.org/pep-0668/
[1] https://packaging.python.org/en/latest/specifications/externally-managed-environments/#externally-managed-environments
Comment 61 commit-hook freebsd_committer freebsd_triage 2025-04-05 20:37:31 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=de7c5ca4a2d40df82a8ca46b37c8b859a412b89c

commit de7c5ca4a2d40df82a8ca46b37c8b859a412b89c
Author:     Charlie Li <vishwin@FreeBSD.org>
AuthorDate: 2025-04-05 20:12:38 +0000
Commit:     Charlie Li <vishwin@FreeBSD.org>
CommitDate: 2025-04-05 20:12:38 +0000

    devel/py-wheel044: "temporarily" add

    The sole purpose of this port is to build setuptools < 70.1.0 under
    USE_PYTHON=pep517, as a stopgap to allow newer Python
    distributions/interpreters to land while the setuptools update
    continues to be worked on. This port will be removed once setuptools
    is updated. Nothing else is to declare this as a dependency; continue
    to use devel/py-wheel elsewhere.

    As of setuptools 70.1.0, bdist_wheel (what setuptools executes as
    part of ${PEP517_BUILD_CMD} to build the wheel) has moved from wheel
    to setuptools [0], so once setuptools is updated past said version,
    consumers should not continue declaring devel/py-wheel in BUILD_DEPENDS
    unless the package needs functionality beyond what moved into
    setuptools.

    [0] https://setuptools.pypa.io/en/latest/history.html#v70-1-0

    PR: 271673, 274671

 devel/Makefile                                     |  1 +
 devel/py-wheel044/Makefile (new)                   | 26 +++++++++
 devel/py-wheel044/distinfo (new)                   |  3 ++
 .../files/patch-src_wheel___bdist__wheel.py (new)  | 62 ++++++++++++++++++++++
 devel/py-wheel044/pkg-descr (new)                  | 13 +++++
 5 files changed, 105 insertions(+)
Comment 62 commit-hook freebsd_committer freebsd_triage 2025-04-05 20:58:45 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=8b5ae17d2f43388fade30e266615a9e34bf06abd

commit 8b5ae17d2f43388fade30e266615a9e34bf06abd
Author:     Charlie Li <vishwin@FreeBSD.org>
AuthorDate: 2025-04-05 20:45:55 +0000
Commit:     Charlie Li <vishwin@FreeBSD.org>
CommitDate: 2025-04-05 20:45:55 +0000

    devel/py-setuptools: convert to USE_PYTHON=pep517

    The previous method of building/bootstrapping setuptools via the
    built-in distutils module no longer works in Python 3.12+ as it has
    been removed [0]. Since USE_PYTHON=pep517 has its own bootstrapping
    process, use this method to build setuptools.

    This allows newer Python distributions/interpreters to land in the
    tree, for the purpose of having buildable packages.

    While here, remove a slew of dead code and adjust WWW

    [0] https://peps.python.org/pep-0632/

    PR: 271673, 274671

 devel/py-setuptools/Makefile                       | 39 ++++++----------------
 .../files/easy-install.pth.dist (gone)             |  2 --
 .../patch-setuptools_package__index.py (gone)      | 11 ------
 devel/py-setuptools/files/pkg-message.in (gone)    |  8 -----
 4 files changed, 10 insertions(+), 50 deletions(-)
Comment 63 Charlie Li freebsd_committer freebsd_triage 2025-04-05 21:08:56 UTC
After the above two commits, this can land.

(In reply to Charlie Li from comment #60)
Even extracting those two commits was a lot of effort, as my local overlay is often times multiple steps ahead of what I present publicly, both for extensive dogfooding and limited compatibility windows between individual components.
Comment 64 Charlie Li freebsd_committer freebsd_triage 2025-04-06 03:21:54 UTC
Took wen@'s attachment, cleaned it up a bit, and updated to 3.12.9 at review D49679. USE_PYTHON=pep517 successfully bootstraps and setuptools builds, so all packages should be buildable. I plan on committing this in the next day or so but feel free to test for any showstoppers.
Comment 65 commit-hook freebsd_committer freebsd_triage 2025-04-08 02:01:28 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=deb31699f1e7aef1498a7507cf219be6e338332b

commit deb31699f1e7aef1498a7507cf219be6e338332b
Author:     Wen Heping <wen@FreeBSD.org>
AuthorDate: 2025-04-08 01:54:12 +0000
Commit:     Charlie Li <vishwin@FreeBSD.org>
CommitDate: 2025-04-08 01:59:29 +0000

    lang/python312: add 3.12.9

    What's new: https://docs.python.org/3/whatsnew/3.12.html

    PR: 271673
    Co-authored-by: vishwin
    Differential Revision: https://reviews.freebsd.org/D49679

 Mk/Uses/python.mk                                  |    2 +-
 Mk/bsd.default-versions.mk                         |    2 +-
 lang/Makefile                                      |    1 +
 lang/python312/Makefile (new)                      |  160 +
 lang/python312/Makefile.version (new)              |    7 +
 lang/python312/distinfo (new)                      |    3 +
 .../libressl/patch-Modules___hashopenssl.c (new)   |   26 +
 .../files/libressl/patch-Modules___ssl.c (new)     |   11 +
 lang/python312/files/patch-Makefile.pre.in (new)   |   65 +
 .../files/patch-Misc__python-config.sh.in (new)    |   11 +
 lang/python312/files/patch-configure (new)         |   11 +
 lang/python312/files/pkg-message.in (new)          |   12 +
 lang/python312/pkg-descr (new)                     |    2 +
 lang/python312/pkg-plist (new)                     | 7977 ++++++++++++++++++++
 14 files changed, 8288 insertions(+), 2 deletions(-)
Comment 66 Baptiste Daroussin freebsd_committer freebsd_triage 2025-04-08 10:00:20 UTC
thank you for the resolution!