Bug 271673 - lang/python312: New port, update to 3.12.7
Summary: lang/python312: New port, update to 3.12.7
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-python (Nobody)
URL:
Keywords:
Depends on: 268283 270358 270510
Blocks:
  Show dependency treegraph
 
Reported: 2023-05-27 11:50 UTC by Wen Heping
Modified: 2024-11-02 23:35 UTC (History)
23 users (show)

See Also:


Attachments
- 3.12.0 beta 1 (41.12 KB, application/x-gzip)
2023-05-27 11:50 UTC, Wen Heping
no flags Details
python-3.12.0 beta2 (41.36 KB, application/x-gzip)
2023-06-08 02:11 UTC, Wen Heping
no flags Details
python-3.12.0 RC1 (41.42 KB, application/x-gzip)
2023-08-07 08:04 UTC, Wen Heping
no flags Details
python-3.12.0rc2 (41.08 KB, application/x-gzip)
2023-09-07 09:38 UTC, wen
no flags Details
Update to 3.12.0rc3 (41.37 KB, application/x-gzip)
2023-09-22 10:30 UTC, Wen Heping
no flags Details
Update to 3.12.0 Final (41.37 KB, application/x-gzip)
2023-10-03 11:27 UTC, Wen Heping
no flags Details
update to 3.12.1 (41.58 KB, application/x-gzip)
2023-12-09 07:57 UTC, Wen Heping
no flags Details
Update to 3.12.2 (41.56 KB, application/x-gzip)
2024-02-08 12:32 UTC, Wen Heping
no flags Details
Update to 3.12.3 (42.25 KB, application/x-gzip)
2024-04-12 09:17 UTC, Wen Heping
no flags Details
Update to 3.12.4 (42.27 KB, application/x-gzip)
2024-06-07 08:42 UTC, Wen Heping
no flags Details
Update to 3.12.5 (42.36 KB, application/x-gzip)
2024-08-08 01:21 UTC, Wen Heping
no flags Details
Update to 3.12.7 (42.41 KB, application/x-gzip)
2024-10-03 02:19 UTC, Wen Heping
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Wen Heping freebsd_committer freebsd_triage 2023-05-27 11:50:26 UTC
Created attachment 242441 [details]
- 3.12.0 beta 1

New port, now is 3.12.0 beta 1.
Comment 1 Wen Heping freebsd_committer freebsd_triage 2023-05-27 11:53:30 UTC
Shall we import python-3.12.0 just for test, currently not hook it into Uses/python.mk ?
Comment 2 Wen Heping freebsd_committer freebsd_triage 2023-05-27 11:53:59 UTC
The test result:

== Tests result: FAILURE ==

434 tests OK.

10 slowest tests:
- test_tools: 5 min 34 sec
- test_smtpnet: 5 min
- test_signal: 3 min 56 sec
- test_concurrent_futures: 2 min 10 sec
- test_socket: 1 min 46 sec
- test_multiprocessing_spawn: 1 min 36 sec
- test_multiprocessing_forkserver: 1 min 17 sec
- test_multiprocessing_fork: 1 min 10 sec
- test_nntplib: 1 min 6 sec
- test_io: 35.4 sec

5 tests failed:
    test_dtrace test_posix test_shutil test_tempfile test_tools

1 test altered the execution environment:
    test_capi

27 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils test_dbm_gnu test_devpoll
    test_epoll test_gdb test_idle test_ioctl test_launcher test_msilib
    test_peg_generator test_perf_profiler test_perfmaps test_spwd
    test_sqlite3 test_startfile test_tcl test_tix test_tkinter
    test_ttk test_ttk_textonly test_turtle test_winconsoleio
    test_winreg test_winsound test_wmi test_zipfile64
0:18:51 load avg: 0.30
0:18:51 load avg: 0.30 Re-running failed tests in verbose mode
0:18:51 load avg: 0.30 Re-running test_tools in verbose mode (matching: test_freeze_simple_script)
test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script) ... creating the script to be frozen at /tmp/tmps_u2puux/app.py
copying the source tree into /tmp/tmps_u2puux/cpython...
configuring python in /tmp/tmps_u2puux/python-build...
CalledProcessError: Command '['/tmp/tmps_u2puux/cpython/python', '-c', 'import sysconfig; print(sysconfig.get_config_var("CONFIG_ARGS"))']' returned non-zero exit status 1.
--- STDOUT ---

--- STDERR ---
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/tmps_u2puux/cpython/Lib/sysconfig.py", line 740, in get_config_var
    return get_config_vars().get(name)
           ^^^^^^^^^^^^^^^^^
  File "/tmp/tmps_u2puux/cpython/Lib/sysconfig.py", line 723, in get_config_vars
    _init_config_vars()
  File "/tmp/tmps_u2puux/cpython/Lib/sysconfig.py", line 670, in _init_config_vars
    _init_posix(_CONFIG_VARS)
  File "/tmp/tmps_u2puux/cpython/Lib/sysconfig.py", line 536, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_sysconfigdata__freebsd14_'

---- END ---
Comment 3 Charlie Li freebsd_committer freebsd_triage 2023-05-27 12:08:29 UTC
Not a good idea to import until packaging/setuptools particularly is fixed up. Will cause too much distracting noise on mailing lists and such from those trying to test it with their packages.
Comment 4 Daniel Engberg freebsd_committer freebsd_triage 2023-05-27 12:48:19 UTC
A few core libs are also going away / being replaced
https://peps.python.org/pep-0594/
Comment 5 Wen Heping freebsd_committer freebsd_triage 2023-06-08 02:11:44 UTC
Created attachment 242677 [details]
python-3.12.0 beta2

Python-3.12.0 beta2
Comment 6 Wen Heping freebsd_committer freebsd_triage 2023-08-07 08:04:05 UTC
Created attachment 243920 [details]
python-3.12.0 RC1

python-3.12.0 RC1.

Result of `make test`:
439 tests OK.

10 slowest tests:
- test_tools: 6 min 1 sec
- test_socket: 4 min 15 sec
- test_smtpnet: 3 min 10 sec
- test_concurrent_futures: 2 min 7 sec
- test_multiprocessing_fork: 1 min 28 sec
- test_multiprocessing_spawn: 1 min 28 sec
- test_multiprocessing_forkserver: 1 min 17 sec
- test_ssl: 1 min 12 sec
- test_nntplib: 1 min 3 sec
- test_signal: 46.9 sec

2 tests failed:
    test_dtrace test_tools

26 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils test_dbm_gnu test_devpoll
    test_epoll test_gdb test_idle test_ioctl test_launcher test_msilib
    test_perf_profiler test_perfmaps test_spwd test_sqlite3
    test_startfile test_tcl test_tix test_tkinter test_ttk
    test_ttk_textonly test_turtle test_winconsoleio test_winreg
    test_winsound test_wmi test_zipfile64

2 re-run tests:
    test_dtrace test_tools

Total duration: 15 min 25 sec
Comment 7 wen 2023-09-07 07:05:37 UTC
python-3.12.0RC2:

453 tests OK.

10 slowest tests:
- test_smtpnet: 5 min
- test_math: 3 min 25 sec
- test_tarfile: 2 min 38 sec
- test_subprocess: 2 min 30 sec
- test.test_multiprocessing_spawn.test_processes: 2 min 10 sec
- test.test_multiprocessing_forkserver.test_processes: 1 min 49 sec
- test_signal: 1 min 48 sec
- test_tokenize: 1 min 36 sec
- test.test_multiprocessing_fork.test_processes: 1 min 32 sec
- test_zipfile: 1 min 25 sec

2 tests failed:
    test_dtrace test_tempfile

26 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils test_dbm_gnu test_devpoll
    test_epoll test_gdb test_idle test_ioctl test_launcher test_msilib
    test_perf_profiler test_perfmaps test_spwd test_sqlite3
    test_startfile test_tcl test_tix test_tkinter test_ttk
    test_ttk_textonly test_turtle test_winconsoleio test_winreg
    test_winsound test_wmi test_zipfile64

4 re-run tests:
    test_dtrace test_shutil test_tempfile test_tools

2 tests run no tests:
    test_shutil test_tools

Total duration: 30 min 7 sec
Total tests: run=40,183 failures=8 skipped=1,274
Total test files: success=453 failed=2 skipped=26 resource_denied=1 rerun=4 run_no_tests=2
Comment 8 wen 2023-09-07 09:38:30 UTC
Created attachment 244689 [details]
python-3.12.0rc2
Comment 9 Wen Heping freebsd_committer freebsd_triage 2023-09-22 10:30:26 UTC
Created attachment 245113 [details]
Update to 3.12.0rc3

Python 3.12.0rc3
Comment 10 Wen Heping freebsd_committer freebsd_triage 2023-10-03 11:27:20 UTC
Created attachment 245399 [details]
Update to 3.12.0 Final

- Update to 3.12.0 Final
Comment 11 Mohamed Akram 2023-11-24 19:45:18 UTC
Any update on this?
Comment 12 Wen Heping freebsd_committer freebsd_triage 2023-12-09 07:57:00 UTC
Created attachment 246924 [details]
update to 3.12.1

Update to 3..12.1
Comment 13 Henrich Hartzer 2023-12-30 01:46:48 UTC
I'm also interested in this. Could this be merged? Would be great to have it before 2023Q1!
Comment 14 Miroslav Lachman 2024-01-07 08:49:54 UTC
Is there some known problem that prevents lang/python312 from being committed to the ports tree?
Comment 15 Charlie Li freebsd_committer freebsd_triage 2024-01-07 12:23:26 UTC
Not until the setuptools-related PRs linked are completed. Otherwise you will not have many, if at all, Python package ports to use due to our assumption that distutils is available in the standard library, which is no longer the case in 3.12. The road to getting that done is delicate and must be in a specific order to minimise breakage and distractions.
Comment 16 Wen Heping freebsd_committer freebsd_triage 2024-02-08 12:32:55 UTC
Created attachment 248255 [details]
Update to 3.12.2

Update to 3.12.2
Comment 17 Wen Heping freebsd_committer freebsd_triage 2024-04-12 09:17:27 UTC
Created attachment 249927 [details]
Update to 3.12.3

Update to 3.12.3
Comment 18 Amar Takhar 2024-05-22 18:21:19 UTC
This fails to install for me:

===>  Installing for python312-3.12.3
===>  Checking if python312 is already installed
===>   Registering installation for python312-3.12.3
pkg-static: Unable to access file /usr/ports/lang/python312/work/stage/usr/local/man/man1/python3.12.1.gz:No such file or directory
Comment 19 Amar Takhar 2024-05-22 18:21:39 UTC
This fails to install for me:

===>  Installing for python312-3.12.3
===>  Checking if python312 is already installed
===>   Registering installation for python312-3.12.3
pkg-static: Unable to access file /usr/ports/lang/python312/work/stage/usr/local/man/man1/python3.12.1.gz:No such file or directory
Comment 20 Amar Takhar 2024-05-22 18:22:28 UTC
Sorry for the double comment Bugzilla was lagging didn't realise I pressed Save twice.
Comment 21 Wen Heping freebsd_committer freebsd_triage 2024-06-07 08:42:39 UTC
Created attachment 251264 [details]
Update to 3.12.4

Update to 3.12.4
Comment 22 takeda 2024-07-05 20:26:45 UTC
(In reply to Charlie Li from comment #15)

Python 3.12 was released 9 months ago and 3.13 will be soon released.

It looks like we are now having issues because we were clever and released core parts of python as a separate packages and now removal of distutils complicates this version.

Why don't we do the same thing we do with other packages and provide options to enable/disable them? There is much more of dependencies within python than just sqlite that are optional and can be disabled, so singling it out and making it a separate package only adds more work for us):

https://github.com/NixOS/nixpkgs/blob/bc50dd54e8caaa8e3718cc93d2cf32d2aefdc3c6/pkgs/development/interpreters/python/default.nix#L103-L130
Comment 23 Charlie Li freebsd_committer freebsd_triage 2024-07-05 20:37:50 UTC
(In reply to takeda from comment #22)
It doesn't work like that. What nix does is not really relevant to us here.

The distutils deprecation in 3.11 and subsequent removal in this version is a much bigger problem than just separating out sqlite et al from the base Python distribution. Fixing the separated components are relatively easy despite some caveats (namely PEP-517 is only intended for Python 3). They are only separate ports because building some of the libraries they bind would result in circular dependencies back to Python itself (esp if the library uses meson or ninja to build), and not because they are truly optional parts of the base Python distribution.
Comment 24 takeda 2024-07-05 22:34:48 UTC
(In reply to Charlie Li from comment #23)

I see, I was actually wondering why they had the Python3Minimal, was kind of specific and they didn't provide separate one for each python version. I guess that was their solution for this chicken and egg problem.
Comment 25 Wen Heping freebsd_committer freebsd_triage 2024-08-08 01:21:26 UTC
Created attachment 252600 [details]
Update to 3.12.5

Update to 3.12.5
Comment 26 Eren Türkay 2024-09-15 09:37:33 UTC
I've compiled and installed 3.12.5 and it seems to be working fine except sqlite3 module. I cannot get IPython working since it depends on sqlite3. Virtual environments work just fine.

Is there any update regarding to this issue?
Comment 27 Alexey Vyskubov 2024-09-15 11:07:05 UTC
JFYI: I have 3.12.6 installed with mise, and everything seems to work.

🌐 eldanna in ~
❯ ipython
Python 3.12.6 (main, Sep  8 2024, 13:55:00) [GCC 13.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.26.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: print((1+2)*3)
9

In [2]: import sqlite3

In [3]:
Do you really want to exit ([y]/n)?
🌐 eldanna in ~ took 47s
❯ freebsd-version
14.1-RELEASE-p4
Comment 28 Eren Türkay 2024-09-15 12:10:03 UTC
> JFYI: I have 3.12.6 installed with mise, and everything seems to work.

I believe this is due to the fact that mise includes sqlite3 module when compiling Python. I've compiled using the diff provided in this bug report and I have an issue with sqlite3 module. It's supposed to be provided apart from the package itself.

I think there is an ongoing discussion on what to do with those modules.
Comment 29 Charlie Li freebsd_committer freebsd_triage 2024-09-15 12:16:46 UTC
There is no discussion regarding the sqlite3 (and by extension others nominally in the base distribution) module; it will continue to be provided as separate port due to circular dependencies. Any pain point pertains to continued Python 2 (lang/python27) support while Python 3 marches on.

(In reply to Eren Türkay from comment #26)
All dependent bugs must be fixed (two in a specific order) before this can proceed. Committing this out of order will result in basically no Python package ports available for use with this version, along with endless amounts of build fallouts and bickering to the same.
Comment 30 Wen Heping freebsd_committer freebsd_triage 2024-10-03 02:19:41 UTC
Created attachment 253972 [details]
Update to 3.12.7

Update to 3.12.7
Comment 31 dan 2024-11-02 23:30:35 UTC
I'd have to argue just having the lang port without any module ports for 3.12/3.13 would be nice. pip should be main way to install new packages. What is frustrating is when you have modules for a python lang in pkg that can overwrite ones you had installed with pip.
Comment 32 dan 2024-11-02 23:35:27 UTC
personally I install pip-review and execute
pip-review --local --interactive

This keeps you up to date faster than using FreeBSD pkg.