Bug 249337

Summary: [meta] Ports broken by Python 2.7 End-of-Life and removal
Product: Ports & Packages Reporter: Steve Wills <swills>
Component: Individual Port(s)Assignee: freebsd-python (Nobody) <python>
Status: Closed Overcome By Events    
Severity: Affects Many People CC: ahmedsayeed1982, dch, dev.ashevchuk, emaste, grahamperrin, koobs, lbartoletti, mandree, portmaster, rene, rhurlin, swills, tcberner
Priority: Normal    
Version: Latest   
Hardware: Any   
OS: Any   
URL: https://reviews.freebsd.org/D28665
Bug Depends on: 225543, 226416, 237776, 242494, 244540, 244541, 244542, 245519, 246013, 246978, 247278, 247308, 247342, 249302, 249341, 249345, 249484, 249534, 249549, 249550, 249570, 249573, 249576, 249577, 249580, 249581, 249582, 249583, 249584, 249585, 249586, 249587, 249588, 249589, 249590, 249591, 249592, 249593, 249594, 249595, 249596, 249597, 249598, 249599, 249600, 249601, 249602, 249603, 249604, 249605, 249606, 249607, 249608, 249609, 249610, 249611, 249612, 249613, 249614, 249615, 249616, 249617, 249618, 249619, 249620, 249621, 249622, 249623, 249624, 249625, 249626, 249627, 249628, 249629, 249630, 249631, 249632, 249633, 249634, 249635, 249636, 249637, 249638, 249639, 249640, 249641, 249642, 249643, 249644, 249645, 249646, 249647, 249648, 249649, 249650, 249651, 249652, 249653, 249654, 249655, 249656, 249657, 249658, 249659, 249660, 249661, 249662, 249663, 249664, 249665, 249666, 249667, 249668, 249669, 249670, 249671, 249672, 249673, 249674, 249675, 249676, 249677, 249678, 249679, 249680, 249681, 249682, 249683, 249684, 249685, 249686, 249687, 249688, 249689, 249690, 249691, 249692, 249693, 249694, 249695, 249696, 249697, 249698, 249699, 249700, 249701, 249702, 249703, 249704, 249705, 249706, 249707, 249708, 249709, 249710, 249711, 249712, 249713, 249714, 249715, 249716, 249717, 249718, 249719, 249720, 249721, 249722, 249723, 249724, 249725, 249726, 249727, 249728, 249729, 249730, 249731, 249732, 249733, 249734, 249735, 249736, 249737, 249738, 249739, 249740, 249741, 249742, 249743, 249744, 249745, 249746, 249747, 249748, 249749, 249750, 249751, 249752, 249753, 249754, 249755, 249756, 249757, 249758, 249759, 249760, 249762, 249763, 249764, 249765, 249766, 249767, 249768, 249769, 249770, 249771, 249772, 249773, 249774, 249775, 249776, 249777, 249778, 249779, 249780, 249781, 249782, 249783, 249784, 249785, 249786, 249787, 249788, 249789, 249790, 249791, 249792, 249793, 249794, 249795, 249796, 249797, 249798, 249799, 249800, 249801, 249802, 249803, 249804, 249805, 249806, 249807, 249808, 249809, 249810, 249811, 249812, 249813, 249814, 249815, 249816, 249817, 249818, 249819, 249820, 249821, 249822, 249823, 249824, 249825, 249826, 249827, 249828, 249829, 249830, 249831, 249832, 249833, 249834, 249835, 249836, 249837, 249838, 249839, 249840, 249841, 249842, 249843, 249844, 249845, 249846, 249847, 249848, 249849, 249850, 249851, 249852, 249853, 249854, 249855, 249856, 249857, 249858, 249862, 251019    
Bug Blocks: 249547    
Attachments:
Description Flags
raw list of things that directly depend on python 2
none
info on python2 deps none

Description Steve Wills freebsd_committer freebsd_triage 2020-09-15 15:16:21 UTC
Meta ticket for tracking issues and fixes related to removing lang/python27
Comment 1 Steve Wills freebsd_committer freebsd_triage 2020-09-24 14:05:17 UTC
Created attachment 218241 [details]
raw list of things that directly depend on python 2

Here's a current raw list of things that directly depend on python 2.x. Some of these are already in progress, etc. just want to put the list here for the record.
Comment 2 Steve Wills freebsd_committer freebsd_triage 2020-09-24 16:49:25 UTC
Created attachment 218245 [details]
info on python2 deps

I've generated bug reports for all the ports that I could find that directly depend on python 2. The goal was to ensure there are bugs for each of them for tracking purposes. My apologies for the brevity of these reports and the duplicate PRs.

And FWIW, here is a list of how many ports depend on each of these. In case it's useful, here's a histogram of the numbers:

0: 93
1: 105
2: 45
3: 16
4: 4
5: 4
6: 1
7: 2
8: 1
9: 1
11: 1
18: 1
29: 1
33: 1
36: 1
47: 1
99: 1

Point being, at the moment, there are a handful of more important ports (qt5-webengine, chromium, etc.) that are well known, and a larger number of ones which nothing or only a few things depend on.
Comment 3 Rainer Hurling freebsd_committer freebsd_triage 2020-09-25 15:49:03 UTC
Comment on attachment 218241 [details]
raw list of things that directly depend on python 2

(Don't know if this PR is meant to be answered?)


I think, graphics/mapserver is a false positive. The port should build and run with Python 3.

MapServer itself more or less is ready for Python3 since 7.2.0-beta1 release (2018-05-09)[1]


[1] https://github.com/MapServer/MapServer/blob/branch-7-6/HISTORY.TXT
Comment 4 Steve Wills freebsd_committer freebsd_triage 2020-09-25 17:17:13 UTC
(In reply to Rainer Hurling from comment #3)
Yes, see r549981
Comment 5 Rainer Hurling freebsd_committer freebsd_triage 2020-09-25 17:30:51 UTC
(In reply to Steve Wills from comment #4)

Oops, yes. I have missed it, sorry ;)
Comment 6 Matthias Andree freebsd_committer freebsd_triage 2020-11-29 20:14:14 UTC
What efforts are underway to evaluate and deploy python 2.x compatible interpreters, such as Tauthon, or someone else was mentioning PyPy?  Anyone aware of these?

I am asking for my port (mail/mailman) where upstream declined a Python 3.x migration and instead decided to write a new piece of software with a completely different architecture and requirements and feature set that was coincidentally dubbed mailman-3, but is not a viable migration path from mailman-2 for lack of features, lack of migration aids, and more. 
(Mailman is a mailing list driver with web front-end, archival, self-service and other features.)

I suspect there may be a few more ports that would otherwise have to go, and measuring branch lengths and counting leaves is not an appropriate metric to determine the importance of a port.  

We may only know we've expired one too many port when users shout at us in 2021, or, worse, silently turn away from FreeBSD.
Comment 7 Matthias Andree freebsd_committer freebsd_triage 2020-12-02 16:55:14 UTC
So, with the contribution of Tauthon, let's see how many ports that cannot migrate to Python 3.x for whatever reason (mailman 2, where mailman 3 is no substitute) could migrate to Tauthon instead, and thus be removed from the blockers list for this meta/tracking bug. I've marked 251019 a blocker to this.
Comment 8 commit-hook freebsd_committer freebsd_triage 2020-12-16 01:45:41 UTC
A commit references this bug:

Author: linimon
Date: Wed Dec 16 01:45:02 UTC 2020
New revision: 558197
URL: https://svnweb.freebsd.org/changeset/ports/558197

Log:
  Defuse a ticking time-bomb by removing the dependency on devel/bzr.

  It is likely that this will break "make test", so, why am I doing it?

  The answer is that devel/bzr unconditionally depends on python27 (it is
  known not to build with python37) -- and python27 is imminently due for
  removal (see PR 249337).

  This would not matter so much except that py-pip winds up a dependency
  of devel/cmake, and thus, over 2000 ports -- all of which would have
  broken when python27 was removed.

  Unfortunately the time before that removal happens is simply too short
  to have a proper discussion with all stakeholders.  We need to remove
  the dependency immediately so that PR 249337 can proceed.

  PR:		249337 (related to)

Changes:
  head/devel/py-pip/Makefile
Comment 9 Chris Hutchinson 2020-12-16 03:35:39 UTC
(In reply to commit-hook from comment #8)
FWIW I can take a stab at getting devel/bzr on py3x.
I'm pretty confident I can get it done before the
(py2x) deadline. I should have something before the
end of the week.

--Chris
Comment 10 Steve Wills freebsd_committer freebsd_triage 2020-12-16 04:01:35 UTC
(In reply to Chris Hutchinson from comment #9)
Might be better to create a port for Breezy:

https://www.breezy-vcs.org/
Comment 11 Chris Hutchinson 2020-12-16 05:56:28 UTC
(In reply to Steve Wills from comment #10)
LOL figures. I'm about 2/3 done with devel/bzr.
I already maintain devel/codeville and devel/tailor
which both have support for bzr, either directly,
or indirectly.
I only took on BZR here, because it appeared many
other ports in the tree used/depended on it.
Nonetheless, Breezy looks interesting. I'll take
a closer look as soon as I finish bzr (since I'm
already nearly done). :-)

Thanks for pointing it out Steve! :-)
It looks to be a "friendlier" alternative.

--Chris
Comment 12 Steve Wills freebsd_committer freebsd_triage 2021-01-22 17:58:31 UTC
Here's an updated list of things with USES=python:2*:

audio/xmms2
chinese/sunpinyin
devel/electron9
devel/gnatpython
devel/libclc
devel/libhid
devel/libosmocore
devel/py-enum34
devel/rbtools
devel/subversion-lts
devel/viewvc
games/0ad
games/renpy
lang/pypy
lang/spidermonkey185
lang/spidermonkey52
lang/spidermonkey60
mail/mailman
math/gringo
multimedia/mimms
sysutils/uefi-edk2-bhyve
sysutils/uefi-edk2-bhyve-devel
www/chromium
www/mattermost-rss
www/moinmoin
www/node10
www/node12
Comment 13 Steve Wills freebsd_committer freebsd_triage 2021-01-22 22:34:40 UTC
(In reply to Steve Wills from comment #12)
Sorry, ignore www/mattermost-rss, that's one I have locally and haven't committed.
Comment 14 Rene Ladan freebsd_committer freebsd_triage 2021-02-02 22:52:00 UTC
(In reply to Steve Wills from comment #12)
www/qt5-webengine is also a direct consumer of Python 2.7
Comment 15 Dave Cottlehuber freebsd_committer freebsd_triage 2021-04-07 21:03:14 UTC
swills: not sure where best to mention this, but I've:
- removed databases/couchdb2 (dependency on python2 spidermonkeys)
- working on getting databases/couchdb3 to support python3 spidermonkey78 & 86
Comment 16 Ahmed 2021-11-02 19:49:03 UTC
MARKED AS SPAM
Comment 17 Tobias C. Berner freebsd_committer freebsd_triage 2022-09-07 15:35:30 UTC
Moin moin 

Closing this due to old age. Python 2.7 has been handled, as far as it can.

mfg Tobias