Bug 230140

Summary: textproc/py-sphinx: Update to 1.8.1
Product: Ports & Packages Reporter: Yuri Victorovich <yuri>
Component: Individual Port(s)Assignee: Danilo G. Baio <dbaio>
Status: Closed FIXED    
Severity: Affects Many People CC: bwidawsk, dbaio, koobs, lwhsu, portmgr, python
Priority: Normal Flags: antoine: exp-run?
Version: Latest   
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=245629
Bug Depends on:    
Bug Blocks: 233010    
Attachments:
Description Flags
patch
none
Updated patch
none
sphinx-1.8.1.diff none

Description Yuri Victorovich freebsd_committer freebsd_triage 2018-07-29 06:03:56 UTC
Created attachment 195582 [details]
patch
Comment 1 Marcelo Araujo freebsd_committer freebsd_triage 2018-07-30 03:04:03 UTC
LGTM, but could you ask to run an exprun over the ports tree? There are plenty of other ports that depends of py-sphinx!
Comment 2 Yuri Victorovich freebsd_committer freebsd_triage 2018-07-30 03:41:54 UTC
portmgr, the maintainer requests the exprun.
Comment 3 Antoine Brodin freebsd_committer freebsd_triage 2018-07-31 21:20:35 UTC
This is bad,  the dependencies of sphinx are not correct.  How was this tested?
Comment 4 Yuri Victorovich freebsd_committer freebsd_triage 2018-07-31 22:57:07 UTC
Created attachment 195705 [details]
Updated patch

My bad, I didn't test dependencies.
Now I've built several dependencies fine.
Comment 5 Antoine Brodin freebsd_committer freebsd_triage 2018-08-05 07:15:30 UTC
Around 30 ports were not tested due to new failures.

New failures:

+ {"origin"=>"devel/ahven", "phase"=>"build", "errortype"=>"bad_C++_code"}
+ {"origin"=>"devel/cmake-doc", "phase"=>"package", "errortype"=>"bad_C++_code"}
+ {"origin"=>"devel/kf5-extra-cmake-modules", "phase"=>"package", "errortype"=>"bad_C++_code"}
+ {"origin"=>"devel/llvm33", "phase"=>"package", "errortype"=>"bad_C++_code"}
+ {"origin"=>"devel/llvm34", "phase"=>"package", "errortype"=>"bad_C++_code"}
+ {"origin"=>"devel/llvm35", "phase"=>"package", "errortype"=>"bad_C++_code"}
+ {"origin"=>"devel/py-noseofyeti", "phase"=>"build", "errortype"=>"bad_C++_code"}
+ {"origin"=>"devel/py-pathtools", "flavor"=>"py36", "phase"=>"package", "errortype"=>"bad_C++_code"}
+ {"origin"=>"devel/py-pathtools", "phase"=>"package", "errortype"=>"bad_C++_code"}
+ {"origin"=>"devel/py-virtualenvwrapper", "phase"=>"package", "errortype"=>"bad_C++_code"}
+ {"origin"=>"devel/shiboken", "phase"=>"package", "errortype"=>"bad_C++_code"}
+ {"origin"=>"math/yacas", "phase"=>"build", "errortype"=>"bad_C++_code"}
+ {"origin"=>"net/py-gntp", "phase"=>"package", "errortype"=>"bad_C++_code"}
+ {"origin"=>"textproc/py-cloud_sptheme", "flavor"=>"py36", "phase"=>"build", "errortype"=>"bad_C++_code"}
+ {"origin"=>"textproc/py-cloud_sptheme", "phase"=>"build", "errortype"=>"bad_C++_code"}
+ {"origin"=>"www/py-cssselect", "flavor"=>"py36", "phase"=>"package", "errortype"=>"bad_C++_code"}
+ {"origin"=>"www/py-cssselect", "phase"=>"package", "errortype"=>"bad_C++_code"}

New failure logs:

http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/ahven-2.6_4.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/cmake-doc-3.12.0.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/kf5-extra-cmake-modules-5.48.0.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/llvm33-3.3_14.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/llvm34-3.4.2_10.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/llvm35-3.5.2_8.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/py27-noseofyeti-1.5.1.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/py36-pathtools-0.1.2_4.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/py27-pathtools-0.1.2_4.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/py27-virtualenvwrapper-4.8.2_1.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/shiboken-1.2.2_3.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/yacas-1.6.0_3.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/py27-gntp-0.8_5.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/py36-cloud_sptheme-1.9.4.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/py27-cloud_sptheme-1.9.4.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/py36-cssselect-0.9.1_2.log
http://package18.nyi.freebsd.org/data/104amd64-default-PR230140/2018-08-04_12h34m13s/logs/errors/py27-cssselect-0.9.1_2.log
Comment 6 Yuri Victorovich freebsd_committer freebsd_triage 2018-08-05 07:46:09 UTC
Some failures are due to the hardcoded Sphinx version in the file names. Some other failures are due to possible incompatible interface changes.

The maintainer needs to resolve this.

Thanks, Antoine!
Comment 7 Antoine Brodin freebsd_committer freebsd_triage 2018-08-05 08:54:45 UTC
(In reply to Yuri Victorovich from comment #6)
No, you have to work with the maintainers to resolve this and propose a new patch for exp-run.
Comment 8 Kubilay Kocak freebsd_committer freebsd_triage 2018-08-07 23:57:21 UTC
I also have a patch in flight for this. Any issues/bugs with ports that require changes there (in them), should be created as blocking sub-tasks to this issue.

@Marcelo Given the importance of this port for the python port ecosystem, would you mind if maintainership was changed to python@, either during or after this change? We do this for a number of other major/infrastructure ports (setuptools, pip, etc).
Comment 9 Marcelo Araujo freebsd_committer freebsd_triage 2018-08-08 01:54:41 UTC
(In reply to Kubilay Kocak from comment #8)

I would not mind at all, if you have a patch under the way. you can update maintainership to python@, otherwise I don't see a reason to do that prior a patch.
Comment 10 Li-Wen Hsu freebsd_committer freebsd_triage 2018-11-07 20:52:55 UTC
Created attachment 199063 [details]
sphinx-1.8.1.diff

Update patch to 1.8.1 and grab maintainership for python@

Also for textproc/py-sphinxcontrib-websupport because it is a direct dependency of textproc/py-sphinx
Comment 11 Li-Wen Hsu freebsd_committer freebsd_triage 2018-11-07 20:54:09 UTC
Request exprun again
Comment 12 Antoine Brodin freebsd_committer freebsd_triage 2018-11-07 21:00:38 UTC
Where is the patch for all previously detected failures?
Comment 13 Danilo G. Baio freebsd_committer freebsd_triage 2020-04-18 17:12:19 UTC
Hi.

Taking care of it in bug 245629 with a different strategy.

- Repocopy textproc/py-sphinx to textproc/py-sphinx18, then upgrade it to 1.8.5 (latest version from 1.8.X).

  This version supports Python 2 and 3.
  Add test target.

- textproc/py-sphinx: Update to 3.0.1

  Python 3 only, 3.5+
  Add test target.
  Add new ports (PR's will be linked here)

- Mk/Uses/python.mk: Add PY_SPHINX

  To use with flavors and not break ports with USES=python (all versions).
    Python >=3.5  --> textproc/py-sphinx (v3.0.1)
    Python < 3.5  --> textproc/py-sphinx18 (v1.8.5)


More details in bug 245629.

Assign this bug to python@, current maintainer of sphinx.
Comment 14 Li-Wen Hsu freebsd_committer freebsd_triage 2020-04-20 08:31:05 UTC
(In reply to Danilo G. Baio from comment #13)
I like this approach, this gets us a more recent version of sphinx for more and more ports require it and also takes care of the ports require old version. I feel we can mark this one overcome by event and request another exp-run against the patch in bug245629.
Comment 15 Kubilay Kocak freebsd_committer freebsd_triage 2020-04-20 10:49:53 UTC
(In reply to Li-Wen Hsu from comment #14)

This can be resolved FIXED when a 1.8.x update lands (pre or post repocopy)

I'm -1 on the addition of another shared macro (see bug 243665 c10)
Comment 16 Li-Wen Hsu freebsd_committer freebsd_triage 2020-04-20 11:46:12 UTC
(In reply to Kubilay Kocak from comment #15)
I'm fine for having the dependency line in the Makefile of individual port (maybe I'm more prefer to that.)

I want to point out that this ticket gets stuck because of it's not that easy to fix all the fallouts of the upgrade, but there are new ports want to be added because of needing new sphinx (ex: bug233010 ), which is the least thing I want to see. I suggest:

- Let's get a new exp-run or just do a self poudriere build against the 3.0.2
  (IIRC some of the fallouts in comment5 got fixed by newer version (1.8.x in that time), but I forgot to mention in that comment)
- For fallouts, let's check if them needs 1.8.x or other fixes, or even can be deprecated.

And I suggest we can set a time to revise (and deprecate) 1.8.x along with python 2 in ports.

Any comments?
Comment 17 Mathieu Arnold freebsd_committer freebsd_triage 2020-04-20 11:58:21 UTC
Why not simply update sphinx to the latest version and remove all the ports that really really need 2.7.  Those have had 7 or 8 years to switch to Python 3 and are considered abandonware.

All you would be doing is add more work now that will be removed in the next few months.
Comment 18 Li-Wen Hsu freebsd_committer freebsd_triage 2020-04-20 12:11:32 UTC
(In reply to Mathieu Arnold from comment #17)
Oh, that would also work, if all the ports require old sphinx are because of python 2, we can schedule to remove them.

The point is, it's (was) not related to python versions, but also sphinx versions.  It was the reason this updating to 1.8.x patch cannot get in.  What I suggested was  adding newer sphinx because of there is need, and set a deadline for old sphinx. I thought along with python 2 is probably a good timing.
Comment 19 Danilo G. Baio freebsd_committer freebsd_triage 2020-04-20 12:20:12 UTC
The latest version of Sphinx is 3.0.2, there are a few ports that doens't build with it, e.g. the full set of LLVM, more details on bug 245629, it's not just an issue with Python 2.7 here.

Sphinx 1.8 is not the latest, if we use this version, it will continue to be outdated.

Regarding the new shared macro for Sphinx, it can be gone when Python 2.7 is removed, the work is done, see bug 245629 please.
Comment 20 Mathieu Arnold freebsd_committer freebsd_triage 2020-04-20 17:40:17 UTC
Well sphinx is usually used for generating docs, so ports that need 2.7 can loose their extra documentation either until they get updated or get removed.
Comment 21 Danilo G. Baio freebsd_committer freebsd_triage 2020-04-21 14:50:32 UTC
This sphinx update was already postponed for a long time.

If we have two versions of sphinx in tree, which is very likely, a shared macro will be necessary to organize it (as described in comment 13).

Adding the shared macro and having two versions of sphinx is the simple path/harmless IMHO and can be done fast/now, and we won't lose man pages or docs from ports that cannot be built with sphinx 3.X yet. I would go with it.

I've updated bug 245629 with Sphinx to 3.0.2.
Comment 22 Danilo G. Baio freebsd_committer freebsd_triage 2020-05-11 23:58:45 UTC
bug #245629  ports r534966