Created attachment 238375 [details] py-sphinx update to 5.3.0 using the new py build system textproc/py-sphinx is presently available in a version 5.3.0. An update for py-sphinx may be of interest when updating some ports that use sphinx, e.g devel/py-nbclassic (if updating to 0.4.8) and devel/py-pip (if updating to 22.3.1) The attached diff introduces one approach to addressing the following concerns, such that might be encountered in any approach updating to 5.3.0 in py-sphinx; - Sphinx 5.3.0 uses PEP 517 conventions for the build. - This newer build system pyrpoject.toml, rather than a setup.py - via pyproject.toml, the upstream build then uses flit when preparing the sources for distribution. (Contextually, this mayu be for distributions with sdist and/or wheel archives, such that may then be published to intermediate repositories such as PyPI) - Using upstream conventions, the version in this updated build would be derived from Git changelog information. This changelog information is not available in the tar bundle downloaded from cheeseshop In the approached presented with this patch, pyproject-build and flit are used to prepare the sources before building with distutils. The approach of using pyproject-build to prepare the sources was referenced partly on some tooling in the Debian dh-python[1] project. The distutils part of the build is referenced on the existing known-good approaches used in FreeBSD ports. To work around the concern about the dynamic versioning with the upstream pyproject.toml, a new patch is introduced, along with a post-patch call to inject a version string to pyproject.toml. This version string is then available for distutils, without further modification on the upstream sources - e.g if there was a setup.cfg ensuring setuptools_scm would be used directly. There isn't in this project, but it seems it was used anyway. In this approach - mainly, in using pyproject-build and flit to prepare the sources before building with distutils - the package will be installed with all source files, as filtered through flit, in addition to pyc files. This would be for a given set of extensions and other configuration selected onto pyproject.toml for the respective upstream build components (here, using flit, with no extensions in this project's pyproject.toml). Maybe other projects with a pyproject.toml and no setup.py could use a similar approach, however possibly further refined. For this port, this approach has the effect that the sphinx themes and other non-py sphinx resources will be installed along with the pkg. Thus, the basic sphinx theme and other sphinx themes will then be available with the updated port. The 'basic' theme itself may be required for the docs part of some other py-port builds - at least, after an update in each. This may include any ports using the textproc/py-sphinx-basic-ng them and thus transitively, textproc/py-sphinx_rtd_theme [1] https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/dhpython/build/plugin_pyproject.py#L105
Created a p[atch for devel/py-virtualenv to ensure the docs will build with the updated py-sphinx https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=268016 Both of these patches have been tested with poudriere, albeit in a system undergoing some other py-port updates.
Please refer to the referenced bug and associated literature for PEP-517 stuff.
Ah, there's a Wiki page https://wiki.freebsd.org/Python/PEP-517 I'm quite new to the Python programming language lol. I don't believe every project uses flit in itself, from pyproject.toml? Some projects might prefer hatchling e.g, or using the conventional setuptools. For an more recent devel/py-jupyterlab (3.5.0) released more recently than what's in ports right now, there's also the following for instance: ~~~~ [build-system] requires=["jupyter_packaging>=0.9,<2"] build-backend = "jupyter_packaging.build_api" ~~~~ I believe I'd hoped it might be a sufficient hack, to just call setuptools instead. If there's a broader project for addressing PEP-517 though, I think that's good to see. Of course with textproc/py-sphinx if I was only using setuptools and only installing everything it compiled, then that may not be sufficient to ensure that the sphinx themes would be installed with the py-sphinx port - thus to the effect of present patch, to ensure that at least the basic sphinx theme is installed and available for direct application with sphinx or for reference and extension in other sphinx themes. Returning to the PEP-517 topic, after the recent devel/py-jupyterlab release, then after one local test build it seemed that building with jlpm in particular may result in drawing in a lot of NPM packages, with quite a large download in the build process. I believe that's what the distributors may be using in order to produce what we're seeing in the downloads from cheeseshop? Thus, perhaps what we've download from the cheeeshop master site may already have been partially processed - contrasted to building directly from GitHub sources, for instance With sphinx at least, I'd thought it might be a good precaution to filter it again with pyproject-build nonetheless, if not simply installing all the sources files from the distfile archive. My main goal with the patch was simply to ensure that the basic theme is installed, along with the rest of the sphinx components that show up after the filtering with the upstream project's preferred build tool. I think it's great if the more general concern about PEP-517 support is being addressed though
Created attachment 238385 [details] patch proposal for updating to Sphinx 5.3.0 (updated) This patch obsoletes the previous here, also updating the version runtime dependencies from sphinx pyproject.toml Before updating to sphinx 5.3.0, the textproc/py-pygments and textproc/py-docutils ports may need to be updated, to match the updated version requirements in Sphinx pyproject.toml
Comment on attachment 238385 [details] patch proposal for updating to Sphinx 5.3.0 (updated) With this patch in its present state, the updated port would not install any bin scripts, thus not installing any sphinx-build script. This would represent a complete installation and it may break at least the devel/cmake-man build. However the consensus might go for PEP-517 support in ports, here was one proposal lol, after a glance at how Debian is approaching it such as in the dh-python tooling https://salsa.debian.org/python-team/tools/dh-python/
(In reply to Sean Champ from comment #5) would *not* represent a complete installation. Health, all
Created attachment 238397 [details] py-sphinx 5.3.0 patch (updated) It seems that it's possible to install directly from the sdist or wheel created by pyproject-build The wheel archive would include all dependencies. The sdist archive should include only the module code itself. This patch obsoletes the previous, now using pyproject-build to build an sdist archive during do-build, then installing from the sdist during do-install, there using pip. With the PEP 517 support now available in pip - using Python 3.11 at least - this has the effect of installing the module and its bin scripts. The plist is generated dynamically, from all files installed by pip under the cmd signature used in do-install So, this now installs the module code, the themes, and the bin scripts for this updated sphinx version Updated the run dependencies too
Created attachment 238398 [details] py-sphinx 5.3.0 patch (updated 2) This patch obsoletes the previous, here setting FLIT_NO_NETWORK in MAKE_ENV instead of CONFIGURE_ENV. This corresponds to the approach of using flit during do-build, instead of as in the first patch, where flit was called during do-configure Albeit, even with the variable absent from MAKE_ENV it seems that flit had not accessed the network when building under poudriere. The usage of FLIT_NO_NETWORK was referenced onto the flit documentation[1] [1] https://flit.pypa.io/en/stable/cmdline.html#envvar-FLIT_NO_NETWORK
(In reply to Sean Champ from comment #7) In a correction on my earlier comment, the wheel archive might not include all dependencies. Some calls to 'pip install' would install all dependencies, whether or not present in the wheel file itself.
Calling pip or including it as a build dependency is not acceptable for our packaging purposes. pip itself is a circular dependency nightmare, and we have to track exactly, in pkg(8), what is getting installed. With regards to PEP-517, please use/test the framework being worked on in the referenced bug, wiki page, Differential review, etc. You're needlessly duplicating what all the literature describes.
(In reply to Charlie Li from comment #10) Noticing the dependencies introduced in pip, theoretically those deps could be isolated in separate ports - e.g. a py-pip-docs port for providing documentation presently produced with the py-pip docs option. A py-pip-testing port could be created for testing pip, external to the main pip build. The dependencies of the port that would be used for building the main pip code would then be isolated to what's introduced from USES_PYTHON. If there may a branch on the ports tree or some other location where the pep517 option for USE_PYTHON is being tested, I'll take a look. Presently, I'm not certain how the last part of the installation could be approached without pip, after the source-handling under PEP 517 [1] https://wiki.freebsd.org/Python/PEP-517
pip isn't divisible in the way our ports framework needs, particularly with self-hosting. Community momentum from other operating system packagers and others resulted in alternative implementations, blessed by PyPA, more appropriate to situations like ours. Apply review D36290 to your tree locally to try the framework out.
Created attachment 247782 [details] Update to 7.2.6 Hello, My attempt at updating py-sphinx. Update to 7.2.6: - ${DISTVERSION} over ${PORTVERSION}` - Drop ${DISTNAME} - Some port{clippy,fmt,lint} maintenance - Use ${TAR} over hard coded binary name - Build HTML and manual pages - Compress shared build and runtime dependencies into ${_SPHINX_DEPENDS} - Add option for providing APIs to Web Apps https://www.sphinx-doc.org/en/master/changes.html#release-7-2-6-released-sep-13-2023
I have only tested with Python-3.11 poudriere-testport 13_2-{amd64,i386}-release : OK 14_0-{amd64,i386}-release : OK main-{amd64,i386} : OK Regarding the documentation, many runtime dependencies were duplicated across to build time, so as to build the Sphinx HTML and man pages. The shared dependencies have been stored in ${_SPHINX_DEPENDS}, however, I left WEBSUPPORT as an optional, but added it to ${BUILD_DEPENDS} for more complete documentation. Required updates: textproc/py-alabaster bug #276462 textproc/py-sphinxcontrib-serializinghtml bug #276449
Created attachment 247886 [details] Update to 7.2.6 Drop DOCS, it potentially causes cyclic dependency problems.