Off for riscv64 and powerpc64. configure.ac checks $CC, so we need to set it to clang if $CC == cc. LD takes around 300MB for linking so it should be ok to enable it even for 32 bit architectures. This patch should also be merged to older Python. At least 3.10 and 3.9 should build with LTO. It would be nice if it's done before 3.9 is made the default.
Created attachment 231841 [details] patch
I have a WIP for python ports to LTO, PGO, et al (PIE, etc), and more broadly prefer to handle this at the framework level than at an individual ports/software build level. Most build systems that provide pie/lto/etc features are relatively flaky and produce inconsistent results
@Piotr Let's touch base and solve this by making some progress at the higher level, I've made significant progress for establishing a baseline configuration for large swathes of the ports tree for LTO
I saw your site at https://wiki.freebsd.org/KubilayKocak/Ports/LinkTimeOptimization. However, until the toolchain issues are resolved, I'd prefer to enable LTO only for individual ports.
(In reply to Piotr Kubaj from comment #4) Sounds reasonable. Could we: - Wrap this in an LTO option, enabled by default, excluded on architectures its not relevent or ok for. - Update the patch to include changes for all lang/python* ports that support it (LTO). I think more than one does. - Can we get away without the 'CC= environment override', leaving in either: 1) the current check for == cc, or 2) using appropriate ports framework mechanisms to determine whether the compiler is clang? ie: if excluding options based on arch is feasible: if option and correct-compiler If not: if option and correct compiler and not bad-arch Note that while pythons configure checks CC, it also incorrectly detects 'GNU linker' (based on version string), and the build system is not very robust with respect to toolchains, and if we can just do the 'is this the right toolchain for lto' parts in ports, that would be preferable.
Created attachment 232057 [details] v2 Here's the patch for all Python 3 versions. Only Python 3.11 supports Thin LTO. Python 2.7 supports LTO but using it requires --enable-optimizations. Once this patch is committed, I plan to do the next one for --enable-optimizations and then will enable LTO for 2.7.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=9a31e1b6d3bff1eef9fcd5e72268c238ec5335fb commit 9a31e1b6d3bff1eef9fcd5e72268c238ec5335fb Author: Piotr Kubaj <pkubaj@FreeBSD.org> AuthorDate: 2022-03-09 17:30:00 +0000 Commit: Piotr Kubaj <pkubaj@FreeBSD.org> CommitDate: 2022-03-09 17:30:00 +0000 lang/python3*: add LTO option and enable by default everywhere except powerpc64 and riscv64 PR: 261974 Approved by: python (koobs got his commit bit revoked during review process) lang/python310/Makefile | 9 +++++++-- lang/python310/files/patch-configure (new) | 11 +++++++++++ lang/python311/Makefile | 9 +++++++-- lang/python311/files/patch-configure (new) | 11 +++++++++++ lang/python37/Makefile | 10 +++++++--- lang/python37/files/patch-configure (new) | 11 +++++++++++ lang/python38/Makefile | 10 +++++++--- lang/python38/files/patch-configure (new) | 11 +++++++++++ lang/python39/Makefile | 9 +++++++-- lang/python39/files/patch-configure (new) | 11 +++++++++++ 10 files changed, 90 insertions(+), 12 deletions(-)
(In reply to commit-hook from comment #7) Thanks for that Piotr, next time let's ensure a review and explicit accept prior to commit. Don't hesitate to reach out if you need support on leveling up Python ports ^Triage: Assign to committer that resolved
(In reply to Kubilay Kocak from comment #8) OK, I assumed that if you revoked your commit bit, you abandoned FreeBSD development completely.
(In reply to Piotr Kubaj from comment #9) That's not how that's supposed to be interpreted, no.