Bug 251183
| Summary: | archivers/liblz4: Update to 1.9.3, use meson and enable tests | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Ports & Packages | Reporter: | Daniel Engberg <diizzy> | ||||||
| Component: | Individual Port(s) | Assignee: | Alexey Dokuchaev <danfe> | ||||||
| Status: | In Progress --- | ||||||||
| Severity: | Affects Only Me | CC: | danfe, fernape | ||||||
| Priority: | --- | Keywords: | buildisok | ||||||
| Version: | Latest | Flags: | danfe:
maintainer-feedback?
|
||||||
| Hardware: | Any | ||||||||
| OS: | Any | ||||||||
| URL: | https://github.com/lz4/lz4/releases/tag/v1.9.3 | ||||||||
| Attachments: |
|
||||||||
Build and package info is available at https://gitlab.com/swills/freebsd-ports/pipelines/216552224 Thanks for the patch Daniel, could we make the following changes so all the variables are in their sections? # USES block USES +USE_GITHUB USE_LDCONFIG -USE_GITHUB # USES=shebangfix related variables SHEBANG_FILES # USES=meson related variables +MESON_ARGS # WRKSRC block WRKSRC_SUBDIR -MESON_ARGS Thanks! Created attachment 219745 [details]
Patch for liblz4 v2
Make portclippy happy
A commit references this bug: Author: fernape Date: Tue Nov 24 09:26:02 UTC 2020 New revision: 556153 URL: https://svnweb.freebsd.org/changeset/ports/556153 Log: archivers/liblz4: Update to 1.9.3 ChangeLog: https://github.com/lz4/lz4/releases/tag/v1.9.3 * Switch from gmake to meson. * Enable tests PR: 251183 Submitted by: daniel.engberg.lists@pyret.net Approved by: martin@waschbuesch.de (maintainer) Changes: head/archivers/liblz4/Makefile head/archivers/liblz4/distinfo head/archivers/liblz4/pkg-plist Committed, Thanks! I really want to switch back to gmake: this port sits at the top of dependency chain (required by zstd) and meson is such a heavy dep that the build often results in cyclic dependencies. Tests can be enabled simply with "TEST_TARGET=check". GNU make is both sufficient here and *much* lighter. (In reply to Alexey Dokuchaev from comment #6) Personally, I am open to that. Would a switch back create any issue(s) I might not be aware of right now? Does anyone else take issue with switching back? (In reply to Alexey Dokuchaev from comment #6) What dependency loops are you referring to, there are no reported in bugzilla? While I agree that initial population of dependencies is slower it's also a claim that fits a specific use case. While I haven't done an exact count of all consumers but having a quick look I'd estimate that around 80%+ will pull in Python in one way or another. I agree, that doesn't justify usage of Meson/Python/Ninja however it does show that you'll almost certainly will pull in these dependencies anyway so that argument is kinda moot. I made some quick tests to better illustrate why is also would be going backward rather than forward. Looking at some quick numbers from my 4 core ARM box using latest tree and no ports installed: Build and install all dependencies for lz4 in its current form (meson): 4m45.79s real 6m50.23s user 1m36.48s sys gmake and pkgconfig 1m16.68s real 2m16.05s user 29.34s sys Build lz4 in its current form (meson): 5.29s real 7.57s user 1.53s sys gmake 12.61s real 24.54s user 2.30s sys While looking at the build time by itself it's not that much of a difference but gmake is about twice as slow. That it doesn't "cover" the difference between building and installing all dependencies If we look at zstd for example where there are proposals to move from gmake there is quite a bit of a difference: meson: ====> Compressing man pages (compress-man) 13.55s real 36.76s user 4.11s sys cmake: ====> Compressing man pages (compress-man) 22.78s real 1m13.19s user 5.99s sys gmake: ====> Compressing man pages (compress-man) 50.53s real 1m52.57s user 7.65s sys A general trend are that projects are migrating from GNU Autotools (and indirectly gmake) / Makefiles to either CMake and Meson. That doesn't necessarily mean they're better by default but what you usually there's is a significant gain in overall build times and maintence. My point here is that unless you solely rely zstd even if just a few ports migrate to meson you're likely to see an overall improvement in build times and there's more to win by embracing it rather than working against it. (In reply to Daniel Engberg from comment #8) > What dependency loops are you referring to, there are no reported in > bugzilla? It was against my patched tree (to avoid various build issues with qemu-ppc-static, e.g. because recent versions of Python hang during the build under it). Normally this doesn't occur. I've already moved past and didn't keep the logs. > I agree, that doesn't justify usage of Meson/Python/Ninja Good. So be it then! > however it does show that you'll almost certainly will pull in these > dependencies anyway so that argument is kinda moot. There are plenty of things to build that do not require those, and having them being needlessly pulled is a real PITA. I can manage my way around rough corners so long as dependency graph doesn't blow up early. In general, if a *library* can be built with make+shell, it should be built this way. > While looking at the build time by itself it's not that much of a > difference but gmake is about twice as slow. Extra seven (or even seventy) seconds of *machine* time don't bother me as much as hours of *developer's* (e.g. my own) time wasted trying to solve problems Meson/Python/Ninja stack often entails, esp. when they should not arise in the first place, because liblz4's default build mode is with make(1). > A general trend are that projects are migrating from GNU Autotools (and > indirectly gmake) / Makefiles to either CMake and Meson. This is very unfortunate; we must fight this stupidity whenever we can (and when it's feasible). We certainly should not enforce those heavy, high-profile build systems when saner, leaner, more mundane alternatives are sitting right next to it (as in this case). > My point here is that unless you solely rely zstd even if just a few ports > migrate to meson you're likely to see an overall improvement in build times > and there's more to win by embracing it rather than working against it. Whatever build time win you might see is totally outweighed by much larger dependency graph which becomes a lot fragile if even just *one* port in it fails to build for some reason. When it happens, your build times become infinite because software no longer builds. (In reply to Alexey Dokuchaev from comment #9) So it's an issue in your custom tree not the official one, that's valuable information since it explains why there are no bug reports (ie not an issue). There's nothing to fight, we're following upstream projects because there's more to gain especially in the long run. Faster overall build times especially on multi core systems, better integration with ports framework, less maintenance for porters (including upstream) and usually better support upstream when submitting patches. While it might affect some very specific cases the wins outweight the cons by far, pretty much any system you're going to run build (and likely run) is going to utilize Python (just look at the consumers of lz4 and zstd for example) anyway. (In reply to Daniel Engberg from comment #10) > So it's an issue in your custom tree not the official one It is the issue with our tree: a few patches on top of it do not make it less official. The issue is that the port which is just a handful of *.c files and sits near the top of dep tree needlessly pulls tons of shit to build itself. > it explains why there are no bug reports There are no bug reports because normally Python builds fine, e.g. on bare metal or in a VM. It doesn't always build with some of less popular, no well-tested "qemu-$foo-static" helpers though. > There's nothing to fight, we're following upstream projects Well, that's not quite true: even when upstream defaults to some traditional build system, but additionally offers e.g. CMake bits (which are often provided for easier integration with third-party code and/or IDE, not to build the software itself), you'd still rather use it for those dubious reasons of yours. :-/ But yeah, let's follow upstream (per lz4's INSTALL file) and switch to make(1) for sanity's sake (we might not even need GNU make, I'd have to check and readjust my patch) instead of some *contributed* meson bits. (In reply to Daniel Engberg from comment #10) Just for the record: I was not aware that the lz4 upstream is abandoning other build environments in favor of meson. If I am in error and that is the direction, a link specifying that would be helpful. Up until now, I was under the impression that lz4 upstream supports several options w/r to building and that we are making a (very much so!) *deliberate* choice which of those suits us best? Are there compelling reasons for one instead of the other? So far, the arguments seemed to be more of the 'personal preference' type. But neither shorter build times nor following trends seem to me to be very good grounds for selecting the build system going forward? If there are no compelling reasons and all the tests work well without meson, I would prefer switching back to the more light-weight approach. Additional dependencies need justification, not the other way around. (In reply to Martin Waschbüsch from comment #12) > If I am in error and that is the direction, a link specifying that would > be helpful. Official build way is specified in the INSTALL file in the top directory: make make install # this command may require root access LZ4's Makefile supports standard Makefile conventions [and is] compatible with parallel builds. Meson/snap/etc. build glue are provided as *contributed* (i.e. unofficial) alernatives under contrib/ subdirectory (cf. the Makefile in the topdir). > I would prefer switching back to the more light-weight approach. Thank you. I'll push the commit after some more testing. (In reply to Martin Waschbüsch from comment #12) lz4 will very likely ship all variants for a while however consumers of lz4 are in general moving away and looking at our tree practically will also require Python to build one way or another. Meson is in their CI so it's not a second class citizen or such, see for example https://github.com/lz4/lz4/actions/runs/5420015948/jobs/9853704163?pr=1247 A few notes, some tests requires Python (seems like it will be utilized more in next release) For whatever reason the meson binaries are smaller than using the Makefile, possibly because of the seemingly forced debugging in the Makefile. For with it's worth at least Fedora and OpenMandrive have also moved to Meson Reassign to committer who reopened the issue. |
Created attachment 219727 [details] Patch for liblz4 Update libz4 to 1.9.3 Switch from gmake to meson Enable tests FreeBSD 13.0-CURRENT r367711 amd64 (make, make check-plist, make test) Poudriere testport OK 12.2-RELEASE (amd64) Poudriere testport OK 11.4-RELEASE (amd64)