I am unable to build lapack 3.10.0 on my system. FreeBSD xxxx.org 11.4-STABLE FreeBSD 11.4-STABLE #0 stable/11-n215696-62607e8680e: Tue Feb 16 19:28:06 MST 2021 xxxx@xxxx.org:/usr/obj/usr/src/sys/GENERIC amd64 3.9.0 is already installed. Portupgrade fails with "unknown build error". If I attempt simply to so a "make" in /usr/ports/math/lapack, I see that it fails while attempting to generate the packing list with: ===> Staging for lapack-3.10.0 ===> lapack-3.10.0 depends on executable: gfortran10 - found ===> Generating temporary packing list make: exec(/bin/sh) failed (Argument list too long) *** Error code 1
Quiet strange… Packages have been built for FreeBSD-11 for i386 and amd64. Could you please provide a full log?
Created attachment 226303 [details] Full log of "make" of /usr/ports/math/lapack There are no details in here after it begins to create the staging area other than the failure, but here is the complete typescript of the make process as requested.
While the typescript of "make" showed no details of any value, I did a "make -dA" and it produced a full log file that is far too large to post here. However, examining that typescript I find that the error happens when it attempts to do a "for file in " with a complete list of man pages paths. I'm attaching a very tiny excerpt of the "make -dA" typescript, just the last few commands it spewed. It is clearly puking on the enormous number of man pages.
Created attachment 226306 [details] Last few lines of the output of "make -dA" in math/lapack. Here, it is clear that the failure is in an attempt to do a "for" loop over all the man pages to create a temporary packing list. There appear to be simply too many on the "for" line.
Just ran a "wc -w" on the list of man pages in that for line, and it's over 24,000.
Created attachment 226307 [details] testcase for manpages Have ever encountered such a problem with other "big" ports? I guess that this is caused by the list of all manpages: could please replace the file /usr/ports/math/lapack/files/manpages by the (shorter) attached testcase and try again?
(In reply to Thierry Thomas from comment #6) No, I have never had this happen on any other port, but I'm not sure any other port has ever tried to create a dynamic packing list of tens of thousands of files before. By replacing files/manpages with the severely truncated version you suggest, I find that the port no longer crashes in the packaging stage. So the problem is indeed that 12,178 line "files/manpages" that sets MAN3L.
Then the problem is not really caused by this port, but by some limit in your system. Any special configuration? I am going to search for similar cases. Meanwhile, I suggest to keep it with this truncated file if you do not need these manpages, or install the package.
FYI We've also faced this problem trying to build math/lapack with Synth on FreeBSD 11.4-RELEASE-p9 amd64. So I can confirm that this issue is not specific just to russo@ build environment and it affects other people as well.
(In reply to Thierry Thomas from comment #8) My system is a bog-standard source install of FreeBSD 11-STABLE, using the GENERIC kernel configuration. There should be no special configuration. It is "caused" by this port only in the sense that the port is trying to create a packing list from a huge list of files, in such a manner that exceeds the shell's limit on command line arguments. I don't know where the limit on command line arguments comes from, but there is no setting on my system that has specifically limited it to something small, or at least I have not changed anything to reduce the maximum in any way.
(In reply to russo from comment #10) Oddly, "getconf ARG_MAX" shows a much larger value than should be impacting this port's argument list: > getconf ARG_MAX 262144 I did a "wc -w" on the "for" loop in the "make -dA" output, and it was somewhere around 24,000, nowhere near this 262,144 limit. I do not really understand why it should be hitting any limit at all given this large value of ARG_MAX.
11.4-stable/amd64: % getconf ARG_MAX 262144 12.2-stable/amd64: % getconf ARG_MAX 524288 Some will say that 11.x is on the way out. I'm not a big fan of that line of thinking. But it's one answer. Eventually something could hit the limit in 12.x (or later) as well. Two problems are exposed by lots of files in a the lapack package. One is not specific to lapack. This is in generate-plist in bsd.port.mk when PLIST_FILES is long: for file in ${PLIST_FILES}; do \ The other is specific to the current version of lapack - at the bottom of post-install when the man3l files are installed: cd ${WRKDIR}/man ; ${ECHO_CMD} ${MAN3L:[1..6000]:S|^|man3/|} \ | ${XARGS} -J @ ${INSTALL_MAN} @ ${STAGEDIR}${MAN3PREFIX}/man/man3 Both of these trigger 'Argument list too long' issues.
(In reply to John Hein from comment #12) The src commit that bumped up ARG_MAX is base b003cf6c08d3f19063a083991582bc793b2e6e51 (_FreeSD_version 1201506, first release version that had it was 12.2): =========== commit b003cf6c08d3f19063a083991582bc793b2e6e51 Author: Pedro F. Giffuni <pfg@FreeBSD.org> Date: Sat Dec 21 02:40:40 2019 +0000 MFC r355828: Double the size of ARG_MAX on LP64 platforms. . . ===========
Created attachment 226382 [details] [patch] math/lapack: break up PLIST_FILES in generate-plist and MAN3L in post-install Here is a patch that allows lapack to build (actually 'stage') on 11.x/amd64 with the default ARG_MAX - or other platforms that might also have small enough ARG_MAX. It breaks up the PLIST_FILES / MAN3L list into smaller chunks. This includes a change to bsd.port.mk. It is not sufficiently general, so it is nearly a certainty that this would not be acceptable to b.p.m owners. But one can apply this locally to get lapack building on platforms that hit the 'Argument list too long' error. For the same reason, I did not request maintainer approval. There should be a better way.
Created attachment 226395 [details] [patch] math/lapack: stop using excessively long command lines This patch sticks to using files and pipelines rather than long command line argument lists. I also noticed that /usr/local/lib/libtmglib.so.3.10.0 is a broke sym link, but that is a separate bug.
(In reply to John Hein from comment #15) See bug 257139 for the broken lib symlink issue.
Committed (patch by John Hein), thanks for the report!
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=37e3e66b0de6644dabe631c8a68d58c7cca14f32 commit 37e3e66b0de6644dabe631c8a68d58c7cca14f32 Author: John Hein <jcfyecrayz@liamekaens.com> AuthorDate: 2021-07-17 15:09:08 +0000 Commit: Thierry Thomas <thierry@FreeBSD.org> CommitDate: 2021-07-17 15:18:58 +0000 math/lapack: fix "argument list too long" Older FreeBSD versions have a too small ARG_MAX, and making the plist failed with the "argument list too long" error. This patch sticks to using files and pipelines rather than long command line argument lists, authored by John Hein <jcfyecrayz@liamekaens.com>. PR: 257004 Reported by: russo (at) bogodyn.org math/lapack/Makefile | 13 +- math/lapack/files/manpages (gone) | 12178 ------------------------------------ 2 files changed, 7 insertions(+), 12184 deletions(-)
A commit in branch 2021Q3 references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=32dcf30347242e2efa7337bba01044e2c691d0d3 commit 32dcf30347242e2efa7337bba01044e2c691d0d3 Author: John Hein <jcfyecrayz@liamekaens.com> AuthorDate: 2021-07-17 15:09:08 +0000 Commit: Thierry Thomas <thierry@FreeBSD.org> CommitDate: 2021-07-18 10:32:35 +0000 math/lapack: fix "argument list too long" Older FreeBSD versions have a too small ARG_MAX, and making the plist failed with the "argument list too long" error. This patch sticks to using files and pipelines rather than long command line argument lists, authored by John Hein <jcfyecrayz@liamekaens.com>. PR: 257004 Reported by: russo (at) bogodyn.org (cherry picked from commit 37e3e66b0de6644dabe631c8a68d58c7cca14f32) math/lapack/Makefile | 13 +- math/lapack/files/manpages (gone) | 12178 ------------------------------------ 2 files changed, 7 insertions(+), 12184 deletions(-)