Bug 251755 - shells/bash: appears to be broken with -jX
Summary: shells/bash: appears to be broken with -jX
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Emanuel Haupt
URL: https://lists.gnu.org/archive/html/bu...
Keywords:
Depends on:
Blocks:
 
Reported: 2020-12-11 10:34 UTC by Mikael Urankar
Modified: 2020-12-12 13:02 UTC (History)
3 users (show)

See Also:
bugzilla: maintainer-feedback? (ehaupt)


Attachments
build log (77.95 KB, text/plain)
2020-12-11 10:34 UTC, Mikael Urankar
no flags Details
can't invoke error with FORCE_MAKE_JOBS=YES/MAKE_JOBS_NUMBER=25 (209.23 KB, text/plain)
2020-12-11 13:45 UTC, Emanuel Haupt
no flags Details
parallel make issue patch obtained from gentoo (943 bytes, patch)
2020-12-12 06:22 UTC, Emanuel Haupt
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mikael Urankar freebsd_committer freebsd_triage 2020-12-11 10:34:30 UTC
Created attachment 220456 [details]
build log

error:

cc -DPROGRAM='"bash"' -DCONF_HOSTTYPE='"amd64"' -DCONF_OSTYPE='"freebsd13.0"' -DCONF_MACHTYPE='"amd64-portbld-freebsd13.0"' -DCONF_VENDOR='"portbld"' -DLOCALEDIR='"/usr/local/share/locale"' -DPACKAGE='"bash"' -
DSHELL  -DHAVE_CONFIG_H -DHEREDOC_PIPESIZE=4096   -I.  -I. -I./include -I./lib  -DDEFAULT_PATH_VALUE='"/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin"'  -DSTANDARD_UTILS_PATH='"/sbin:/bin:/usr/sbi
n:/usr/bin"' -DLIBICONV_PLUG -I/usr/local/include -isystem /usr/local/include -O2 -pipe  -DLIBICONV_PLUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -Wl,-export-dynamic -L/usr/loc
al/lib -fstack-protector-strong   -O2 -pipe  -DLIBICONV_PLUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing    -O2 -pipe  -DLIBICONV_PLUG -fstack-protector-strong -isystem /usr/local/
include -fno-strict-aliasing -o bashversion ./support/bashversion.c buildversion.o 
--- bashline.o ---
bashline.c:65:10: fatal error: 'builtins/builtext.h' file not found
#include "builtins/builtext.h"          /* for read_builtin */
         ^~~~~~~~~~~~~~~~~~~~~
Comment 1 Emanuel Haupt freebsd_committer freebsd_triage 2020-12-11 11:47:47 UTC
I've had similar reports, always failing at different stages. Can you try setting 

MAKE_JOBS_UNSAFE=yes

in the Makefile?
Comment 2 freebsd 2020-12-11 12:21:27 UTC
I can confirm that on all Poudriere jails (32/64bit FreeBSD12) adding "MAKE_JOBS_UNSAFE=yes" to the jails' make.conf makes bash compile. Are you interested in 'before and after' output of a 32 bit and a 64 bit jail?

Having "MAKE_JOBS_UNSAFE=yes" on is not preferable, so if that helps you pinpoint the trigger, that's fine with me.
Comment 3 Emanuel Haupt freebsd_committer freebsd_triage 2020-12-11 12:39:07 UTC
Not having to set MAKE_JOBS_UNSAFE=yes in the port is the preferred solution. Any help finding the broken part is gratefully appreciated.

We might have to set it to MAKE_JOBS_UNSAFE=yes as an interim solution.
Comment 4 freebsd 2020-12-11 13:11:31 UTC
FreeBSD 12 amd64 - failed bash build: https://pastebin.com/DXgjFHRi
FreeBSD 12 amd64 - succeeded bash build: https://pastebin.com/UEpUnDBe

FreeBSD 12 i386 - failed bash build: https://pastebin.com/Kkzih5wT
FreeBSD 12 i386 - succeeded bash build: https://pastebin.com/tBcVHFDz

HTH!
Comment 5 Emanuel Haupt freebsd_committer freebsd_triage 2020-12-11 13:20:29 UTC
Unfortunately the build log does not show exactly where the problem in the make code is. This can be very hard and tedious to find.

See also 6.5.1 in https://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/book.html

I'll look into it as time permits.
Comment 6 freebsd 2020-12-11 13:28:26 UTC
It seems to center around the mkbuiltins step but I realise that that's hardly helpful.
Comment 7 Jakob Breivik Grimstveit 2020-12-11 13:34:16 UTC
Seems to fail at the same location every time for me:

$ uname -a
FreeBSD core24.grimstveit.no 12.2-RELEASE-p2 FreeBSD 12.2-RELEASE-p2 30ef47ef8fb(releng/12.2) CORE24  amd64

$ grep MAKE /etc/make.conf
FORCE_MAKE_JOBS=YES
MAKE_JOBS_NUMBER=25


Ending output of `make stage` below


--- make_cmd.o ---
1 warning generated.
--- bashhist.o ---
bashhist.c:726:15: warning: using the result of an assignment as a condition without parentheses [-Wparentheses]
  while (temp = previous_history ())
         ~~~~~^~~~~~~~~~~~~~~~~~~~~
bashhist.c:726:15: note: place parentheses around the assignment to silence this warning
  while (temp = previous_history ())
              ^
         (                         )
bashhist.c:726:15: note: use '==' to turn this assignment into an equality comparison
  while (temp = previous_history ())
              ^
              ==
--- bashline.o ---
1 error generated.
*** [bashline.o] Error code 1

make[2]: stopped in /usr/ports/shells/bash/work/bash-5.1
--- pathexp.o ---
2 warnings generated.
--- general.o ---
1 warning generated.
--- y.tab.c ---
touch parser-built
--- ./builtins/builtext.h ---
2 warnings generated.
*** [./builtins/builtext.h] Error code 1

make[2]: stopped in /usr/ports/shells/bash/work/bash-5.1
--- bashhist.o ---
1 warning generated.
--- shell.o ---
1 warning generated.
--- braces.o ---
2 warnings generated.
--- ./support/man2html ---
1 warning generated.
*** [./support/man2html] Error code 1

make[2]: stopped in /usr/ports/shells/bash/work/bash-5.1
--- ./builtins/pipesize.h ---
*** [./builtins/pipesize.h] Error code 1

make[2]: stopped in /usr/ports/shells/bash/work/bash-5.1
4 errors

make[2]: stopped in /usr/ports/shells/bash/work/bash-5.1
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/shells/bash
*** Error code 1

Stop.
make: stopped in /usr/ports/shells/bash
Comment 8 Emanuel Haupt freebsd_committer freebsd_triage 2020-12-11 13:38:22 UTC
I'm still failing to reproduce it on my lowly 8 core build system :-) So far I've had no luck with setting.

MAKEFLAGS=      -j50

It's clearly a parallel build issue as it's been reported by at least 2 other people via private mail.
Comment 9 Emanuel Haupt freebsd_committer freebsd_triage 2020-12-11 13:45:06 UTC
Created attachment 220458 [details]
can't invoke error with FORCE_MAKE_JOBS=YES/MAKE_JOBS_NUMBER=25

...same with:

FORCE_MAKE_JOBS=YES
MAKE_JOBS_NUMBER=25
Comment 10 rcarter 2020-12-12 00:47:07 UTC
Gentoo thinks they have solved it here:

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4c2ebbf4b8bc660beb98cc2d845c73375d6e4f50
Comment 11 Emanuel Haupt freebsd_committer freebsd_triage 2020-12-12 06:22:26 UTC
Created attachment 220476 [details]
parallel make issue patch obtained from gentoo

(In reply to rcarter from comment #10)
Interesting. Can someone who can trigger the parallel build error confirm that the attached patch for the FreeBSD solves the issue? Poudriere test builds all pass the new patch but then again I'm still unable to trigger the error myself.
Comment 12 commit-hook freebsd_committer freebsd_triage 2020-12-12 09:59:10 UTC
A commit references this bug:

Author: ehaupt
Date: Sat Dec 12 09:58:19 UTC 2020
New revision: 557806
URL: https://svnweb.freebsd.org/changeset/ports/557806

Log:
  In some cases bash fails to build parallel make jobs. Provide a patch.

  [1] https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4c2ebbf4b8bc660beb98cc2d845c73375d6e4f50

  PR:		251755
  Reported by:	mikael, freebsd@bengrimm.net, jakob@grimstveit.no
  Submitted by:	rcarter@pinyon.org
  Obtained from:	gentoo [1]

Changes:
  head/shells/bash/files/patch-Makefile.in
Comment 13 freebsd 2020-12-12 10:28:46 UTC
Built without safe jobs on FreeBSD 12.2 amd64 successfully.

https://pastebin.com/PKyuAskH
Comment 14 Emanuel Haupt freebsd_committer freebsd_triage 2020-12-12 10:30:03 UTC
(In reply to freebsd from comment #13)
Thanks for confirming.
Comment 15 freebsd 2020-12-12 10:36:27 UTC
(In reply to freebsd from comment #13)

Try https://pastebin.com/raw/PKyuAskH if that link crashed your mobile chrome/brave consistently ;)
Comment 16 Emanuel Haupt freebsd_committer freebsd_triage 2020-12-12 11:32:18 UTC
A patch has been committed, thanks!