Bug 186829 - lang/ghc is not jobs-safe
lang/ghc is not jobs-safe
Status: Closed FIXED
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s)
Latest
Any Any
: Normal Affects Only Me
Assigned To: haskell
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-02-17 04:10 UTC by Nikolai Lifanov
Modified: 2014-03-30 22:44 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nikolai Lifanov 2014-02-17 04:10:00 UTC
	The GHC port is not jobs-safe.
	This should either be fixed (I'd imagine this is hard) or marked as make jobs unsafe.

CROSS_COMPILE="" "utils/ghc-cabal/dist-install/build/tmp/ghc-cabal" install "/tmp/lifanov/ghc/work/ghc-7.6.3-boot/lib/ghc-7.6.3/ghc" "/tmp/lifanov/ghc/work/ghc-7.6.3-boot/lib/ghc-7.6.3/ghc-pkg" "strip" "/tmp/lifanov/ghc/work/ghc-7.6.3-boot/lib/ghc-7.6.3" libraries/old-locale dist-install '' '/tmp/lifanov/ghc/work/ghc-7.6.3-boot' '/tmp/lifanov/ghc/work/ghc-7.6.3-boot/lib/ghc-7.6.3' '/tmp/lifanov/ghc/work/ghc-7.6.3-boot/share/doc/ghc/html/libraries' NO  
Installing library in
/tmp/lifanov/ghc/work/ghc-7.6.3-boot/lib/ghc-7.6.3/old-locale-1.0.0.5
Warning: cannot determine version of
/tmp/lifanov/ghc/work/ghc-7.6.3-boot/lib/ghc-7.6.3/ghc :
""
ghc-cabal: ghcInvocation: the programVersion must not be Nothing
gmake[3]: *** [install_packages] Error 1
gmake[2]: *** [install] Error 2
gmake[2]: Leaving directory `/tmp/lifanov/ghc/work/ghc-7.6.3-boot'
*** [pre-configure] Error code 2

make[1]: stopped in /tmp/lifanov/ghc
1 error

make[1]: stopped in /tmp/lifanov/ghc
*** [build] Error code 2

make: stopped in /tmp/lifanov/ghc
1 error

make: stopped in /tmp/lifanov/ghc

Fix: 

Mark lang/ghc port as jobs unsafe.
	In this case, make -j1 works.
How-To-Repeat: 	cd /usr/ports/lang/ghc
	make -j8
Comment 1 Edwin Groothuis freebsd_committer 2014-02-17 04:11:29 UTC
Responsible Changed
From-To: freebsd-ports-bugs->haskell

Over to maintainer (via the GNATS Auto Assign Tool)
Comment 2 Gabor Pali freebsd_committer 2014-03-22 19:13:02 UTC
Hi Nikolai,

Yes, you were right, the build of GHC is not really thread-safe, so
running this on multiple threads is going to fail at some point.  I
have been already notified about this by Baptiste (Daroussin) a few
months ago when he did the experimental runs for activating multiple
jobs by default.

That is when I suggested to introduce a new variable called
MAKE_JOBS_NUMBER_LIMIT to allow using multiple jobs but still in a
careful fashion by setting a limit to the parallelization [1].

However, I am not sure this would prevent somebody from passing `-jN`
directly to make(1).  And I am not either sure if this is recommended
at all.  Instead, you should use MAKE_JOBS_NUMBER=N, that is:

# make MAKE_JOBS_NUMBER=8

MAKE_JOBS_NUMBER is automatically set to the value of the hw.ncpu
sysctl(3) variable.  In that case, I could then add this
MAKE_JOBS_NUMBER_LIMIT variable to the Makefile of lang/ghc so the
port would allow parallel builds but only in a limited way.

Can you please test if adding this line below to the Makefile fixes
your problem?

MAKE_JOBS_NUMBER_LIMIT= 4

Thanks!

[1] http://svnweb.freebsd.org/changeset/ports/323100
Comment 3 Nikolai Lifanov 2014-03-22 21:11:34 UTC
On 2014-03-22 15:13, Gabor Pali wrote:
> Hi Nikolai,
> 
> Yes, you were right, the build of GHC is not really thread-safe, so
> running this on multiple threads is going to fail at some point.  I
> have been already notified about this by Baptiste (Daroussin) a few
> months ago when he did the experimental runs for activating multiple
> jobs by default.
> 
> That is when I suggested to introduce a new variable called
> MAKE_JOBS_NUMBER_LIMIT to allow using multiple jobs but still in a
> careful fashion by setting a limit to the parallelization [1].
> 
> However, I am not sure this would prevent somebody from passing `-jN`
> directly to make(1).  And I am not either sure if this is recommended
> at all.  Instead, you should use MAKE_JOBS_NUMBER=N, that is:
> 
> # make MAKE_JOBS_NUMBER=8
> 
> MAKE_JOBS_NUMBER is automatically set to the value of the hw.ncpu
> sysctl(3) variable.  In that case, I could then add this
> MAKE_JOBS_NUMBER_LIMIT variable to the Makefile of lang/ghc so the
> port would allow parallel builds but only in a limited way.
> 
> Can you please test if adding this line below to the Makefile fixes
> your problem?
> 
> MAKE_JOBS_NUMBER_LIMIT= 4
> 
> Thanks!
> 
> [1] http://svnweb.freebsd.org/changeset/ports/323100

I verified that setting MAKE_JOBS_NUMBER_LIMIT= 4 fixes the build for 
me. This is 4x better than marking it as jobs unsafe!

- Nikolai Lifanov
Comment 4 dfilter freebsd_committer 2014-03-23 08:55:19 UTC
Author: pgj
Date: Sun Mar 23 08:55:15 2014
New Revision: 348842
URL: http://svnweb.freebsd.org/changeset/ports/348842
QAT: https://qat.redports.org/buildarchive/r348842/

Log:
  - Limit built parallelism as GHC is not fully jobs-safe
  
  PR:		ports/186829
  Reported by:	Nikolai Lifanov <lifanov@mail.lifanov.com>
  Obtained from:	FreeBSD Haskell

Modified:
  head/lang/ghc/Makefile

Modified: head/lang/ghc/Makefile
==============================================================================
--- head/lang/ghc/Makefile	Sun Mar 23 08:09:29 2014	(r348841)
+++ head/lang/ghc/Makefile	Sun Mar 23 08:55:15 2014	(r348842)
@@ -15,6 +15,8 @@ COMMENT=	Compiler for the functional lan
 
 ONLY_FOR_ARCHS=	i386 amd64
 
+MAKE_JOBS_NUMBER_LIMIT=		4
+
 GHC_VERSION=		${PORTVERSION}
 HSCOLOUR_VERSION=	1.20.3
 
_______________________________________________
svn-ports-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-ports-all
To unsubscribe, send any mail to "svn-ports-all-unsubscribe@freebsd.org"
Comment 5 Gabor Pali freebsd_committer 2014-03-24 15:56:58 UTC
State Changed
From-To: open->patched

Fixed in head.
Comment 6 Nikolai Lifanov 2014-03-25 22:53:05 UTC
I, the submitter, verified that the commited patch fixes the build for 
me for all my systems (9.2-RELEASE, 10.0-RELEASE, 11.0-CURRENT, etc.).

This PR can be closed.

- Nikolai Lifanov
Comment 7 Gabor Pali freebsd_committer 2014-03-26 17:30:46 UTC
2014-03-26 0:00 GMT+01:00 Nikolai Lifanov <lifanov@mail.lifanov.com>:
>  This PR can be closed.

Sorry that I have not yet closed the PR -- I just set the state the
"patched", because I would like to merge this change to the 2014Q1
quarterly (stable) branch, if portmgr approves it.
Comment 8 Gabor Pali freebsd_committer 2014-03-30 22:43:33 UTC
State Changed
From-To: patched->closed

The 2014Q1 is now closed, no merge is needed.  So the PR can be closed.