Bug 202566 - lang/ghc : new version fails with "permission denied" on DragonFly poudriere
Summary: lang/ghc : new version fails with "permission denied" on DragonFly poudriere
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-haskell (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-08-22 11:09 UTC by John Marino
Modified: 2015-09-10 08:28 UTC (History)
1 user (show)

See Also:
bugzilla: maintainer-feedback? (haskell)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description John Marino freebsd_committer freebsd_triage 2015-08-22 11:09:07 UTC
I am wondering if lang/ghc now builds in an illegal area.  When trying to build the new version of GHC (using a patched version of lang/ghc to bring in 7.6.3 bootstrap from dragonfly) I get this error:

[89 of 89] Compiling Main             ( utils/ghc-cabal/Main.hs, bootstrapping/Main.o )
Linking utils/ghc-cabal/dist/build/tmp/ghc-cabal ...
"touch" utils/ghc-cabal/dist/build/tmp/ghc-cabal
"cp" utils/ghc-cabal/dist/build/tmp/ghc-cabal inplace/bin/ghc-cabal
"inplace/bin/ghc-cabal" configure libraries/binary dist-boot "" --with-ghc="/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3" --with-ghc-pkg="/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-pkg-7.6.3"  --package-db=/wrkdirs/lang/ghc/work/ghc-7.10.2/libraries/bootstrapping.conf --disable-library-for-ghci --enable-library-vanilla --disable-library-profiling --disable-shared --with-hscolour="/wrkdirs/lang/ghc/work/slaves_prefix/bin/HsColour" --configure-option=CFLAGS="-pipe -I/usr/local/include -O2 -fno-strict-aliasing -fno-stack-protector   " --configure-option=LDFLAGS="  " --configure-option=CPPFLAGS="   " --gcc-options="-pipe -I/usr/local/include -O2 -fno-strict-aliasing -fno-stack-protector      " --configure-option=--with-iconv-includes="/usr/local/include" --configure-option=--with-iconv-libraries="/usr/local/lib" --configure-option=--with-gmp-includes="/usr/local/include" --configure-option=--with-gmp-libraries="/usr/local/lib"   --constraint "binary == 0.7.5.0"   --constraint "Cabal == 1.22.4.0"   --constraint "hpc == 0.6.0.2"   --constraint "bin-package-db == 0.0.0.0"   --constraint "hoopl == 3.10.0.2"   --constraint "transformers == 0.4.2.0"   --constraint "terminfo == 0.4.0.1" --with-gcc="gcc" --configure-option=--with-cc="gcc" --with-ar="/usr/bin/ar"  
"inplace/bin/ghc-cabal" configure utils/hsc2hs dist "" --with-ghc="/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3" --with-ghc-pkg="/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-pkg-7.6.3"  --package-db=/wrkdirs/lang/ghc/work/ghc-7.10.2/libraries/bootstrapping.conf --disable-library-for-ghci --disable-library-vanilla --disable-library-profiling --disable-shared --with-hscolour="/wrkdirs/lang/ghc/work/slaves_prefix/bin/HsColour" --configure-option=CFLAGS="-pipe -I/usr/local/include -O2 -fno-strict-aliasing -fno-stack-protector   " --configure-option=LDFLAGS="  " --configure-option=CPPFLAGS="   " --gcc-options="-pipe -I/usr/local/include -O2 -fno-strict-aliasing -fno-stack-protector      " --configure-option=--with-iconv-includes="/usr/local/include" --configure-option=--with-iconv-libraries="/usr/local/lib" --configure-option=--with-gmp-includes="/usr/local/include" --configure-option=--with-gmp-libraries="/usr/local/lib"   --constraint "binary == 0.7.5.0"   --constraint "Cabal == 1.22.4.0"   --constraint "hpc == 0.6.0.2"   --constraint "bin-package-db == 0.0.0.0"   --constraint "hoopl == 3.10.0.2"   --constraint "transformers == 0.4.2.0"   --constraint "terminfo == 0.4.0.1" --with-gcc="gcc" --configure-option=--with-cc="gcc" --with-ar="/usr/bin/ar"  
Configuring hsc2hs-0.67...
Configuring binary-0.7.5.0...
ghc-cabal: /usr/bin/ar: permission denied
utils/hsc2hs/ghc.mk:15: recipe for target 'utils/hsc2hs/dist/package-data.mk' failed


My guess is /usr/bin/ar is trying to assemble something in a read-only mount in poudriere, which should never happen.  (This is also a violation on FreeBSD but poudriere there doesn't break when it happens)


Can you guys confirm this is indeed what is happening and if so, where is it building and how can we make it build in $WRKDIR instead?

full log (won't be there forever):
http://muscles.dragonflybsd.org/bulk/bleeding-edge-potential/20150822_035817/logs/errors/ghc-7.10.2.log
Comment 1 Gabor Pali freebsd_committer freebsd_triage 2015-08-22 16:54:30 UTC
I believe everything works in the same way as before, the build is still done in $WRKDIR and $WRKSRC.  Although, there may be problems when you are trying to build GHC on too many threads.  For some reasons, the build does not really work well with multiple threads, that is why we have the limit in the Makefile -- but that was based on vague empirical findings, so it may not apply properly to DragonFly.
Comment 2 John Marino freebsd_committer freebsd_triage 2015-08-22 17:03:51 UTC
you might be right, it doesn't seem to be deterministic.  Next run (which seem to get one step past linking):

--with-gcc="gcc" --configure-option=--with-cc="gcc" --with-ar="/usr/bin/ar"  
Configuring hsc2hs-0.67...
Configuring binary-0.7.5.0...
  GHC PKG libraries/binary/dist-boot/package-data.mk
Reading package info from "libraries/binary/dist-boot/inplace-pkg-config" ... done.
binary-0.7.5.0: Warning: Unrecognized field data-dir on line 31
binary-0.7.5.0: Warning: Unrecognized field key on line 4
binary-0.7.5.0: Warning: haddock-interfaces: /wrkdirs/lang/ghc/work/ghc-7.10.2/libraries/binary/dist-boot/doc/html/binary/binary.haddock doesn't exist or isn't a file
binary-0.7.5.0: file Data/Binary.hi is missing (ignoring)
binary-0.7.5.0: file Data/Binary/Put.hi is missing (ignoring)
binary-0.7.5.0: file Data/Binary/Get.hi is missing (ignoring)
binary-0.7.5.0: file Data/Binary/Get/Internal.hi is missing (ignoring)
binary-0.7.5.0: file Data/Binary/Builder.hi is missing (ignoring)
binary-0.7.5.0: file Data/Binary/Builder/Internal.hi is missing (ignoring)
binary-0.7.5.0: file Data/Binary/Builder/Base.hi is missing (ignoring)
binary-0.7.5.0: file Data/Binary/Class.hi is missing (ignoring)
binary-0.7.5.0: file Data/Binary/Generic.hi is missing (ignoring)
Comment 3 John Marino freebsd_committer freebsd_triage 2015-08-22 17:09:51 UTC
i limited jobs to 2, and now I'm back to before.

Configuring hsc2hs-0.67...
Configuring binary-0.7.5.0...
ghc-cabal: /usr/bin/ar: permission denied
utils/hsc2hs/ghc.mk:15: recipe for target 'utils/hsc2hs/dist/package-data.mk' failed


Also, the error before was using "poudriere testport" and the two others were just "poudriere bulk".  That might be important.
Comment 4 John Marino freebsd_committer freebsd_triage 2015-08-22 17:15:09 UTC
yep.  Using testport I get one set of errors and using bulk I get another.

Have you ever built ghc with testport, by chance?
Comment 5 John Marino freebsd_committer freebsd_triage 2015-08-22 17:33:52 UTC
fyi, even with 1 job limit I get the same result:


Configuring binary-0.7.5.0...
ghc-cabal: /usr/bin/ar: permission denied
libraries/binary/ghc.mk:3: recipe for target 'libraries/binary/dist-boot/package-data.mk' failed
gmake[2]: *** [libraries/binary/dist-boot/package-data.mk] Error 1
Makefile:71: recipe for target 'all' failed
gmake[1]: *** [all] Error 2
gmake[1]: Leaving directory '/wrkdirs/lang/ghc/work/ghc-7.10.2'
Comment 6 Gabor Pali freebsd_committer freebsd_triage 2015-08-22 17:42:47 UTC
(In reply to John Marino from comment #4):

No, I have not used "tesport" for testing lang/ghc.  What difference does it make compared to "bulk"?
Comment 7 John Marino freebsd_committer freebsd_triage 2015-08-22 17:45:20 UTC
The differences are subtle, but I would need to dig into the code to tell you.

Right now with jobs limited to 1, ghc seems to be building in testport (although it failed in bulk).

This is all very strange.

I suspect testport may have jobs disabled by default.

A good way for me to test this is to set "MAKE_JOBS_UNSAFE=yes" and try again with poudriere bulk.
Comment 8 John Marino freebsd_committer freebsd_triage 2015-08-22 20:34:01 UTC
The interesting news:

The testport (presumably jobs-disabled) completed.


The bad news:

It failed plist-check:

===========================================================================
====> Running Q/A tests (stage-qa)
====> Checking for pkg-plist issues (check-plist)
===> Parsing plist
===> Checking for items in STAGEDIR missing from pkg-plist
Error: Orphaned: bin/haddock
Error: Orphaned: %%GHC_LIBDIR%%/package.conf.d/package.cache
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-124.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-33.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-36.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-43.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-45.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-47.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-58.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-60.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-62.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-92.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-A.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-All.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-B.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-C.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-D.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-E.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-F.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-G.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-H.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-I.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-J.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-K.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-L.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-M.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-N.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-O.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-P.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-Q.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-R.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-S.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-T.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-U.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-V.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-W.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-X.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-Y.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/doc-index-Z.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/hslogo-16.png
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/index-frames.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/index.html
Error: Orphaned: %%PORTDOCS%%%%DOCSDIR%%/html/libraries/synopsis.png
===> Checking for items in pkg-plist which are not in STAGEDIR
===> Error: Plist issues found.
*** Error code 1




Did you verify plist-is correct for 7.10 ?
Comment 9 John Marino freebsd_committer freebsd_triage 2015-08-22 20:40:02 UTC
based on this output, I'm going to modify the port (in dports) as follows:

1) set PORTDOCS= html
2) set MAKE_JOBS_UNSAFE=yes
3) on post-install, remove %%GHC_LIBDIR%%/package.conf.d/package.cache
4) on pkg-plist, add bin/haddock
Comment 10 John Marino freebsd_committer freebsd_triage 2015-08-22 20:51:32 UTC
hmmm, weird, post-install has rm bin/haddock, but it didn't work

"rm" -f "/wrkdirs/lang/ghc/work/stage/usr/local/bin/haddock" 


That is in the lock, but plist-check says it is orphaned.
Comment 11 John Marino freebsd_committer freebsd_triage 2015-08-22 20:54:58 UTC
never mind, I see a symlink is getting created in its place (after it's removed) but the symlink is not in the pkg-plist.
Comment 12 John Marino freebsd_committer freebsd_triage 2015-08-22 21:15:13 UTC
i don't think it's a multi-job issue anymore.  I'm back to thinking /usr/bin/ar is operating on a restricted area (which happens to be writable in testport which is why it works there)
Comment 13 John Marino freebsd_committer freebsd_triage 2015-08-22 22:12:29 UTC
why was "MAKE_ENV+=      TMPDIR=${TMPDIR}" removed?

Adding it back doesn't solve the problem, but I can't tell why it's not used anymore (TMPDIR is still defined in the makefile)
Comment 14 Gabor Pali freebsd_committer freebsd_triage 2015-08-23 09:14:05 UTC
(In reply to John Marino from comment #8):

The plist issue you raised is known, we have been doing this like that for a while now.  That is because the files you saw are automatically generated and gradually updated on installation of each hs- port.  (This is a global index of documentation, so one cannot guarantee that they are going to be exactly the same files on removing the lang/ghc port.)  Of course, if you have a solution for this problem, it is welcome for sure.


(In reply to John Marino from comment #10):

The symlink for haddock is used for letting the port being able to disable the generation of the (API) documentation.  When DOCS is not set, the haddock binary is not available in the port, but it may be installed later by the devel/hs-haddock port.  That is why this trickery is done.


(In reply to John Marino from comment #12):

Sorry, but I am afraid I cannot help you more with finding the solution for the problem you have been experiencing.


(In reply to John Marino from comment #13):

That line was removed because it turned out that TMPDIR was actually set twice in MAKE_ENV to exactly the same value.  So I did not see much sense to keep both of those redundant settings.
Comment 15 John Marino freebsd_committer freebsd_triage 2015-08-23 09:41:28 UTC
(In reply to Gabor Pali from comment #14)
"Of course, if you have a solution for this problem, it is welcome for sure"

doesn't defining "PORTSDOCS=html" fix it correctly in every use case?

"That is why this trickery is done."

If you intentionally don't want to install bin/haddock (only the symlink), then you need to remove ${STAGEDIR}${PREFIX}/bin/haddock in post-install right?

"Sorry, but I am afraid I cannot help you more with finding the solution for the problem you have been experiencing."

Technically you are experiencing it too, but the build is not breaking.  So far my attempts at making ghc-cabal set -v with /usr/bin/ar invocation has failed.

Unfortunately the penalty for failure is removing every haskell port from dports.  I am trying to avoid that but this has already taken hours and hours of my time.

"So I did not see much sense to keep both of those redundant settings."

I changed this to DEFAULT_TMPDIR per instructions https://downloads.haskell.org/~ghc/7.0.1/docs/html/users_guide/separate-compilation.html
but that didn't solve my immediate problem.
It's probably not necessary though.
Comment 16 John Marino freebsd_committer freebsd_triage 2015-08-23 09:46:40 UTC
actually it's probably "PORTDOCS=html"

(not PORTSDOCS)
Comment 17 John Marino freebsd_committer freebsd_triage 2015-08-23 09:54:11 UTC
okay, so bin/haddock should be added to PLIST_FILES

.e.g.

DOCS_PLIST_FILES= bin/haddock

(after .include <bsd.port.options.mk>, but I haven't tested yet)
Comment 18 John Marino freebsd_committer freebsd_triage 2015-08-23 11:04:41 UTC
I set the default verbosity to "deafening":

Configuring hsc2hs-0.67...
creating dist
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3",["--numeric-version"])
Configuring binary-0.7.5.0...
creating dist-boot
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3",["--numeric-version"])
/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3 is version 7.6.3
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-pkg-7.6.3",["--version"])
/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3 is version 7.6.3
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-pkg-7.6.3",["--version"])
/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-pkg-7.6.3 is version 7.6.3
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3",["--supported-languages"])
/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-pkg-7.6.3 is version 7.6.3
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3",["--supported-languages"])
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3",["--info"])
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3",["--info"])
Reading installed packages...
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-pkg-7.6.3",["dump","--global","-v0"])
Reading installed packages...
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-pkg-7.6.3",["dump","--global","-v0"])
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-pkg-7.6.3",["dump","--package-db=/wrkdirs/lang/ghc/work/ghc-7.10.2/libraries/bootstrapping.conf","-v0"])
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3",["--print-libdir"])
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-pkg-7.6.3",["dump","--package-db=/wrkdirs/lang/ghc/work/ghc-7.10.2/libraries/bootstrapping.conf","-v0"])
Dependency base ==4.*: using base-4.6.0.1
Dependency containers >=0.2 && <0.6: using containers-0.5.0.0
Dependency directory >=1 && <1.3: using directory-1.2.0.1
Dependency filepath >=1 && <1.5: using filepath-1.3.0.1
Dependency process >=1 && <1.3: using process-1.1.0.2
Searching for alex in path.
Cannot find alex on the path
ghc-cabal: /usr/bin/ar: permission denied
utils/hsc2hs/ghc.mk:15: recipe for target 'utils/hsc2hs/dist/package-data.mk' failed
gmake[2]: *** [utils/hsc2hs/dist/package-data.mk] Error 1
gmake[2]: *** Waiting for unfinished jobs....
("/wrkdirs/lang/ghc/work/ghc-7.6.3-boot/bin/ghc-7.6.3",["--print-libdir"])
Dependency array -any: using array-0.4.0.1
Dependency base >=3.0 && <5: using base-4.6.0.1
Dependency bytestring >=0.9: using bytestring-0.10.0.2
Dependency containers -any: using containers-0.5.0.0
Searching for alex in path.
Cannot find alex on the path
ghc-cabal: /usr/bin/ar: permission denied
libraries/binary/ghc.mk:3: recipe for target 'libraries/binary/dist-boot/package-data.mk' failed
gmake[2]: *** [libraries/binary/dist-boot/package-data.mk] Error 1
Makefile:71: recipe for target 'all' failed
gmake[1]: *** [all] Error 2
gmake[1]: Leaving directory '/wrkdirs/lang/ghc/work/ghc-7.10.2'
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make: stopped in /usr/ports/lang/ghc
=> Cleaning up wrkdir
===>  Cleaning for ghc-7.10.2
build of lang/ghc ended at Sun Aug 23 04:01:15 PDT 2015
build time: 00:01:59
!!! build failure encountered !!!


Just before the permission error, I'm seeing "can't find alex on the path".  I don't know if that is important or not.  (I don't know what "alex" is)
Comment 19 John Marino freebsd_committer freebsd_triage 2015-08-23 11:08:38 UTC
This appears to be the exact issue I'm seeing:

https://github.com/haskell/cabal/issues/2653
Comment 20 John Marino freebsd_committer freebsd_triage 2015-08-23 11:16:45 UTC
If I understood the quick reading of this:
https://ghc.haskell.org/trac/ghc/blog/weekly20150721#MacOSXElCapitansupport

basically ghc 7.8 is broken.  It's likely the same issue is on ghc 7.6.

The way forward may be to build a new bootstrap 7.10 without the RO environment then then presumably the new bootstrap will be able to get past this part.
Comment 21 John Marino freebsd_committer freebsd_triage 2015-08-23 11:39:24 UTC
Interesting ...

The "Configuring hsc2hs-0.67..." and "Configuring binary-0.7.5.0..." is not jobs safe.  Those need to be run in series.

I'd better if you can hack it such that the configure commands are always done under jobs=1 regardless of jobs setting, ghc would then be jobs safe.
Comment 22 Gabor Pali freebsd_committer freebsd_triage 2015-08-23 19:21:25 UTC
(In reply to John Marino from comment #15):

Thanks for your hint, I have not yet tried to use PORTDOCS=html to work around the problem with the orphaned files for check-plist.  I will take a try with that.

Haddock is bundled with GHC, but it is disabled when no documentation is requested by the user.  However, haddock can be also installed as a separate port, since it is actually an independently installable software.  The reason why it is still tied to GHC is that it uses the same grammar definition for parsing.

I am still not convinced that I have been experiencing the problem that you have.  I could not reproduce it, which would be basically essential to fix it.  And interesting question though, why the build in poudriére does not break on FreeBSD where it does on DragonFly -- if you are right.

The DEFAULT_TMPDIR variable is to tell GHC where to put its intermediate files during compilation.  This is /tmp by default and should not do much harm, since GHC will not fill up so quickly.


(In reply to John Marino from comment #15):

Alex is a lexer generator, it is like flex(1) but implemented in and for Haskell.  This is not required for building GHC as the release source tarballs include the pre-generated lexers (and parsers, produced by Happy) for the grammar of GHC.


(In reply to John Marino from comment #20):

I do not really see the connection between the "rootless" mode of Mac OS X and the experienced behavior on DragonFly BSD.  The reason why I chose to update the bootstrap compiler to 7.10 is that it has better support for building with Clang.  But it may happen that it also solves the problem in the bundled unix library -- as I said earlier, I have not bumped into this issue with ar(1) before, even with 7.6.3 as bootstrap.  (And I think this was not an issue for the update to 7.8.3 on DragonFly as well.)


(In reply to John Marino from comment #21):

Off top of my head, I have no idea how to achieve the serialization of the configuration of the hsc2hs and binary packages.  In addition, I believe there are many other places where the build of GHC may become unsafe when building on multiple threads.
Comment 23 John Marino freebsd_committer freebsd_triage 2015-08-23 19:37:10 UTC
(In reply to Gabor Pali from comment #22)
"I have not yet tried to use PORTDOCS=html to work around the problem with the orphaned files for check-plist."

I've tried it; it fixes it.
Also setting PLIST_SUBS for bin/haddock and the symlink for DOCS options fixes the plist

"I am still not convinced that I have been experiencing the problem that you have."

When I said you "experienced" it before, I mean silently (meaning it did not break so you didn't detect it).  Now I don't think that happened.

"And interesting question though, why the build in poudriére does not break on FreeBSD where it does on DragonFly"

Because the programs are implemented differently.  FreeBSD does not set the base to "read only" because it permits ports to install anywhere, including /usr/bin.  DragonFly made the decision that this behavior is absolutely not allowed and thus all these are mounted separately and set to read only.

"I do not really see the connection between the "rootless" mode of Mac OS X and the experienced behavior on DragonFly BSD."

Because GHC <= 7.8 is attempting to do something that fails when rootless / readonly filesystem.  It's a bug.

"as I said earlier, I have not bumped into this issue with ar(1) before,"

I don't think it's /usr/bin/ar per se, but this issue described in that bug report.  It's just showing up on /usr/bin/ar first.

I'm 80% sure that updating the bootstrap to 7.10 will fix it because that version fixes the GHC bug that I think I'm hitting.
Comment 24 Gabor Pali freebsd_committer freebsd_triage 2015-08-23 20:23:21 UTC
(In reply to John Marino from comment #23):

The "PORTDOCS= html" line you suggested is actually implemented by the following lines at the "post-install-script" target:

       @${FIND} -ds ${STAGEDIR}${DOCSDIR} -type f -print | ${SED} -E -e \
               '/^${STAGEDIR:C/\//\\\//g}${DOCSDIR:C/\//\\\//g}\/html\/libraries\/doc-index-[^.]+\.html/d' -e \
               '/^${STAGEDIR:C/\//\\\//g}${DOCSDIR:C/\//\\\//g}\/html\/libraries\/index[^\/.]*\.html/d' -e \
               '/^${STAGEDIR:C/\//\\\//g}${DOCSDIR:C/\//\\\//g}\/html\/libraries\/[^\/]+\.png/d' -e \
               's,^${STAGEDIR}${PREFIX}/?,,' >> ${TMPPLIST}

The reason why the index files generated by Haddock are left off is that they will gradually change on installing each of the hs- ports (that comes with generated documentation), so the SHA256 of the files will also change.  The case of "package.cache" is similar.

While you can happily satisfy the "check-plist" target by letting the generated files enter the plist, after installing a few dependents you will soon get this when trying remove the ghc package:

pkg: /usr/local/share/doc/ghc-7.10.2/html/libraries/doc-index-B.html different from original checksum, not removing
...

Note that the succeeding lines are to take care of the generation and deletion of the documentation indexes and package.cache on installation and deinstallation of GHC:

.if ${PORT_OPTIONS:MDOCS}
        @${ECHO} "@unexec ${RM} -f %D/${DOCSDIR_REL}/html/libraries/doc-index-*.html" >> ${TMPPLIST}
        @${ECHO} "@unexec ${RM} -f %D/${DOCSDIR_REL}/html/libraries/index*.html" >> ${TMPPLIST}
        @${ECHO} "@unexec ${RM} -f %D/${DOCSDIR_REL}/html/libraries/*.png" >> ${TMPPLIST}
.endif
        @${ECHO} '@unexec ${RM} -f %D/${GHC_LIBDIR_REL}/package.conf.d/package.cache' >>${TMPPLIST}
        @${ECHO} "@exec %D/bin/ghc-pkg recache" >>${TMPPLIST}
.if ${PORT_OPTIONS:MDOCS}
        @${ECHO} '@exec ${SH} -c "cd %D/${DOCSDIR_REL}/html/libraries && ./gen_contents_index"' >> ${TMPPLIST}
.endif


Can it be done better?
Comment 25 John Marino freebsd_committer freebsd_triage 2015-08-23 20:33:28 UTC
wow, that's a first.
I don't know if any package systems can handle that gracefully.
Apparently GHC ecosystem did not consider this at all when adopting such an approach.

I guess it's basically impossible to pass QA checks then when this kind of workarounds (and no, I can't think of "better" ways that don't involve reworking documentation from scratch)
Comment 26 John Marino freebsd_committer freebsd_triage 2015-08-24 09:06:16 UTC
okay, my theory was correct.
It took me a while to make the bootstrap properly (it's different from FreeBSD, it's fully-built, not partially built).  Once I did, lang/ghc builds.

I made the following modifications to it:
https://raw.githubusercontent.com/DragonFlyBSD/DeltaPorts/master/ports/lang/ghc/diffs/Makefile.diff

https://raw.githubusercontent.com/DragonFlyBSD/DeltaPorts/master/ports/lang/ghc/diffs/distinfo.diff


and because an existing patch didn't modify for DragonFly, I had to add this patch (It could easily be incorporated into existing patch):

https://raw.githubusercontent.com/DragonFlyBSD/DeltaPorts/master/ports/lang/ghc/dragonfly/patch-libraries_Cabal_Cabal_Distribution_Simple_GHC.hs
Comment 27 John Marino freebsd_committer freebsd_triage 2015-09-10 08:28:54 UTC
Closing.

Answer is that new bootstrap is required for use in the environment that DragonFly poudriere provides.  A new bootstrap was created so there's no more issue.