Bug 237878

Summary: net/geoipupdate: textproc/hs-pandoc is incomplete
Product: Ports & Packages Reporter: Trond Endrestøl <Trond.Endrestol>
Component: Individual Port(s)Assignee: Gleb Popov <arrowd>
Status: Closed FIXED    
Severity: Affects Only Me CC: adamw, haskell, rhurlin
Priority: ---    
Version: Latest   
Hardware: Any   
OS: Any   

Description Trond Endrestøl 2019-05-13 14:00:30 UTC
For some reason pandoc from textproc/hs-pandoc refers to files created during the build of textproc/hs-pandoc and not captured by the package, causing these issues for net/geoipupdate:

--  Phase: build
--------------------------------------------------------------------------------
===>  Building for geoipupdate-4.0.2
gmake[2]: Entering directory '/construction/xports/net/geoipupdate/work/geoipupdate-4.0.2'
mkdir -p build
(cd cmd/geoipupdate && go build -ldflags '-X main.defaultConfigFile=/usr/local/etc/GeoIP.conf -X main.defaultDatabaseDirectory=/usr/local/share/GeoIP')
sed -e 's|CONFFILE|/usr/local/etc/GeoIP.conf|g' -e 's|DATADIR|/usr/local/share/GeoIP|g' conf/GeoIP.conf.default > build/GeoIP.conf
sed -e 's|CONFFILE|/usr/local/etc/GeoIP.conf|g' -e 's|DATADIR|/usr/local/share/GeoIP|g' doc/GeoIP.conf.md > build/GeoIP.conf.md
sed -e 's|CONFFILE|/usr/local/etc/GeoIP.conf|g' -e 's|DATADIR|/usr/local/share/GeoIP|g' doc/geoipupdate.md > build/geoipupdate.md
dev-bin/make-man-pages.pl
Could not find data file /construction/xports/textproc/hs-pandoc/work/cabal-home/.cabal/share/x86_64-freebsd-ghc-8.6.3/pandoc-2.7.1/data/templates/default.man
pandoc failed at dev-bin/make-man-pages.pl line 32.
gmake[2]: *** [Makefile:49: build/GeoIP.conf.5] Error 25
gmake[2]: *** Waiting for unfinished jobs....
cp cmd/geoipupdate/geoipupdate build
gmake[2]: Leaving directory '/construction/xports/net/geoipupdate/work/geoipupdate-4.0.2'
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make[1]: stopped in /xports/net/geoipupdate
*** Error code 1

Stop.
make: stopped in /xports/net/geoipupdate

Once you take care of ${WRKDIRPREFIX}/${PORTSDIR}/textproc/hs-pandoc/work/cabal-home/.cabal/share/x86_64-freebsd-ghc-8.6.3/pandoc-2.7.1/data/templates/default.man, then ${WRKDIRPREFIX}/${PORTSDIR}/textproc/hs-pandoc/work/cabal-home/.cabal/share/x86_64-freebsd-ghc-8.6.3/pandoc-2.7.1/data/abbreviations pops up.

I think Haskell's/cabal's notion of ${HOME} should rather be ${LOCALBASE}/share and thus be captured and preserved by the package(s) in a more reasonable way.

Short term mitigation is to fake the missing directories and files:

pre-build:
	mkdir -p ${WRKDIRPREFIX}/${PORTSDIR}/textproc/hs-pandoc/work/cabal-home/.cabal/share/x86_64-freebsd-ghc-8.6.3/pandoc-2.7.1/data/templates
	touch    ${WRKDIRPREFIX}/${PORTSDIR}/textproc/hs-pandoc/work/cabal-home/.cabal/share/x86_64-freebsd-ghc-8.6.3/pandoc-2.7.1/data/templates/default.man
	touch    ${WRKDIRPREFIX}/${PORTSDIR}/textproc/hs-pandoc/work/cabal-home/.cabal/share/x86_64-freebsd-ghc-8.6.3/pandoc-2.7.1/data/abbreviations

While this problem occured when building net/geoipupdate, the root cause lies within the Haskell ecosystem. Feel free to redirect to the Haskell maintainer(s).
Comment 1 Adam Weinberger freebsd_committer freebsd_triage 2019-05-13 14:56:51 UTC
Thanks for this! @arrowd recently committed a fix involving enabling by default the pandoc option that bundles everything into the binary.

The problem there is that it's an option, so anybody with it disabled (including all users who installed the current pandoc package before the clusters rebuilt it) are still unable to build geoipupdate.

I'm going to play with some other md-to-man formatters and see if they work, because pandoc is an absolutely ludicrous dependency for something this simple.
Comment 2 commit-hook freebsd_committer freebsd_triage 2019-05-13 15:24:06 UTC
A commit references this bug:

Author: adamw
Date: Mon May 13 15:23:42 UTC 2019
New revision: 501577
URL: https://svnweb.freebsd.org/changeset/ports/501577

Log:
  Use lowdown to build manpages

  Pandoc is a fantastic application. It's wonderful. I use it all the
  time to write papers in markdown and have them converted to beautiful
  LaTeX-driven PDFs. I have nothing against pandoc.

  However, using it to make manpages is like driving a Mack truck down the
  driveway to check the mailbox. On my system (8-core all-SSD on ZFS) it
  takes nearly 6 hours to build pandoc, and also these days it doesn't
  work (see PR below).

  Instead, use lowdown to generate the manpages. They don't come out
  exactly right, but it's close enough to be usable.

  PR:		237878

Changes:
  head/net/geoipupdate/Makefile
Comment 3 Adam Weinberger freebsd_committer freebsd_triage 2019-05-13 15:26:45 UTC
Ok I papered over the geoipupdate problem, but @arrowd you may still want to look at the underlying hs-pandoc problem here.
Comment 4 Gleb Popov freebsd_committer freebsd_triage 2019-05-13 16:21:22 UTC
(In reply to Adam Weinberger from comment #3)

Yes, hang this PR onto me.

RE

> On my system (8-core all-SSD on ZFS) it takes nearly 6 hours to build pandoc

This shouldn't be true after I added USES=cabal functionality into ports. Now the port compiles pretty quickly and the packages installs in a blink of eye.
Comment 5 commit-hook freebsd_committer freebsd_triage 2019-06-13 13:57:43 UTC
A commit references this bug:

Author: arrowd
Date: Thu Jun 13 13:57:22 UTC 2019
New revision: 504093
URL: https://svnweb.freebsd.org/changeset/ports/504093

Log:
  For ports with USES=cabal install a wrapper shell script instead of a real
  executable into /bin/, that sets some environment variables to workaround
  https://github.com/haskell/cabal/issues/5997

  While there, update bunch of Haskell ports, install their data files.

  PR:		237878
  Reviewed by:	tcberner
  Approved by:	tcberner (mentor)
  Differential Revision:	https://reviews.freebsd.org/D20556

Changes:
  head/Mk/Uses/cabal.mk
  head/converters/hs-aeson-pretty/Makefile
  head/devel/hs-ShellCheck/Makefile
  head/devel/hs-alex/Makefile
  head/devel/hs-bytestring-nums/Makefile
  head/devel/hs-c2hs/Makefile
  head/devel/hs-cpphs/Makefile
  head/devel/hs-darcs/Makefile
  head/devel/hs-ghc-events/Makefile
  head/devel/hs-ghc-events/distinfo
  head/devel/hs-git-annex/Makefile
  head/devel/hs-git-annex/pkg-plist
  head/devel/hs-haddock/Makefile
  head/devel/hs-haddock/distinfo
  head/devel/hs-haddock/pkg-plist
  head/devel/hs-happy/Makefile
  head/devel/hs-happy/pkg-plist
  head/devel/hs-hasktags/Makefile
  head/devel/hs-hlint/Makefile
  head/devel/hs-hoogle/Makefile
  head/devel/hs-hoogle/distinfo
  head/devel/hs-hspec-discover/Makefile
  head/devel/hs-hspec-discover/distinfo
  head/devel/hs-mueval/Makefile
  head/devel/hs-shake/Makefile
  head/devel/hs-shake/distinfo
  head/devel/hs-threadscope/Makefile
  head/devel/stack/Makefile
  head/games/hedgewars-server/Makefile
  head/games/hs-scroll/Makefile
  head/lang/hs-brainfuck/Makefile
  head/lang/hs-unlambda/Makefile
  head/print/hs-hscolour/Makefile
  head/textproc/cgrep/Makefile
  head/textproc/hs-lhs2tex/Makefile
  head/textproc/hs-pandoc/Makefile
  head/textproc/hs-pandoc/distinfo
  head/textproc/hs-pandoc/pkg-plist
  head/textproc/hs-pandoc-citeproc/Makefile
  head/textproc/hs-pandoc-citeproc/distinfo
  head/textproc/hs-pandoc-citeproc/pkg-plist
  head/textproc/hs-yaml/Makefile
  head/www/hs-DAV/Makefile
  head/www/hs-hjsmin/Makefile
  head/www/hs-wai-app-static/Makefile
  head/www/hs-wai-app-static/distinfo
  head/www/hs-yesod-bin/Makefile
  head/x11/hs-xmobar/Makefile
  head/x11/hs-xmobar/distinfo
  head/x11/hs-xmobar/files/patch-fix_bug378.diff