Bug 234126 - misc/freebsd-doc-en: blank page!! from .ps.eps suffix rule (later leads to pnmtopng: bad magic number 0x2d67)
Summary: misc/freebsd-doc-en: blank page!! from .ps.eps suffix rule (later leads to pn...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Document Engineering Group (Nobody)
Keywords: patch-ready
: 226091 234127 (view as bug list)
Depends on:
Reported: 2018-12-18 09:53 UTC by O. Hartmann
Modified: 2020-04-24 22:18 UTC (History)
6 users (show)

See Also:
mandree: maintainer-feedback+

poudriere build log of r498480 (96.30 KB, application/x-xz-compressed)
2019-04-13 16:52 UTC, Matthias Andree
no flags Details
another poudriere log (xz-compressed) (96.80 KB, application/octet-stream)
2020-04-06 15:03 UTC, Matthias Andree
no flags Details
gs->texlive (1.73 KB, patch)
2020-04-07 07:54 UTC, Dima Panov
no flags Details | Diff
default to A4 (747 bytes, patch)
2020-04-07 14:51 UTC, Dima Panov
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description O. Hartmann 2018-12-18 09:53:18 UTC
Port misc/freebsd-doc-en fails to build, see error below.

Settings in /etc/make.conf (if matters):
DOC_LANG=en_US.ISO8859-1 de_DE.ISO8859-1
FORMATS=html-split pdf
WEB_LANG=de_DE.ISO8859-1 en_US.ISO8859-1

/usr/local/bin/perl /usr/ports/misc/freebsd-doc-en/work/doc/share/misc/epsgeom -offset 100 100 /usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps  | /usr/local/bin/gs -q -dBATCH -dGraphicsAlphaBits=4 -dTextAlphaBits=4  -dEPSCrop -r100x100  -dNOPAUSE -dSAFER -sDEVICE=pnm -sOutputFile=-  -g`/usr/local/bin/perl /usr/ports/misc/freebsd-doc-en/work/doc/share/misc/epsgeom -geom 100 100 /usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps` -  | /usr/local/bin/pnmtopng > /usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.png
Error: no BoundingBox found: /usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps
Error: no BoundingBox found: /usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps
pnmtopng: bad magic number 0x2d67 - not a PPM, PGM, PBM, or PAM file
*** Error code 1
Comment 1 Matthias Andree freebsd_committer 2019-04-13 16:52:45 UTC
Created attachment 203647 [details]
poudriere build log of r498480

up-to-date poudriere build log showing the same problem as of 52910 aka ports r498480.
Comment 2 Matthias Andree freebsd_committer 2019-04-13 17:18:10 UTC
It would seem that the ..../net-routing.eps file is incomplete, it starts like this:

save countdictstack mark newpath /showpage {} def /setpagedevice {pop} def
%%Page: 1 1
%%BeginDocument: /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps
/setpacking where{

Debugging this further, I find that the .ps file has 

%%DocumentMedia: Default 595 842 0 () ()
<< /PageSize [ 595 842 ] /ImagingBBox null >> setpagedevice

which corresponds to A4, where the doc.images.mk HARDCODES -sPAPERSIZE=letter from somewhere, and then ps2epsi complains "blank page!!". Omitting -sPAPERSIZE=letter appears to fix the complaints, but I couldn't check output.
Comment 3 Matthias Andree freebsd_committer 2019-04-13 17:19:48 UTC
this is the relevant snipped from the build:
[mandree@freeryzen /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook]$ make
tmpfile=$(mktemp /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps.XXXXXXXX);  groff -p -S -Wall -mtty-char -man /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.pic > $tmpfile && /bin/mv -f $tmpfile /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps
tmpfile=$(mktemp /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps.XXXXXXXX);  /usr/local/bin/gs -q -dNOPAUSE -dBATCH -dSAFER -dDELAYSAFER  -sPAPERSIZE=letter -r72 -sDEVICE=bbox  -sOutputFile=/dev/null  /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps > $tmpfile 2>&1;  /usr/bin/env outfile=$tmpfile /usr/local/bin/gs -q -dNOPAUSE -dSAFER -dDELAYSAFER  -sPAPERSIZE=letter -r72 -sDEVICE=bit  -sOutputFile=/dev/null  ps2epsi.ps < /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps 1>&2;  (echo "save countdictstack mark newpath /showpage {} def /setpagedevice {pop} def"; echo "%%EndProlog"; echo "%%Page: 1 1"; echo "%%BeginDocument: /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps"; ) >> $tmpfile;  /usr/bin/sed	-e '/^%%BeginPreview:/,/^%%EndPreview[^!-~]*$/d'  -e '/^%!PS-Adobe/d'  -e '/^%%[A-Za-z][A-Za-z]*[^!-~]*$/d' -e '/^%%[A-Za-z][A-Za-z]*: /d' < /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps >> $tmpfile;  (echo "%%EndDocument"; echo "%%Trailer"; echo "cleartomark countdictstack exch sub { end } repeat restore"; echo "%%EOF"; ) >> $tmpfile;  /bin/mv -f $tmpfile /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps
blank page!!
/usr/local/bin/perl /usr/ports.svn/misc/freebsd-doc-en/work/doc/share/misc/epsgeom -offset 100 100 /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps  | /usr/local/bin/gs -q -dBATCH -dGraphicsAlphaBits=4 -dTextAlphaBits=4  -dEPSCrop -r100x100  -dNOPAUSE -dSAFER -sDEVICE=pnm -sOutputFile=-  -g`/usr/local/bin/perl /usr/ports.svn/misc/freebsd-doc-en/work/doc/share/misc/epsgeom -geom 100 100 /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps` -  | /usr/local/bin/pnmtopng > /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.png
Error: no BoundingBox found: /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps
Error: no BoundingBox found: /usr/ports.svn/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps
pnmtopng: bad magic number 0x2d67 - not a PPM, PGM, PBM, or PAM file
*** Error code 1
Comment 4 Dries Michiels 2019-10-26 17:13:47 UTC
I can confirm this issue, anyone been able to check why this happens? I'm running 12-stable.
Comment 5 Walter Schwarzenfeld freebsd_triage 2019-10-26 18:32:22 UTC
*** Bug 226091 has been marked as a duplicate of this bug. ***
Comment 6 Matthias Andree freebsd_committer 2020-01-05 02:24:13 UTC
bug persists in early 2020.
Comment 7 Dries Michiels 2020-01-05 10:40:43 UTC
I think its a very, very, very specific race condition. Sometimes I'm able to build the doc tree without any problem. Other times it fails with this error message.
Comment 8 Matthias Andree freebsd_committer 2020-01-05 16:00:33 UTC
I agree it's subtle and specific and possibly hard to find, but I have been unable to build en-freebsd-doc for more than 15 months now, on various computers (all hosted in Germany or the Netherlands, if that matters with paper sizes, time zones, date formats, or whatnot), without a single success.
Comment 9 Vladimir Omelchuk 2020-04-05 13:14:18 UTC
I confirm this issue too!
Very long time can't update freebsd-doc! :(

# uname -srm
FreeBSD 12.1-RELEASE-p3 amd64
Comment 10 Dima Panov freebsd_committer 2020-04-05 20:48:46 UTC
12.1, clean build in poudriere, HTML-SPLIT and PDF selected via ports options

Builds fine for -en. -de, -ru

Comment 11 Matthias Andree freebsd_committer 2020-04-05 20:59:24 UTC
(In reply to Dima Panov from comment #10)

Dima, "works for me" is not a useful argument here. We've seen that some people find no build troubles, for other, it is troublesome. We haven't yet established if that's due to locale, paper size, deviations in environment, or whatever.
Comment 12 Dima Panov freebsd_committer 2020-04-06 12:24:58 UTC
(In reply to Matthias Andree from comment #11)

Mattias, I've tried alot of variations to build and can't reproduce error at all :(

Locales (en_GB.UTF-8, de_DE.ISO8859-1, ja_JP.eucJP, nl_NL.ISO8859-1), timezones, html/split/pdf, real 12.1/11.3 and poudriere are always build fine.

As I see in provided logs, broken file come to pnmtopng via doc/share/misc/epsgeom perl script. 
Looks like perl construction
while(<IN>) {
        last if (($x,$y,$width,$height) =
        #print STDERR "DEBUG: $_";
close IN;
is broken, perl cannot read and parse string from properly opened file. 

if (not defined($x)) {
        die "Error: no BoundingBox found: $file\n";
confirms parsing error. 

At first, I suspect messed up perl installation. 
Just for test, can you reinstall the whole perl with modules on any of machines which breaks build for docs?
Comment 13 Dima Panov freebsd_committer 2020-04-06 12:47:59 UTC
Second suspect is scr2png + netpbm 

scr2png depends on libpng16 which was messed some time ago while updates from previous version. I still remember libpng/zlib issues in late 2018

Broken scr2png is may be an issue to produce non-valid .png file and than .eps via png2eps
Comment 14 Matthias Andree freebsd_committer 2020-04-06 14:58:54 UTC
I've tried in a poudriere jail, same result

tmpfile=$(mktemp /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps.XXXXXXXX);  groff -p -S -Wall -mtty-char -man /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.pic > $tmpfile && /bin/mv -f $tmpfile /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps
tmpfile=$(mktemp /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps.XXXXXXXX);  /usr/local/bin/gs -q -dNOPAUSE -dBATCH -dSAFER -dDELAYSAFER  -sPAPERSIZE=letter -r72 -sDEVICE=bbox  -sOutputFile=/dev/null  /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps > $tmpfile 2>&1;  /usr/bin/env outfile=$tmpfile /usr/local/bin/gs -q -dNOPAUSE -dSAFER -dDELAYSAFER  -sPAPERSIZE=letter -r72 -sDEVICE=bit  -sOutputFile=/dev/null  ps2epsi.ps < /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps 1>&2;  (echo "save countdictstack mark newpath /showpage {} def /setpagedevice {pop} def"; echo "%%EndProlog"; echo "%%Page: 1 1"; echo "%%BeginDocument: /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps"; ) >> $tmpfile;  /usr/bin/sed	-e '/^%%BeginPreview:/,/^%%EndPreview[^!-~]*$/d'  -e '/^%!PS-Adobe/d'  -e '/^%%[A-Za-z][A-Za-z]*[^!-~]*$/d' -e '/^%%[A-Za-z][A-Za-z]*: /d' < /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.ps >> $tmpfile;  (echo "%%EndDocument"; echo "%%Trailer"; echo "cleartomark countdictstack exch sub { end } repeat restore"; echo "%%EOF"; ) >> $tmpfile;  /bin/mv -f $tmpfile /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps
blank page!!
/usr/local/bin/perl /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/share/misc/epsgeom -offset 100 100 /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps  | /usr/local/bin/gs -q -dBATCH -dGraphicsAlphaBits=4 -dTextAlphaBits=4  -dEPSCrop -r100x100  -dNOPAUSE -dSAFER -sDEVICE=pnm -sOutputFile=-  -g`/usr/local/bin/perl /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/share/misc/epsgeom -geom 100 100 /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps` -  | /usr/local/bin/pnmtopng > /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.png
Error: no BoundingBox found: /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps
Error: no BoundingBox found: /wrkdirs/usr/ports/misc/freebsd-doc-en/work/doc/en_US.ISO8859-1/books/handbook/../../../share/images/books/handbook/advanced-networking/net-routing.eps
pnmtopng: bad magic number 0x2d67 - not a PPM, PGM, PBM, or PAM file
*** Error code 1

Comment 15 Matthias Andree freebsd_committer 2020-04-06 15:03:16 UTC
Created attachment 213127 [details]
another poudriere log (xz-compressed)
Comment 16 Kurt Jaeger freebsd_committer 2020-04-06 18:33:29 UTC
(In reply to Dima Panov from comment #12)
How many build threads does your poudriere run use ? My assumption is that
it happens if a certain amount of parallelism is used during the build.
Comment 17 Kurt Jaeger freebsd_committer 2020-04-06 18:34:43 UTC
(In reply to Kurt Jaeger from comment #16)
Search for

[00:00:00] MAKE_JOBS_NUMBER=    10

in the build log.
Comment 18 Matthias Andree freebsd_committer 2020-04-06 19:57:44 UTC
Kurt, I don't think we need to look there,

$ grep -n MAKE_JOBS /usr/ports.svn/misc/freebsd-doc-en/Makefile 
Comment 19 Dima Panov freebsd_committer 2020-04-07 04:35:31 UTC
It is not a parallelism, sure. this is a bad file after first run of groff/gs

I was able to reproduce in one of my poudrieres. 
12.1, and only differs in gs (and parents) options. 

after groff we have a valid .ps
firs run of gs (gs -q -dNOPAUSE -dBATCH -dSAFER -dDELAYSAFER  -sPAPERSIZE=letter -r72 -sDEVICE=bbox  -sOutputFile=/dev/null net-routing.ps >& net-routing.eps.xxxx) produces a valid header

%%BoundingBox: 71 793 447 831
%%HiResBoundingBox: 71.800781 793.691406 446.597656 830.089844)

but the second run of gs with ps2epsi.ps filter in this pipeline produces a no output at all

gs -q -dNOPAUSE -dBATCH -dSAFER -dDELAYSAFER  -sPAPERSIZE=letter -r72 -sDEVICE=bit  -sOutputFile=/dev/null ps2epsi.ps <net-routing.ps  >& net-routing.eps.yyyy)

net-routing.eps.yyyy is empty
Comment 20 Dima Panov freebsd_committer 2020-04-07 04:53:10 UTC
And more

If I call /usr/local/bin/ps2epsi by hand, it produces .epsi file but complain about error in source

ps2epsi net-routing.ps
Error: /undefined in getenv
Operand stack:
Execution stack:
   %interp_exit   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   --nostringval--   --nostringval--   false   1   %stopped_push   1990   1   4   %oparray_pop   1989   1   4   %oparray_pop   1977   1   4   %oparray_pop   1833   1   4   %oparray_pop   --nostringval--   %errorexec_pop   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--
Dictionary stack:
   --dict:739/1123(ro)(G)--   --dict:0/20(G)--   --dict:82/200(L)--   --dict:5/30(L)--   --dict:11/30(L)--
Current allocation mode is local
Current file position is 8386
GPL Ghostscript 9.52: Unrecoverable error, exit code 1
Comment 21 Dima Panov freebsd_committer 2020-04-07 07:54:16 UTC
Created attachment 213145 [details]

Confirmed, this is an ghostscript issue. need further investigation which dependency break it.

however, switch over ps2eps from texlive produces a valid .eps file without dances with awk, echo and buggy ghostscript
Comment 22 Dima Panov freebsd_committer 2020-04-07 14:51:32 UTC
Created attachment 213156 [details]
default to A4

Better way: default papersize to a4

groff/grops is already able to deal with not only letter but a4 and many other paper formats.

Unconditional set paper parameters to a4 size for groff and gs allows build a valid images/eps/pdf in any circumstances
Comment 23 Matthias Andree freebsd_committer 2020-04-07 15:00:48 UTC
Comment on attachment 213156 [details]
default to A4

While this appears to work, there seems to be a bug in the Makefiles which use a lot of ";" where they should probably use "&&" - the $tmpfile is moved into the output although the PS2EPS has failed.  We should fix that first so that we don't let b0rked output files (nonzero exit from generator) go unnoticed.

This affects, for instance, the .ps.eps: suffix rule on ll. 143ff. of share/mk/doc.images.mk.
Comment 24 Matthias Andree freebsd_committer 2020-04-07 16:58:26 UTC
(In reply to Dima Panov from comment #22)
LGTM: with your patch, I can now build misc/freebsd-doc-all (which entails all languages) successfully.

NOTE: there is an unrelated failure if the DISPLAY environment variable set but the display does not work properly (xauth data expired, ssh forward stalled, whatever), so perhaps the build should also strip DISPLAY from the environment to prevent any X11 popup window attempts.
Comment 25 Dries Michiels 2020-04-08 09:48:00 UTC
I can confirm that the patch to default to A4 fixes this issue consistently for me as well.
Comment 26 Matthias Andree freebsd_committer 2020-04-22 17:50:18 UTC
*** Bug 234127 has been marked as a duplicate of this bug. ***
Comment 27 Matthias Andree freebsd_committer 2020-04-22 17:51:30 UTC
maintainer feedback timeout (doceng@, 15 days since https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=234126#c22 above)
Comment 28 commit-hook freebsd_committer 2020-04-22 18:15:10 UTC
A commit references this bug:

Author: mandree
Date: Wed Apr 22 18:14:26 UTC 2020
New revision: 532494
URL: https://svnweb.freebsd.org/changeset/ports/532494

  Fix docs build that is broken on some systems.

  Apparently, changes in the docproj toolchain sensitivized ps2epsi.ps
  (ghostscript) against paper size mismatches in some circumstances.
  Set the paper size consistently to A4 for PIC2PS (groff), PS2EPS (gs),
  PS2BBOX (also gs). [1]

  While here, blank DISPLAY so that the textproc/fop tool can make progress
  if the DISPLAY isn't working (stalled ssh forward, xauth broken, whatever).

  Remove MAKE_JOBS_UNSAFE, this was brought in in an attempt to fix the
  build issues which turned out to be blank gs (Ghostscript) output rather
  than broken dependencies in the Makefiles.

  PR:		234126
  PR:		234127 (duplicate of 234126)
  Submitted by:	fluffy@FreeBSD.org [1]
  Reported by:	ohartmann@walstatt.org
  Reviewed by:	driesm.michiels@gmail.com
  Approved by:	maintainer timeout (doceng@, 15 days)

Comment 29 Dries Michiels 2020-04-24 08:31:24 UTC
Hi guys, I was wondering if we could fix this in the doc branch?
I'm building docs from the doc tree directly instead of the port.
Comment 30 Matthias Andree freebsd_committer 2020-04-24 20:12:09 UTC
Dries, Marc Fonvieille committed r54073 earlier that fixes doc/ for me.
Comment 31 Dries Michiels 2020-04-24 22:18:53 UTC
Oh yeah I see, thanks for the heads-up!