Bug 214766

Summary: sysutils/lcdproc: Fix build with MAKE_JOBS (parallel make)
Product: Ports & Packages Reporter: Lawrence Stewart <lstewart>
Component: Individual Port(s)Assignee: Thomas Zander <riggs>
Status: Closed FIXED    
Severity: Affects Some People CC: bsdfan, glebius, koobs, riggs
Priority: --- Keywords: needs-qa, patch
Version: LatestFlags: riggs: maintainer-feedback-
riggs: merge-quarterly+
Hardware: amd64   
OS: Any   
Attachments:
Description Flags
Adds required make dependency on libLCD.a to glcd.so target and libLCD.a/libbignum.a to hd44780.so target so that make can correctly order parallel jobs
none
Adds required make dependency on libLCD.a to glcd.so target and libLCD.a/libbignum.a to hd44780.so target so that make can correctly order parallel jobs none

Description Lawrence Stewart freebsd_committer freebsd_triage 2016-11-23 06:29:18 UTC
Created attachment 177310 [details]
Adds required make dependency on libLCD.a to glcd.so target and libLCD.a/libbignum.a to hd44780.so target so that make can correctly order parallel jobs

Building this port on a system with 28 CPUs that is loaded with other work sometimes fails with:


--- glcd.so ---
cc -fPIC -Wall -O2 -pipe  -fstack-protector -fno-strict-aliasing -Wno-unused-function -shared -fstack-protector -L/usr/lib  -o glcd.so glcd
-glcd_drv.o glcd-glcd-render.o libLCD.a  glcd-glcd-t6963.o t6963_low.o glcd-glcd-glcd2usb.o glcd-glcd-picolcdgfx.o    -lusb -lkvm
cc: error: no such file or directory: 'libLCD.a'


Building the port in a continuous loop on the loaded system will typically fail within 5mins. With the attached patch added to the port's "files" dir, the build loop survived multiple hours without a failure.
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2016-11-23 06:33:19 UTC
@Lawrence Is this something that can be sent upstream? 

I'm happy to (ports) approve this for you to commit if you can confirm the changes pass QA (poudriere)
Comment 2 Lawrence Stewart freebsd_committer freebsd_triage 2016-11-23 06:40:17 UTC
@Koobs: Yes, the upstream Makefile.am appears to also have the bug and could/should be patched there:

http://lcdproc.cvs.sourceforge.net/viewvc/lcdproc/lcdproc/server/drivers/Makefile.am?view=markup

However, I don't have a SourceForge account which is required to open a bug report, so hoping the port maintainer will take care of upstreaming.

Re poudriere, no promises but if I get to it I'll provide an update here.
Comment 3 Lawrence Stewart freebsd_committer freebsd_triage 2016-11-23 06:43:27 UTC
Hold that thought...

Just hit a different parallel jobs build failure after hundreds of successful builds:

--- hd44780.so ---
cc -fPIC -Wall -O2 -pipe  -fstack-protector -fno-strict-aliasing -Wno-unused-function -shared  -fstack-protector -L/usr/lib -o hd44780.so h
d44780-hd44780.o libLCD.a hd44780-hd44780-serial.o hd44780-hd44780-lis2.o hd44780-hd44780-usblcd.o hd44780-hd44780-4bit.o hd44780-hd44780-e
xt8bit.o hd44780-lcd_sem.o hd44780-hd44780-winamp.o hd44780-hd44780-serialLpt.o hd44780-hd44780-bwct-usb.o hd44780-hd44780-lcd2usb.o hd4478
0-hd44780-uss720.o hd44780-hd44780-usbtiny.o hd44780-hd44780-usb4all.o hd44780-hd44780-ethlcd.o hd44780-hd44780-i2c.o hd44780-hd44780-pipla
te.o -lusb  libbignum.a -lkvm
cc: error: no such file or directory: 'libLCD.a'
cc: error: no such file or directory: 'libbignum.a'

Seems there are multiple build targets with missing dependencies. Not sure if it's worth trying to find/fix them all or whether the port should just be marked as MAKE_JOBS_UNSAFE=yes
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2016-11-23 07:18:35 UTC
since, MARK_JOBS_UNSAFE=yes is correct for the ports current status currently, we can commit that now/first (referencing this PR: in commit log) and leave the issue open for the subsequent multiple/permanent-concurrent-build-failure-fix commits, as they all relate to this issue.

All of the above changes are 'blanket' and are therefore "Approved by: portmgr (blanket)".

You may go ahead and commit the MARK_JOBS_UNSAFE=yes addition (Approved by: koobs (ports))

Note: All of these commits should also be merged to the quarterly branch.
Comment 5 Kubilay Kocak freebsd_committer freebsd_triage 2016-11-23 07:19:35 UTC
Re merging: so include "MFH: 2016Q4" in the commit log message
Comment 6 Lawrence Stewart freebsd_committer freebsd_triage 2016-11-23 07:25:42 UTC
Comment on attachment 177310 [details]
Adds required make dependency on libLCD.a to glcd.so target and libLCD.a/libbignum.a to hd44780.so target so that make can correctly order parallel jobs

--- server/drivers/Makefile.am.orig     2016-11-23 04:12:38.949674000 +0000
+++ server/drivers/Makefile.am  2016-11-23 07:18:24.271646000 +0000
@@ -45,11 +45,11 @@
 CwLnx_LDADD =        libLCD.a libbignum.a
 g15_LDADD =          @LIBG15@
 glcd_LDADD =         libLCD.a @GLCD_DRIVERS@ @FT2_LIBS@ @LIBPNG_LIBS@ @LIBSERDISP@ @LIBUSB_LIBS@ @LIBX11_LIBS@
-glcd_DEPENDENCIES =  @GLCD_DRIVERS@ glcd-glcd-render.o
+glcd_DEPENDENCIES =  @GLCD_DRIVERS@ glcd-glcd-render.o libLCD.a
 glcdlib_LDADD =      @LIBGLCD@
 glk_LDADD =          libbignum.a
 hd44780_LDADD =      libLCD.a @HD44780_DRIVERS@ @LIBUSB_LIBS@ @LIBFTDI_LIBS@ libbignum.a
-hd44780_DEPENDENCIES = @HD44780_DRIVERS@
+hd44780_DEPENDENCIES = @HD44780_DRIVERS@ libLCD.a libbignum.a
 i2500vfd_LDADD =     @LIBFTDI_LIBS@
 imon_LDADD =         libLCD.a libbignum.a
 imonlcd_LDADD =      libLCD.a
--- server/drivers/Makefile.in.orig     2016-11-23 04:12:32.966160000 +0000
+++ server/drivers/Makefile.in  2016-11-23 07:16:31.989194000 +0000
@@ -565,11 +565,11 @@
 CwLnx_LDADD = libLCD.a libbignum.a
 g15_LDADD = @LIBG15@
 glcd_LDADD = libLCD.a @GLCD_DRIVERS@ @FT2_LIBS@ @LIBPNG_LIBS@ @LIBSERDISP@ @LIBUSB_LIBS@ @LIBX11_LIBS@
-glcd_DEPENDENCIES = @GLCD_DRIVERS@ glcd-glcd-render.o
+glcd_DEPENDENCIES = @GLCD_DRIVERS@ glcd-glcd-render.o libLCD.a
 glcdlib_LDADD = @LIBGLCD@
 glk_LDADD = libbignum.a
 hd44780_LDADD = libLCD.a @HD44780_DRIVERS@ @LIBUSB_LIBS@ @LIBFTDI_LIBS@ libbignum.a
-hd44780_DEPENDENCIES = @HD44780_DRIVERS@
+hd44780_DEPENDENCIES = @HD44780_DRIVERS@ libLCD.a libbignum.a
 i2500vfd_LDADD = @LIBFTDI_LIBS@
 imon_LDADD = libLCD.a libbignum.a
 imonlcd_LDADD = libLCD.a
Comment 7 Lawrence Stewart freebsd_committer freebsd_triage 2016-11-23 07:29:09 UTC
Created attachment 177314 [details]
Adds required make dependency on libLCD.a to glcd.so target and libLCD.a/libbignum.a to hd44780.so target so that make can correctly order parallel jobs

Looks like glcd and hd44780 were the only driver build targets missing the right deps, so v2 of the patch attached.
Comment 8 Lawrence Stewart freebsd_committer freebsd_triage 2016-11-23 07:34:22 UTC
I've started a fresh build loop with v2 of the patch in the files dir, and will let that run for the evening to see if it catches any new build failures.
Comment 9 Kubilay Kocak freebsd_committer freebsd_triage 2016-11-23 07:37:41 UTC
Thanks Lawrence

If you're uncomfortable committing this when QA is complete, re-assign to me and I'll take care of it for you (Though I prefer src committers get the practice)
Comment 10 Lawrence Stewart freebsd_committer freebsd_triage 2016-11-23 13:41:45 UTC
No build failures with the v2 patch during ~6h of build loop, so it seems to be a solid commit candidate.
Comment 11 Lawrence Stewart freebsd_committer freebsd_triage 2016-11-23 13:50:44 UTC
Oh, and for the record, all my testing has been with the following port options:

# This file is auto-generated by 'make config'.
# Options for lcdproc-0.5.7_2
_OPTIONS_READ=lcdproc-0.5.7_2
_FILE_COMPLETE_OPTIONS_LIST=USB FTDI HID FREETYPE PNG SERDISP X11
OPTIONS_FILE_SET+=USB
OPTIONS_FILE_UNSET+=FTDI
OPTIONS_FILE_UNSET+=HID
OPTIONS_FILE_UNSET+=FREETYPE
OPTIONS_FILE_UNSET+=PNG
OPTIONS_FILE_UNSET+=SERDISP
OPTIONS_FILE_UNSET+=X11
Comment 12 Thomas Zander freebsd_committer freebsd_triage 2017-03-12 11:34:39 UTC
Maintainer timeout
Comment 13 commit-hook freebsd_committer freebsd_triage 2017-03-12 11:37:00 UTC
A commit references this bug:

Author: riggs
Date: Sun Mar 12 11:36:22 UTC 2017
New revision: 435983
URL: https://svnweb.freebsd.org/changeset/ports/435983

Log:
  Fix parallel build (fix port-internal make dependencies) on many cores

  PR:		214766
  Submitted by:	lstewart
  MFH:		2017Q1 (build fix blanket)

Changes:
  head/sysutils/lcdproc/files/patch-server_drivers_Makefile.am
  head/sysutils/lcdproc/files/patch-server_drivers_Makefile.in
Comment 14 commit-hook freebsd_committer freebsd_triage 2017-03-12 11:58:20 UTC
A commit references this bug:

Author: riggs
Date: Sun Mar 12 11:57:36 UTC 2017
New revision: 435984
URL: https://svnweb.freebsd.org/changeset/ports/435984

Log:
  MFH: r435983

  Fix parallel build (fix port-internal make dependencies) on many cores

  PR:		214766
  Submitted by:	lstewart

  Approved by:	ports-secteam (build fix blanket)

Changes:
_U  branches/2017Q1/
  branches/2017Q1/sysutils/lcdproc/files/patch-server_drivers_Makefile.am
  branches/2017Q1/sysutils/lcdproc/files/patch-server_drivers_Makefile.in