Bug 210648 - sysutils/slurm-wlm: update 15.06.12 -> 16.05.4 and fix build hangs
Summary: sysutils/slurm-wlm: update 15.06.12 -> 16.05.4 and fix build hangs
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Kurt Jaeger
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-06-27 23:15 UTC by Walter Schwarzenfeld
Modified: 2016-08-25 03:21 UTC (History)
3 users (show)

See Also:
pi: maintainer-feedback+


Attachments
svn patch to fix issue building docs (12.42 KB, patch)
2016-07-11 04:04 UTC, Joseph Mingrone
jrm: maintainer-approval+
Details | Diff
svn diff to fix haging build and the task affinity plugins (17.07 KB, patch)
2016-07-11 04:50 UTC, Joseph Mingrone
jrm: maintainer-approval+
Details | Diff
svn diff to update sysutils/slurm-wlm to version 16.05.4 (6.21 KB, patch)
2016-08-24 20:47 UTC, Joseph Mingrone
jrm: maintainer-approval+
Details | Diff
svn diff to update sysutils/slurm-wlm to version 16.05.4 (7.03 KB, patch)
2016-08-24 20:59 UTC, Joseph Mingrone
jrm: maintainer-approval+
Details | Diff
slurm-typescript_build.log (905.45 KB, text/plain)
2016-08-24 23:12 UTC, Walter Schwarzenfeld
no flags Details
svn diff to fix hanging build and to update to version 16.05.4 (7.99 KB, patch)
2016-08-25 00:07 UTC, Joseph Mingrone
jrm: maintainer-approval+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Walter Schwarzenfeld freebsd_triage 2016-06-27 23:15:46 UTC
Build hangs here:

Making all in man1
gmake[6]: Entering directory '/usr/ports/sysutils/slurm-wlm/work/slurm-15.08.12/doc/man/man1'
`dirname sacct.1`/../man2html.py 15.08 ./../../html/header.txt ./../../html/footer.txt sacct.1
`dirname sacctmgr.1`/../man2html.py 15.08 ./../../html/header.txt ./../../html/footer.txt sacctmgr.1
>>>>>>> sacctmgr.mhtml
('Converting', 'sacctmgr.mhtml', '->', 'sacctmgr.html')

[...SNIP...]

`dirname sview.1`/../man2html.py 15.08 ./../../html/header.txt ./../../html/footer.txt sview.1
>>>>>>> sview.mhtml
('Converting', 'sview.mhtml', '->', 'sview.html')

https://forums.freebsd.org/threads/56753/
Comment 1 Walter Schwarzenfeld freebsd_triage 2016-06-27 23:34:23 UTC
If I rum four times make, it compiles (it hangs three times).
Comment 2 Joseph Mingrone freebsd_committer 2016-06-28 00:09:16 UTC
The problem is with the upstream autoconf scripts.  They try to be "helpful" by dynamically installing components based on environment.  However, for some of these options there is no explicit argument to turn installation on or off.

I've been communicating with upstream and there may be a fix soon.  I may also try to fix the autoconf scripts myself.  In the meantime, you can build the port in a clean environment (e.g. with poudriere) or install the package.
Comment 3 Walter Schwarzenfeld freebsd_triage 2016-06-28 00:40:01 UTC
It hangs in man1/sview.1, man5/wiki.conf.1 and man8/spank.1. Each is the last file in the directory. It seems there is a problem to change the directories.
But it builds with poudriere (seems an enviroment "thing").
Comment 4 Walter Schwarzenfeld freebsd_triage 2016-06-28 00:41:01 UTC
Should be: change into other directories.
Comment 5 Joseph Mingrone freebsd_committer 2016-07-11 04:04:01 UTC
Created attachment 172365 [details]
svn patch to fix issue building docs

I've attached an unsatisfactory, temporary workaround:

CONFIGURE_ENV+=		ac_have_man2html=no

The problem seems to occur outside of poudriere as long as textproc/man2html is installed, regardless whether the DOCS option is on.  This will prevent the HTML files from being built in all cases.

While here, add other configuration lines:

GTK2_CONFIGURE_ENV_OFF=	ac_gtk_test=no
MYSQL_CONFIGURE_with=	mysql
Comment 6 Joseph Mingrone freebsd_committer 2016-07-11 04:50:19 UTC
Created attachment 172366 [details]
svn diff to fix haging build and the task affinity plugins

This second patch also includes source patches submitted by the former maintainer, Jason Bacon (bacon4000@gmail.com), which fix the task affinity plugin on FreeBSD.  These patches will be included in future upstream versions, so they will only be needed in the port temporarily.

portlint: OK
testport: OK (poudriere: 9.3-RELEASE-p36, i386,  default options)
testport: OK (poudriere: 9.3-RELEASE-p30, amd64, default options)
testport: OK (poudriere: 10.3-RELEASE,    i386,  default options)
testport: OK (poudriere: 10.3-RELEASE,    amd64, default options)

Thanks Jason.
Comment 7 Walter Schwarzenfeld freebsd_triage 2016-08-24 09:06:18 UTC
Maintainer timeout ?
Comment 8 Walter Schwarzenfeld freebsd_triage 2016-08-24 09:09:19 UTC
Sorry, no timout but needs a committer.
Comment 9 Joseph Mingrone freebsd_committer 2016-08-24 13:48:37 UTC
This is OK to be committed.  Hopefully it will be a moot point after I finish work on the upgrade to Slurm 16.
Comment 10 Kurt Jaeger freebsd_committer 2016-08-24 17:21:06 UTC
testbuilds@work
Comment 11 Kurt Jaeger freebsd_committer 2016-08-24 18:09:36 UTC
Testbuilds have issues with the pkg-plist provided.

See for example:

http://people.freebsd.org/~pi/logs/sysutils__slurm-wlm-cur-1472059253.txt
Comment 12 Joseph Mingrone freebsd_committer 2016-08-24 20:47:01 UTC
Created attachment 174027 [details]
svn diff to update sysutils/slurm-wlm to version 16.05.4

Instead of patching code that is already retired, let's try version 16.05.4, which includes Jason's task affinity plugins.  I've done some basic tests on our cluster and it seems fine.  However, I'm not confident that the issue with the hanging build is fixed because man2html.py hasn't changed since version 15.  I am not able to reproduce the problem, so w.schwarzenfeld can you confirm that the hanging build is an issue in version 16?

portlint: OK
testport: OK (poudriere: 9.3-RELEASE-p36, i386,  default options)
testport: OK (poudriere: 9.3-RELEASE-p30, amd64, default options)
testport: OK (poudriere: 10.3-RELEASE,    i386,  default options)
testport: OK (poudriere: 10.3-RELEASE,    amd64, default options)

[1] http://pkg.awarnach.mathstat.dal.ca/data/9i386-default/2016-08-24_17h32m14s/logs/slurm-wlm-16.05.4.log

[2] http://pkg.awarnach.mathstat.dal.ca/data/9amd64-default/2016-08-24_17h32m21s/logs/slurm-wlm-16.05.4.log

[3] http://pkg.awarnach.mathstat.dal.ca/data/10i386-default/2016-08-24_17h32m29s/logs/slurm-wlm-16.05.4.log

[4] http://pkg.awarnach.mathstat.dal.ca/data/10amd64-default/2016-08-24_16h51m17s/logs/slurm-wlm-16.05.4.log

----

P.S. The last patch built for me in poudriere [*] because I had DOCS included in OPTIONS_UNSET in make.conf.

[*] http://pkg.awarnach.mathstat.dal.ca/data/10amd64-default/2016-08-24_15h38m01s/logs/slurm-wlm-15.08.12.log
Comment 13 Joseph Mingrone freebsd_committer 2016-08-24 20:59:43 UTC
Created attachment 174029 [details]
svn diff to update sysutils/slurm-wlm to version 16.05.4

Take 2.  The last diff didn't include a new patch in files/.
Comment 14 Walter Schwarzenfeld freebsd_triage 2016-08-24 21:09:37 UTC
Sorry, it hangs in the port in the same as before.
Comment 15 Walter Schwarzenfeld freebsd_triage 2016-08-24 21:10:49 UTC
..in the same way...
Comment 16 Joseph Mingrone freebsd_committer 2016-08-24 21:14:08 UTC
Can you send the full build log as an attachment, the output of pkg info, and the output of uname -a?  Do you have anything in /etc/make.conf?
Comment 17 Walter Schwarzenfeld freebsd_triage 2016-08-24 21:28:51 UTC
Has nothing to do with /etc/make.conf - now I have renamed it and tried it again. Same result.
(But I can live with the "little" problem, I opened the PR cause of a post in FreeBSD Forum.
If it is not solvable, I can close this PR, if you want).
Comment 18 Walter Schwarzenfeld freebsd_triage 2016-08-24 23:00:47 UTC
Sorry, something went last try wrong.
Now I tried again, and it works (maybe I have forgot make clean - I don't know).
But now it works, thanks.
Comment 19 Joseph Mingrone freebsd_committer 2016-08-24 23:03:22 UTC
No, there still is a problem.  I'm finally able to reproduce it, so now I can hopefully get to the bottom of it.
Comment 20 Walter Schwarzenfeld freebsd_triage 2016-08-24 23:11:05 UTC
You are right, it does not, last try DOC were off. here the build-log.
Comment 21 Walter Schwarzenfeld freebsd_triage 2016-08-24 23:12:01 UTC
Created attachment 174033 [details]
slurm-typescript_build.log
Comment 22 Joseph Mingrone freebsd_committer 2016-08-24 23:24:42 UTC
Thanks.  I can reproduce it even with all options off (including DOCS).  Here is a more direct route:

cd /usr/ports/sysutils/slurm-wlm
make clean
make configure
cd work/slurm-16.05.4/doc/man/man1
gmake

Instead of running gmake in the last step, running this command directly will trigger the hang:
../man2html.py 16.05 ../../html/header.txt ../../html/footer.txt sacct.1

So it's getting clearer that the problem is directly in the script.  Now I just have to learn a little python to figure out what's up.
Comment 23 Joseph Mingrone freebsd_committer 2016-08-25 00:07:12 UTC
Created attachment 174035 [details]
svn diff to fix hanging build and to update to version 16.05.4

The man2html.py script was calling man2html like

man2html sacct.1> sacct.mhtml

but man2html expects input from standard input, thus it was waiting on input.

The latest patch fixes this issue by changing the call to

man2html < sacct.1> sacct.mhtml

and it also updates the port to slurm version 16.05.4.
Comment 24 Walter Schwarzenfeld freebsd_triage 2016-08-25 00:25:13 UTC
YES ! Now it works, thank you much.
Comment 25 Kurt Jaeger freebsd_committer 2016-08-25 02:48:56 UTC
testbuilds@work
Comment 26 commit-hook freebsd_committer 2016-08-25 03:20:45 UTC
A commit references this bug:

Author: pi
Date: Thu Aug 25 03:20:24 UTC 2016
New revision: 420833
URL: https://svnweb.freebsd.org/changeset/ports/420833

Log:
  sysutils/slurm-wlm: 15.06.12 -> 16.05.4 and fix build hangs

  PR:		210648
  Changes:	https://github.com/SchedMD/slurm/blob/master/NEWS
  Reported by:	w.schwarzenfeld@utanet.at
  Submitted by:	Joseph Mingrone <jrm@ftfl.ca> (maintainer)

Changes:
  head/sysutils/slurm-wlm/Makefile
  head/sysutils/slurm-wlm/distinfo
  head/sysutils/slurm-wlm/files/patch-doc_man_man2html.py
  head/sysutils/slurm-wlm/files/patch-src_plugins_node__features_knl__cray_node__features__knl__cray.c
  head/sysutils/slurm-wlm/pkg-plist
Comment 27 Kurt Jaeger freebsd_committer 2016-08-25 03:21:06 UTC
Committed, thanks!