From http://thunderx1.nyi.freebsd.org/data/head-arm64-default/p454157_s325785/logs/errors/openjdk8-8.144.1.log: Compiling 9467 files for BUILD_JDK ====>> Killing runaway build after 7200 seconds with no output I tested openjdk build on my SoftIron OverDrive 1000 and the package built successfully. It does have a relatively long period of no output in the "BUILD_JDK" stage.
Isn't it possible to make the build more verbose? 2 hours without any output on physical machine is.. a bit strange.
The individual build steps are taking exceptionally long on the ThunderX official builds though; from my build on the OD1000, on a 12-CURRENT kernel with WITNESS enabled, ## Finished langtools (build time 00:16:51) ## Finished hotspot (build time 00:03:57) ## Finished corba (build time 00:07:23) ## Finished jaxp (build time 00:10:57) ## Finished jaxws (build time 00:12:22) ## Finished jdk (build time 01:27:50) ## Finished demos (build time 00:00:59) ## Finished nashorn (build time 00:04:55) ## Finished images (build time 00:03:35) Excerpted build times, from the aborted official build: ## Finished langtools (build time 01:45:27) ## Finished hotspot (build time 00:50:23) ## Finished corba (build time 00:58:04) ## Finished jaxp (build time 00:49:54) ## Finished jaxws (build time 01:00:41)
I just completed a non-parallel openjdk8 build on a 96-core ThunderX system at packet.net (so it had 1 core building openjdk, and 95 idle cores). The build took about 3.5 hours, with component build times as follows: ## Finished langtools (build time 00:24:52) ## Finished hotspot (build time 00:02:43) ## Finished corba (build time 00:10:58) ## Finished jaxp (build time 00:16:26) ## Finished jaxws (build time 00:18:57) ## Finished jdk (build time 02:07:51) ## Finished demos (build time 00:00:34) ## Finished nashorn (build time 00:07:06) ## Finished images (build time 00:05:08) This is using Packet's arm64 testing image: FreeBSD freebsd-java-test 12.0-CURRENT FreeBSD 12.0-CURRENT #0: Sun Nov 19 00:36:40 UTC 2017 root@packet-build-arm64:/usr/obj/usr/src/arm64.aarch64/sys/PACKET arm64 This image has WITNESS on (as did my OD1000 image).
Even with a 4 hours timeout it fails: http://thunderx1.nyi.freebsd.org/data/110arm64-default/454917/logs/errors/openjdk8-8.152.16.log
I was mistaken: timings I reported above for the od1000 and 96-core ThunderX were with the ports system's default of building on all CPUs. I a spot check of the build and saw only one process running, but presume I must have just been unlucky and happened to look during a stage in the build which was inherently non-parallel. With MAKE_JOBS_NUMBER=2 in /etc/make.conf the build reported the following times: ----- Build times ------- Start 2017-12-01 02:07:47 End 2017-12-01 06:01:19 00:11:04 corba 00:02:39 demos 00:08:57 hotspot 00:05:11 images 00:16:28 jaxp 00:18:36 jaxws 02:18:41 jdk 00:24:47 langtools 00:07:08 nashorn 03:53:32 TOTAL ------------------------- This is on an otherwise unused 96 core ThunderX at packet.net, running a recent FreeBSD 12-CURRENT GENERIC-NODEBUG kernel, with a local SSD.
For comparison, with MAKE_JOBS_NUMBER=1: Start 2017-12-01 12:41:42 End 2017-12-01 17:21:15 00:19:46 corba 00:05:07 demos 00:15:55 hotspot 00:06:04 images 00:16:30 jaxp 00:19:01 jaxws 02:45:02 jdk 00:24:52 langtools 00:07:16 nashorn 04:39:33 TOTAL
Created attachment 188454 [details] patch to increase verbosity of openjdk8 build
Possible solution to increase verbosity of build; from the openJDK readme: >Q: I want to see the output of the commands that make runs, like in the old build. How do I do that? >A: You specify the LOG variable to make. There are several log levels: > warn — Default and very quiet. > info — Shows more progress information than warn. > debug — Echos all command lines and prints all macro calls for compilation definitions. > trace — Echos all $(shell) command lines as well. appending LOG=debug to MAKE_ENV in the openJDK makefile seems to produce more output during the BUILD_JDK stage. see patch.
Created attachment 188455 [details] patch to increase verbosity of openjdk8 build
(In reply to Zak from comment #9) If my memory serves, it won't fix the problem. Basically, all you can see is "javac ..." after "Compiling 9467 files for BUILD_JDK" because just one command is taking too long.
(In reply to Jung-uk Kim from comment #10) I thought LOG=debug produced some output for each of the files in that stage.
(In reply to Ed Maste from comment #11) I just tested the patch and I verified that the exact command you see on amd64 is: (/usr/local/bootstrap-openjdk8/bin/java -Xms64M -Xmx1600M -XX:ThreadStackSize=1536 "-Xbootclasspath/p:/wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/langtools/dist/bootstrap/lib/javac.jar" -cp /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/langtools/dist/bootstrap/lib/javac.jar com.sun.tools.javac.Main -bootclasspath /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/classes -source 8 -target 8 -encoding ascii -XDignore.symbol.file=true -Xlint:-unchecked,-deprecation,-overrides,auxiliaryclass,classfile,dep-ann,divzero,empty,try,varargs -Werror -implicit:none -sourcepath "/wrkdirs/usr/ports/java/openjdk8/work/openjdk/jdk/src/share/classes:/wrkdirs/usr/ports/java/openjdk8/work/openjdk/jdk/src/solaris/classes:/wrkdirs/usr/ports/java/openjdk8/work/openjdk/jdk/src/bsd/classes:/wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/gensrc:/wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/gensrc_no_srczip" -d /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/classes -h /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/gensrc_headers.BUILD_JDK.tmp @/wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/classes/_the.BUILD_JDK_batch.tmp && /bin/mv /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/classes/_the.BUILD_JDK_batch.tmp /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/classes/_the.BUILD_JDK_batch) This one liner compiles 9469 files.
If there are many cores, we can improve compile time with sjavac. I tried it couple of years ago but I had to back it out because some people reported it didn't work for them. See bug 205544. https://svnweb.freebsd.org/changeset/ports/403748 I didn't try harder at the time, though.
Can we try just increasing the build timeout for the time being?
It seems this was fixed a few months ago.