Bug 223806
| Summary: | java/openjdk8: [PATCH] arm64 package builds do not allow enough time for build to complete | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Ports & Packages | Reporter: | Ed Maste <emaste> | ||||||
| Component: | Ports Framework | Assignee: | Port Management Team <portmgr> | ||||||
| Status: | Closed FIXED | ||||||||
| Severity: | Affects Only Me | CC: | java, jkim, ports-bugs, swills, zsnafzig | ||||||
| Priority: | --- | Keywords: | patch | ||||||
| Version: | Latest | ||||||||
| Hardware: | arm64 | ||||||||
| OS: | Any | ||||||||
| Bug Depends on: | |||||||||
| Bug Blocks: | 203349 | ||||||||
| Attachments: |
|
||||||||
|
Description
Ed Maste
2017-11-22 20:43:52 UTC
Isn't it possible to make the build more verbose? 2 hours without any output on physical machine is.. a bit strange. The individual build steps are taking exceptionally long on the ThunderX official builds though; from my build on the OD1000, on a 12-CURRENT kernel with WITNESS enabled, ## Finished langtools (build time 00:16:51) ## Finished hotspot (build time 00:03:57) ## Finished corba (build time 00:07:23) ## Finished jaxp (build time 00:10:57) ## Finished jaxws (build time 00:12:22) ## Finished jdk (build time 01:27:50) ## Finished demos (build time 00:00:59) ## Finished nashorn (build time 00:04:55) ## Finished images (build time 00:03:35) Excerpted build times, from the aborted official build: ## Finished langtools (build time 01:45:27) ## Finished hotspot (build time 00:50:23) ## Finished corba (build time 00:58:04) ## Finished jaxp (build time 00:49:54) ## Finished jaxws (build time 01:00:41) I just completed a non-parallel openjdk8 build on a 96-core ThunderX system at packet.net (so it had 1 core building openjdk, and 95 idle cores). The build took about 3.5 hours, with component build times as follows: ## Finished langtools (build time 00:24:52) ## Finished hotspot (build time 00:02:43) ## Finished corba (build time 00:10:58) ## Finished jaxp (build time 00:16:26) ## Finished jaxws (build time 00:18:57) ## Finished jdk (build time 02:07:51) ## Finished demos (build time 00:00:34) ## Finished nashorn (build time 00:07:06) ## Finished images (build time 00:05:08) This is using Packet's arm64 testing image: FreeBSD freebsd-java-test 12.0-CURRENT FreeBSD 12.0-CURRENT #0: Sun Nov 19 00:36:40 UTC 2017 root@packet-build-arm64:/usr/obj/usr/src/arm64.aarch64/sys/PACKET arm64 This image has WITNESS on (as did my OD1000 image). Even with a 4 hours timeout it fails: http://thunderx1.nyi.freebsd.org/data/110arm64-default/454917/logs/errors/openjdk8-8.152.16.log I was mistaken: timings I reported above for the od1000 and 96-core ThunderX were with the ports system's default of building on all CPUs. I a spot check of the build and saw only one process running, but presume I must have just been unlucky and happened to look during a stage in the build which was inherently non-parallel. With MAKE_JOBS_NUMBER=2 in /etc/make.conf the build reported the following times: ----- Build times ------- Start 2017-12-01 02:07:47 End 2017-12-01 06:01:19 00:11:04 corba 00:02:39 demos 00:08:57 hotspot 00:05:11 images 00:16:28 jaxp 00:18:36 jaxws 02:18:41 jdk 00:24:47 langtools 00:07:08 nashorn 03:53:32 TOTAL ------------------------- This is on an otherwise unused 96 core ThunderX at packet.net, running a recent FreeBSD 12-CURRENT GENERIC-NODEBUG kernel, with a local SSD. For comparison, with MAKE_JOBS_NUMBER=1: Start 2017-12-01 12:41:42 End 2017-12-01 17:21:15 00:19:46 corba 00:05:07 demos 00:15:55 hotspot 00:06:04 images 00:16:30 jaxp 00:19:01 jaxws 02:45:02 jdk 00:24:52 langtools 00:07:16 nashorn 04:39:33 TOTAL Created attachment 188454 [details]
patch to increase verbosity of openjdk8 build
Possible solution to increase verbosity of build; from the openJDK readme:
>Q: I want to see the output of the commands that make runs, like in the old build. How do I do that?
>A: You specify the LOG variable to make. There are several log levels:
> warn — Default and very quiet.
> info — Shows more progress information than warn.
> debug — Echos all command lines and prints all macro calls for compilation definitions.
> trace — Echos all $(shell) command lines as well.
appending LOG=debug to MAKE_ENV in the openJDK makefile seems to produce more output during the BUILD_JDK stage. see patch.
Created attachment 188455 [details]
patch to increase verbosity of openjdk8 build
(In reply to Zak from comment #9) If my memory serves, it won't fix the problem. Basically, all you can see is "javac ..." after "Compiling 9467 files for BUILD_JDK" because just one command is taking too long. (In reply to Jung-uk Kim from comment #10) I thought LOG=debug produced some output for each of the files in that stage. (In reply to Ed Maste from comment #11) I just tested the patch and I verified that the exact command you see on amd64 is: (/usr/local/bootstrap-openjdk8/bin/java -Xms64M -Xmx1600M -XX:ThreadStackSize=1536 "-Xbootclasspath/p:/wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/langtools/dist/bootstrap/lib/javac.jar" -cp /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/langtools/dist/bootstrap/lib/javac.jar com.sun.tools.javac.Main -bootclasspath /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/classes -source 8 -target 8 -encoding ascii -XDignore.symbol.file=true -Xlint:-unchecked,-deprecation,-overrides,auxiliaryclass,classfile,dep-ann,divzero,empty,try,varargs -Werror -implicit:none -sourcepath "/wrkdirs/usr/ports/java/openjdk8/work/openjdk/jdk/src/share/classes:/wrkdirs/usr/ports/java/openjdk8/work/openjdk/jdk/src/solaris/classes:/wrkdirs/usr/ports/java/openjdk8/work/openjdk/jdk/src/bsd/classes:/wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/gensrc:/wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/gensrc_no_srczip" -d /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/classes -h /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/gensrc_headers.BUILD_JDK.tmp @/wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/classes/_the.BUILD_JDK_batch.tmp && /bin/mv /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/classes/_the.BUILD_JDK_batch.tmp /wrkdirs/usr/ports/java/openjdk8/work/openjdk/build/bsd-x86_64-normal-server-release/jdk/classes/_the.BUILD_JDK_batch) This one liner compiles 9469 files. If there are many cores, we can improve compile time with sjavac. I tried it couple of years ago but I had to back it out because some people reported it didn't work for them. See bug 205544. https://svnweb.freebsd.org/changeset/ports/403748 I didn't try harder at the time, though. Can we try just increasing the build timeout for the time being? It seems this was fixed a few months ago. |