Bug 260319 - java/openjdk11: various segfaults with llvm13
Summary: java/openjdk11: various segfaults with llvm13
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-java (Nobody)
URL: https://bugs.java.com/bugdatabase/vie...
Keywords:
: 260702 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-12-10 15:01 UTC by Mikael Urankar
Modified: 2024-08-16 18:51 UTC (History)
12 users (show)

See Also:
bugzilla: maintainer-feedback? (java)


Attachments
aarch64 14.0-CURRENT poudriere build log (90.04 KB, text/plain)
2021-12-22 09:22 UTC, Dave Cottlehuber
no flags Details
openjfx14 amd64 build (260.89 KB, text/plain)
2021-12-29 06:23 UTC, Jonathan Chen
no flags Details
eclipse amd64 build (73.65 KB, text/plain)
2021-12-29 06:24 UTC, Jonathan Chen
no flags Details
PyCharm crash report (83.65 KB, text/plain)
2022-01-10 08:04 UTC, Ruslan Makhmatkhanov
no flags Details
Makefile.local for llvm12 (129 bytes, text/plain)
2022-02-08 16:19 UTC, Steve Wills
no flags Details
Remove custom LLVM - build with LLVM from base (1.89 KB, patch)
2024-06-24 13:18 UTC, Vladimir Druzenko
vvd: maintainer-approval?
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mikael Urankar freebsd_committer freebsd_triage 2021-12-10 15:01:04 UTC
I'm seeing some segfaults on openjfx14 and other ports on -current amd64 and aarch64 (it's the first failure I ever see on this port)

http://beefy18.nyi.freebsd.org/data/main-amd64-default/p379ef98b2210_sa6765c4b44/logs/openjfx14-14.0.2.1+1_4.log

#  SIGSEGV (0xb) at pc=0x000039d0f7ebf036, pid=63533, tid=810729
#
# JRE version: OpenJDK Runtime Environment (11.0.13+8) (build 11.0.13+8-1)
# Java VM: OpenJDK 64-Bit Server VM (11.0.13+8-1, mixed mode, tiered, compressed oops, g1 gc, bsd-amd64)
# Problematic frame:
# V  [libjvm.so+0xebf036]  JVM_RaiseSignal+0x3d15c6


on aarch64, java/apache-commons-httpclient:
#  Internal Error (assembler_aarch64.hpp:237), pid=62432, tid=0x0000000000032de4
#  guarantee(val < (1U << nbits)) failed: Field too big for insn
#
# JRE version:  (8.0_312-b07) (build )
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode bsd-aarch64 compressed oops)

This affects openjdk8 and openjdk11.

I noticed that the patch in bug #258954 was removed, was it intentional? The crash goes away if I compile openjdk with llvm12
Comment 1 Jonathan Chen 2021-12-20 18:50:53 UTC
This also happens on stable/12, with its recent update to clang-13
Comment 2 Dave Cottlehuber freebsd_committer freebsd_triage 2021-12-22 09:21:04 UTC
this is supposedly addressed upstream, see https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8247766 but I can't navigate to a specific patch from this.

Versions (Unresolved/Resolved/Fixed)

JDK 11 	- 11.0.11 Fixed
JDK 8 	- 8u291 Fixed
OTHER   - openjdk8u332 Fixed

I only see this on 14.0-CURRENT aarch64, not on 13.0-RELEASE.
Comment 3 Dave Cottlehuber freebsd_committer freebsd_triage 2021-12-22 09:22:10 UTC
Created attachment 230307 [details]
aarch64 14.0-CURRENT poudriere build log
Comment 4 Jonathan Chen 2021-12-22 21:16:20 UTC
Oops. I meant to say that it openjdk11 is current failing on stable/13, with its latest update to clang-13. This is on amd64
Comment 5 Dan Kotowski 2021-12-26 20:34:39 UTC
*** Bug 260702 has been marked as a duplicate of this bug. ***
Comment 6 Greg Lewis freebsd_committer freebsd_triage 2021-12-29 04:47:02 UTC
(In reply to Jonathan Chen from comment #4)

Jon, do you have a trace for the amd64 failure?  The arm64 trace refers to a specific arm header (assembler_aarch64.hpp) and I'm wondering if the amd64 is similar or not?
Comment 7 Jonathan Chen 2021-12-29 06:23:53 UTC
Created attachment 230520 [details]
openjfx14 amd64 build
Comment 8 Jonathan Chen 2021-12-29 06:24:30 UTC
Created attachment 230521 [details]
eclipse amd64 build
Comment 9 Jonathan Chen 2021-12-29 06:25:17 UTC
(In reply to Greg Lewis from comment #6)
Greg, I've attached the hs_err_pid*.logs for amd64 builds of openjfx14 and eclipse.
Comment 10 Greg Lewis freebsd_committer freebsd_triage 2021-12-29 07:13:17 UTC
Thanks Jon.  I'll take a look over those.

I tried to reproduce this on my local machine which is running 12.3 by first building openjdk11 with llvm13 and then building openjfx14, but that appeared to work without any problems.  This makes me think there are more interactions going on here with other system libraries being built with llvm13 too.
Comment 11 Ruslan Makhmatkhanov freebsd_committer freebsd_triage 2022-01-10 08:03:56 UTC
The same here on 14-current/amd64 with openjdk11 and various versions of PyCharm, both pro and -ce.
Comment 12 Ruslan Makhmatkhanov freebsd_committer freebsd_triage 2022-01-10 08:04:50 UTC
Created attachment 230860 [details]
PyCharm crash report
Comment 13 Ruslan Makhmatkhanov freebsd_committer freebsd_triage 2022-01-10 08:06:50 UTC
And yes, I wasn't able to build openjfx14 too. And is also missed in pkg repo for amd64. So I just used the package, that was built in my poudriere earlier in November.
Comment 14 Steve Wills freebsd_committer freebsd_triage 2022-02-08 14:46:37 UTC
I'm seeing this too. Any suggestions for work around?
Comment 15 Ruslan Makhmatkhanov freebsd_committer freebsd_triage 2022-02-08 15:18:11 UTC
(In reply to Steve Wills from comment #14)

I just rebuilt openjdk with llvm12 in poudriere and installed it manually.
Comment 16 Steve Wills freebsd_committer freebsd_triage 2022-02-08 15:40:12 UTC
(In reply to Ruslan Makhmatkhanov from comment #15)
Interesting. Did you modify the port to do that or make.conf?
Comment 17 Steve Wills freebsd_committer freebsd_triage 2022-02-08 16:19:39 UTC
Created attachment 231646 [details]
Makefile.local for llvm12

FWIW, here's a Makefile.local that I used to get this building with llvm12. With this, I don't see the issue any more. I'm using 14-CURRENT and openjdk11-11.0.14+9.1
Comment 18 Steve Wills freebsd_committer freebsd_triage 2022-02-08 16:24:20 UTC
(In reply to Steve Wills from comment #17)
Forgot to mention, I'm using amd64 for this.
Comment 19 Mikael Urankar freebsd_committer freebsd_triage 2022-02-08 18:15:17 UTC
In case it's not clear: it's bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258954
Comment 20 Steve Wills freebsd_committer freebsd_triage 2022-02-08 18:44:27 UTC
(In reply to Mikael Urankar from comment #19)

This commit:

https://cgit.freebsd.org/ports/commit/?id=0877cf18c3a0f1caa2a6eef7a4130465cf6ae125

on 2021-10-31 removed the workaround added in this commit:

https://cgit.freebsd.org/ports/commit/?id=3822416493cfbbed8fe7a487391b40bec956d671

on 2021-10-16. It needs to be added back.
Comment 21 Steve Wills freebsd_committer freebsd_triage 2022-02-08 22:23:37 UTC
I think these failures are related:

https://portsfallout.com/fallout?port=java%2Fopenjfx14&maintainer=&env=&category=&flavor=
Comment 22 commit-hook freebsd_committer freebsd_triage 2022-02-09 04:45:49 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=846ff4e9529104b008946f89b9fb3d3a27c13773

commit 846ff4e9529104b008946f89b9fb3d3a27c13773
Author:     Greg Lewis <glewis@FreeBSD.org>
AuthorDate: 2022-02-09 04:42:45 +0000
Commit:     Greg Lewis <glewis@FreeBSD.org>
CommitDate: 2022-02-09 04:45:06 +0000

    java/openjdk11: Restore LLVM/Clang 13 workaround

    This restores the previous LLVM/Clang 13 workaround.  I had thought
    the upstream changes would address this, but still seeing reports of
    crashes when compiled with LLVM/Clang 13.
    PR:             260319
    Reported by:    various

 java/openjdk11/Makefile | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
Comment 23 Steve Wills freebsd_committer freebsd_triage 2022-02-09 13:33:21 UTC
(In reply to commit-hook from comment #22)
Thanks.

The upstream change that fixes it seems to be:

https://github.com/openjdk/jdk/commit/ae5615c6142a4dc0d9033462f4880d7b3c127e26

and seems to be present in openjdk14 and later but not openjdk13 or earlier. But my ports tree is slightly out of date, please verify.

Assuming that's correct, the work around needs to be restored for java/openjdk8, java/openjdk12 and java/openjdk13 as well.
Comment 24 Jonathan Chen 2022-02-09 18:37:40 UTC
(In reply to commit-hook from comment #22)

Does the port not require a PORTREVISION bump as well? There was a window of time where 11.0.14+9.1 could have been compiled with clang13 resulting in a dodgy binary.
Comment 25 commit-hook freebsd_committer freebsd_triage 2022-02-12 04:31:10 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=399e3772d37e1ca597304d7e6784d4ee410ba869

commit 399e3772d37e1ca597304d7e6784d4ee410ba869
Author:     Greg Lewis <glewis@FreeBSD.org>
AuthorDate: 2022-02-12 04:28:15 +0000
Commit:     Greg Lewis <glewis@FreeBSD.org>
CommitDate: 2022-02-12 04:28:15 +0000

    java/openjdk11: Bump PORTREVISION

    Bump PORTREVISION, missed in the last commit.

    PR:             260319
    Reported by:    Jonathan Chen

 java/openjdk11/Makefile | 1 +
 1 file changed, 1 insertion(+)
Comment 26 Greg Lewis freebsd_committer freebsd_triage 2022-02-12 04:34:21 UTC
I've just committed an update for openjdk8 to 8u322.  It also contains some fixes which I think should resolve issues with LLVM/Clang 13.

That does leave openjdk12 and openjdk13 per Steve's previous comment.  I'll look at those next although I think they are lower priority
Comment 27 Piotr Kubaj freebsd_committer freebsd_triage 2022-03-27 19:56:27 UTC
I'm not sure whether it's the same issue, but I'm seeing this on powerpc64 and powerpc64le during building java/openjdk13 after upgrade to 13.1:
Optimizing the exploded image
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x0000000802eacc90, pid=12838, tid=425504
#
# JRE version:  (13.0.10+5) (build )
# Java VM: OpenJDK 64-Bit Server VM (13.0.10+5-1, mixed mode, tiered, compressed oops, serial gc, bsd-ppc64le)
# Problematic frame:
# V  [libjvm.so+0xeacc90]  JVM_RaiseSignal+0x465990
Comment 28 Steve Wills freebsd_committer freebsd_triage 2022-04-04 12:50:16 UTC
Does this issue exist with llvm14?
Comment 29 Dimitry Andric freebsd_committer freebsd_triage 2022-04-04 13:21:37 UTC
(In reply to Steve Wills from comment #28)
Yes, as mentioned in bug 258954, OpenJDK 11 through 13 use undefined behavior, and this will trigger segfaults with llvm >= 13.

This has been fixed in OpenJDK >= 14, but the diffs are very difficult to backport to older OpenJDK versions (and upstream did not do it either, as far as I know). Therefore those older versions must be compiled with devel/llvm12.
Comment 30 Dimitry Andric freebsd_committer freebsd_triage 2022-04-04 13:23:11 UTC
(In reply to Steve Wills from comment #28)
Note that bug 262845 fixes a completely different issue with clang >= 14: a new -Werror warning being triggered. No segfaults in there. :)
Comment 31 Steve Wills freebsd_committer freebsd_triage 2022-04-04 14:30:48 UTC
(In reply to Dimitry Andric from comment #29)
Thanks for the info!
Comment 32 Jonathan Chen 2022-08-13 20:56:53 UTC
I am seeing segfaults when using openjdk11 packages built since the Aug 6 commit of 591a784f324b7d8c45596d758b4c0893626bdbef :
   java/openjdk{8,11}: Remove dependency on devel/llvm12 which is no longer necessary

This is on a recent copy of 13-stable, stable/13-n252080-3ea8c7ad90f: Wed Aug 10 12:24:18 NZST 2022, with clang-14. I also had to apply the patch from PR 265687 to get openjdk11 to build.

Currently, when building java/eclipse, I get a segfault with:
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x000015597d8b9178, pid=32238, tid=119308
#
# JRE version: OpenJDK Runtime Environment (11.0.16+8) (build 11.0.16+8-1)
# Java VM: OpenJDK 64-Bit Server VM (11.0.16+8-1, mixed mode, tiered, compressed oops, g1 gc, bsd-amd64)
# Problematic frame:
# V  [libjvm.so+0xeb9178]  JVM_RaiseSignal+0x3ca628
#
# Core dump will be written. Default location: /construction/xports/java/eclipse/work/eclipse.platform.releng.aggregator-R4_24/java.core
#
# An error report file with more information is saved as:
# /construction/xports/java/eclipse/work/eclipse.platform.releng.aggregator-R4_24/hs_err_pid32238.log
Compiled method (c1)  322509 23471   !   3       org.eclipse.core.internal.jobs.JobManager::startJob (187 bytes)
 total in heap  [0x00001559939a3e10,0x00001559939a5ea8] = 8344
 relocation     [0x00001559939a3f88,0x00001559939a4120] = 408
 main code      [0x00001559939a4120,0x00001559939a55e0] = 5312
 stub code      [0x00001559939a55e0,0x00001559939a56b0] = 208
 oops           [0x00001559939a56b0,0x00001559939a56c0] = 16
 metadata       [0x00001559939a56c0,0x00001559939a5738] = 120
 scopes data    [0x00001559939a5738,0x00001559939a5ae0] = 936
 scopes pcs     [0x00001559939a5ae0,0x00001559939a5d80] = 672
 dependencies   [0x00001559939a5d80,0x00001559939a5d90] = 16
 handler table  [0x00001559939a5d90,0x00001559939a5e68] = 216
 nul chk table  [0x00001559939a5e68,0x00001559939a5ea8] = 64
#
# If you would like to submit a bug report, please visit:
#   https://bugs.freebsd.org/bugzilla/enter_bug.cgi?product=Ports%20%26%20Packages&component=Individual%20Port(s)&short_desc=java/o
#
*** Signal 6
Comment 33 Dan Kotowski 2022-08-20 15:25:36 UTC
* Toolchain:      clang (clang/LLVM)
* C Compiler:     Version 14.0.5 (at /usr/bin/cc)
* C++ Compiler:   Version 14.0.5 (at /usr/bin/c++)

For some reason it's trying to use the hostcc
Comment 34 Dan Kotowski 2022-08-21 12:08:51 UTC
Confirmed that removing `&& ${ARCH:Mpowerpc64*}` brings llvm12 fully back into the mix and openjdk11 can build again



diff --git a/java/openjdk11/Makefile b/java/openjdk11/Makefile
index 4ca7c9933447..3e91d1ea37a2 100644
--- a/java/openjdk11/Makefile
+++ b/java/openjdk11/Makefile
@@ -141,7 +141,7 @@ CONFIGURE_ARGS+=    --with-extra-ldflags="-Wl,-rpath=${LOCALBASE}/lib/gcc${GCC_DEFA
                        --with-extra-cxxflags="-Wl,-rpath=${LOCALBASE}/lib/gcc${GCC_DEFAULT}"
 .else
 MAKE_ENV+=     USE_CLANG=true
-.if ${COMPILER_VERSION} >= 130 && ${ARCH:Mpowerpc64*}
+.if ${COMPILER_VERSION} >= 130# && ${ARCH:Mpowerpc64*}
 LLVM_VER=      12
 BUILD_DEPENDS+=        clang${LLVM_VER}:devel/llvm${LLVM_VER}
 CC=            ${LOCALBASE}/bin/clang${LLVM_VER}
Comment 35 Dimitry Andric freebsd_committer freebsd_triage 2022-08-21 12:16:43 UTC
(In reply to Dan Kotowski from comment #34)
Since nobody wants to spend the time to find the actual bug, please commit this un-fix, so that this ticket can be closed. I'm giving up on the old openjdks, they're unsalvageable.
Comment 36 Jonathan Chen 2022-08-30 20:11:42 UTC
Can a committer please update the port to use llvm12 unconditionally for the build? The package builders are currently failing to build java/eclipse due to the a seg-fault.
Comment 37 commit-hook freebsd_committer freebsd_triage 2022-08-30 23:03:08 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=e3f56d3c663dd21857843e187524d601b7dca29b

commit e3f56d3c663dd21857843e187524d601b7dca29b
Author:     Piotr Kubaj <pkubaj@FreeBSD.org>
AuthorDate: 2022-08-30 23:00:18 +0000
Commit:     Piotr Kubaj <pkubaj@FreeBSD.org>
CommitDate: 2022-08-30 23:02:10 +0000

    java/openjdk11: use LLVM 12 to build if the system compiler is LLVM 13 or newer

    PR:     260319
    Reported by:    jonc@chen.org.nz
    Approved by:    java (maintainer timeout)

 java/openjdk11/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Comment 38 Dan Kotowski 2022-08-31 10:48:38 UTC
Thank you for getting this port at least temporarily building again!

As a non-dev sysadmin, keeping the port at least building is critical for me, but I also would like to contribute what I can to helping fix the actual underlying problem as well.

@Greg Lewis - is it better to track that upstream work here or in your GH repo? Fwiw all of my errors across many months are `use of bitwise '|' with boolean operands [-Werror,-Wbitwise-instead-of-logical]` and all in `libharfbuzz/hb-ot-layout-gpos-table.hh`
Comment 39 Vladimir Druzenko freebsd_committer freebsd_triage 2024-06-24 13:18:26 UTC
Created attachment 251672 [details]
Remove custom LLVM - build with LLVM from base

14.1 amd64.
Just tested build java/openjdk11 with this patch and then java/eclipse and java/openjfx14 using java/openjdk11: build without errors.
Comment 40 commit-hook freebsd_committer freebsd_triage 2024-07-14 14:01:26 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=08cc50220f1d71f12d13d7e2c957a5679b02914d

commit 08cc50220f1d71f12d13d7e2c957a5679b02914d
Author:     Vladimir Druzenko <vvd@FreeBSD.org>
AuthorDate: 2024-07-14 13:44:52 +0000
Commit:     Vladimir Druzenko <vvd@FreeBSD.org>
CommitDate: 2024-07-14 13:44:52 +0000

    java/openjdk11: remove custom LLVM - build with LLVM from base

    Segfaults are no longer observed with current versions of LLVM 17 and 18
    in base r13.3 and 14.1, so there is no need to pull such an old version
    from ports to build.
    For example build java/openjdk11 with LLVM from base and then build
    java/eclipse and java/openjfx14 using java/openjdk11: build and run
    without errors.

    PR:             280056 260319
    Approved by:    java (maintainer, timeout 14 days)

 java/openjdk11/Makefile                              |  6 +-----
 .../patch-make_hotspot_lib_JvmMapfile.gmk (new)      | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+), 5 deletions(-)
Comment 41 commit-hook freebsd_committer freebsd_triage 2024-08-16 18:51:02 UTC
A commit in branch 2024Q3 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=5695d8d0e2aae63368d435790269d310d93d62a7

commit 5695d8d0e2aae63368d435790269d310d93d62a7
Author:     Vladimir Druzenko <vvd@FreeBSD.org>
AuthorDate: 2024-07-14 13:44:52 +0000
Commit:     Vladimir Druzenko <vvd@FreeBSD.org>
CommitDate: 2024-08-16 18:50:04 +0000

    java/openjdk11: remove custom LLVM - build with LLVM from base

    Segfaults are no longer observed with current versions of LLVM 17 and 18
    in base r13.3 and 14.1, so there is no need to pull such an old version
    from ports to build.
    For example build java/openjdk11 with LLVM from base and then build
    java/eclipse and java/openjfx14 using java/openjdk11: build and run
    without errors.

    PR:             280056 260319
    Approved by:    java (maintainer, timeout 14 days)

    (cherry picked from commit 08cc50220f1d71f12d13d7e2c957a5679b02914d)

 java/openjdk11/Makefile                              |  6 +-----
 .../patch-make_hotspot_lib_JvmMapfile.gmk (new)      | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+), 5 deletions(-)