Bug 266229 - java/openjdk11: SIGSEGV with Elasticsearch and Logstash
Summary: java/openjdk11: SIGSEGV with Elasticsearch and Logstash
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-ports-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-09-05 06:34 UTC by Arno
Modified: 2023-07-04 04:54 UTC (History)
5 users (show)

See Also:
fernape: maintainer-feedback? (a.zandink)


Attachments
Logstash dumped info (170.55 KB, text/plain)
2022-09-05 06:34 UTC, Arno
no flags Details
Elasticsearch dumped info (81.98 KB, text/plain)
2022-09-05 08:52 UTC, Arno
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Arno 2022-09-05 06:34:31 UTC
Created attachment 236364 [details]
Logstash dumped info

We are experiencing problems with logstash after upgrading from FreeBSD 13.0 to 13.1.

The logstash service is stopping randomly, I have attached the dumped log information to this ticket.


---------------  S U M M A R Y ------------

Command Line: org.logstash.Logstash --path.settings /usr/local/etc/logstash/config -l /var/log/logstash

Host: AMD EPYC 7571 amd64, 4 cores, 15G, FreeBSD 13.1-RELEASE-p2
Time: Mon Sep  5 07:43:05 2022 CEST elapsed time: 1085.369647 seconds (0d 0h 18m 5s)

----------------------------------------------


Please note that this is my first bug report here.

Kind regards,

Arno
Comment 1 Fernando Apesteguía freebsd_committer freebsd_triage 2022-09-05 08:36:09 UTC
What's the port that is generating the problem? The report says "java/o"
Comment 2 Arno 2022-09-05 08:52:42 UTC
Created attachment 236365 [details]
Elasticsearch dumped info

Elasticsearch also crashed, added the dump to this ticket.
Comment 3 Arno 2022-09-05 08:59:19 UTC
(In reply to Fernando Apesteguía from comment #1)

Thanks for the reply, I will give you as much information as I can.

I created this ticket using the link found in the report (log file). I think this is why javq/o is selected. This sounded logical for me as I am getting errors regarding Elasticsearch and Logstash.

Please provide information if I can improve this ticket or if you think that I should create a bug report at elastic.co.

Additional information:
freebsd-version -uk
13.1-RELEASE-p2
13.1-RELEASE-p2

pkg info | grep 'logstash\|kibana\|elasticsearch'
elasticsearch7-7.17.3          Full-text search engine for Java
kibana7-7.17.3                 Browser based analytics and search interface to ElasticSearch
logstash7-7.17.3               Server-side data ingestion and transformation tool
Comment 4 Fernando Apesteguía freebsd_committer freebsd_triage 2022-09-05 14:23:07 UTC
(In reply to Arno from comment #3)
Which version of openjdk from ports are you using?
Comment 5 Arno 2022-09-06 04:45:21 UTC
(In reply to Fernando Apesteguía from comment #4)

I can see that both 11 and 8 are installed as package:

pkg info | grep openjdk
openjdk11-11.0.16+8.1          Java Development Kit 11
openjdk8-8.342.07.1            Java Development Kit 8

When I look at the running processes I can see that 11 is used:

ps aux | grep openjdk
elasticsearch 45864 278.4 58.5 156104320 9553528  0- I    11:31    1861:49.13 /usr/local/openjdk11/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX
logstash      67257  57.7 10.3   6255912 1681952  -  I    22:56     143:56.65 /usr/local/openjdk11/bin/java -cp /usr/local/logstash/logstash-core/lib/jars/animal-sniffer-annotations-1.14.jar:/usr/loc
Comment 6 MariuszW 2023-07-03 17:59:42 UTC
Hi, 
Maybe problem was connected with problems on llvm13 at all which was reverted to llvm12. 
At the beginning on powerpc on 2022-08-09:
https://cgit.freebsd.org/ports/commit/java/openjdk11/Makefile?id=5a8afa08c5f99fba47509a15c79e2cd2128f48ac

and next on the rest of platforms on 2022-08-30:

https://cgit.freebsd.org/ports/commit/java/openjdk11/Makefile?id=e3f56d3c663dd21857843e187524d601b7dca29b

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260319

and your build is:
" built on Aug  8 2022 09:55:58 by "nobody" with clang FreeBSD Clang 13.0.0"

The second issue that is strange is that 

JAVA_HOME=/usr/local/openjdk8
and java 11 is reported in coredump:

vm_info: OpenJDK 64-Bit Server VM (11.0.16+8-1) for bsd-amd64 JRE (11.0.16+8-1), built on Aug  8 2022 09:55:58 by "nobody" with clang FreeBSD Clang 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
Comment 7 Arno 2023-07-04 04:54:10 UTC
We where able to patch the server and got no further errors. I can confirm that this was resolved later on in the process. I can not confirm that is is related to: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260319.

Closing issue.