We are using VirtualBox for a long time as headless on few machines. Some times ago it started crashing guest running 1 - 2 weeks. No longer uptime possible.
Hosts and guest are running FreeBSD 11.3 amd64. (crashes were seen on 11.2 too before upgrade), one guest running ubuntu-mini (crashes too)
Guest are running different services. One is busy serving Zabbix 4.0 server (crashing more often), one is almost idle doing just IPSec gateway for other machines in LAN.
Today 3 of 4 guests crashed almost at the same time:
on host 1 (only 1 guest running):
Sep 23 09:52:01 XXX kernel: pid 783 (VBoxHeadless), jid 0, uid 920: exited on signal 11
on host 2 (3 guest running):
Sep 23 09:50:33 YYY kernel: pid 19569 (VBoxHeadless), jid 0, uid 920: exited on signal 11
Sep 23 09:50:40 YYY kernel: pid 19551 (VBoxHeadless), jid 0, uid 920: exited on signal 11
All guests has 1 CPU configured.
Am I the only one with this kind of problem?
What is the right way to debug this issue?
Thank you for the report. Could you please provide more information, including:
- Exact FreeBSD host/guest versions (uname -a)
- pkg version -v output on host (as an attachment)
- Information about changes made to the host/guests about the time the crashes started. Old /var/log/messages.* (look for pkg version updates) might be handy
A backtrace of the core dumps may also help. Virtualbox may need to be built with debug symbols.
Created attachment 207766 [details]
pkg version -v on host XXX
Created attachment 207767 [details]
pkg version -v on host YYY
Created attachment 207768 [details]
pkg changes 2019, host XXX
Created attachment 207769 [details]
pkg changes 2019 host YYY
FreeBSD hosts and guests are the same:
FreeBSD aaa.bbb.ccc 11.3-RELEASE-p3 FreeBSD 11.3-RELEASE-p3 #8: Wed Aug 21 01:03:29 CEST 2019 firstname.lastname@example.org:/usr/obj/usr/src/sys/GENERIC amd64
Linux docker-mq 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Both hosts are running VirtualBox for more than 1 year.
Host XXX (running just 1 VM with Zabbix server) started crashing last year
Jul 22 13:25:42 XXX kernel: pid 33223 (VBoxHeadless), uid 920: exited on signal 11
Oct 21 15:15:49 XXX kernel: pid 16858 (VBoxHeadless), uid 920: exited on signal 11
Nov 30 13:39:15 XXX kernel: pid 46647 (VBoxHeadless), uid 920: exited on signal 11
Dec 12 14:57:36 XXX kernel: pid 56079 (VBoxHeadless), uid 920: exited on signal 11
The other host YYY running 3 guests (FreeBSD Ipsec GW + Linux docker) crashed this April 1st.
Apr 1 10:13:50 neon kernel: pid 34673 (VBoxHeadless), uid 920: exited on signal 11
Apr 19 10:37:03 neon kernel: pid 34693 (VBoxHeadless), uid 920: exited on signal 11
May 27 04:08:25 neon kernel: pid 36780 (VBoxHeadless), uid 920: exited on signal 11
Extracted from old messages. Both machines were running 11.2, upgraded to 11.3 about 1 month ago.
I have separate log file for pkg: /var/log/pkg.log
There are no core dumps :(
Both machines will be updated in a few days. pkg version is attached too.
I don't have any debug date, but it still crashes on virtualbox-ose-kmod-5.2.32_1
Strange enough it again crashed 2 VM guests on 2 different machines at the same time.
Host XXX with Zabbix in VM guest
Oct 6 11:57:39 XXX kernel: pid 40180 (VBoxHeadless), jid 0, uid 920: exited on signal 11
Host YYY with 3 VM guest, crashed was guest with Linux ubuntu-mini
Oct 6 11:57:39 neon kernel: pid 52545 (VBoxHeadless), jid 0, uid 920: exited on signal 11
I thing there must be something time dependent or uptime dependent.
Both VM guests were started at the same time = 2019-09-23 09:55 and crashed 2019-10-06 11:57:39