Bug 240775

Summary: emulators/virtualbox-ose random crashes
Product: Ports & Packages Reporter: Miroslav Lachman <000.fbsd>
Component: Individual Port(s)Assignee: vbox (Nobody) <vbox>
Status: Open ---    
Severity: Affects Only Me Keywords: crash, needs-qa
Priority: --- Flags: bugzilla: maintainer-feedback? (vbox)
Version: Latest   
Hardware: Any   
OS: Any   
Description Flags
pkg version -v on host XXX
pkg version -v on host YYY
pkg changes 2019, host XXX
pkg changes 2019 host YYY none

Description Miroslav Lachman 2019-09-23 15:59:04 UTC
We are using VirtualBox for a long time as headless on few machines. Some times ago it started crashing guest running 1 - 2 weeks. No longer uptime possible.
Hosts and guest are running FreeBSD 11.3 amd64. (crashes were seen on 11.2 too before upgrade), one guest running ubuntu-mini (crashes too)

Guest are running different services. One is busy serving Zabbix 4.0 server (crashing more often), one is almost idle doing just IPSec gateway for other machines in LAN.

Today 3 of 4 guests crashed almost at the same time:
on host 1 (only 1 guest running):
Sep 23 09:52:01 XXX kernel: pid 783 (VBoxHeadless), jid 0, uid 920: exited on signal 11

on host 2 (3 guest running):
Sep 23 09:50:33 YYY kernel: pid 19569 (VBoxHeadless), jid 0, uid 920: exited on signal 11
Sep 23 09:50:40 YYY kernel: pid 19551 (VBoxHeadless), jid 0, uid 920: exited on signal 11

All guests has 1 CPU configured.

Am I the only one with this kind of problem?

What is the right way to debug this issue?
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2019-09-24 12:16:26 UTC
Thank you for the report. Could you please provide more information, including:

- Exact FreeBSD host/guest versions (uname -a)
- pkg version -v output on host (as an attachment)
- Information about changes made to the host/guests about the time the crashes started. Old /var/log/messages.* (look for pkg version updates) might be handy

A backtrace of the core dumps may also help. Virtualbox may need to be built with debug symbols.
Comment 2 Miroslav Lachman 2019-09-24 13:25:59 UTC
Created attachment 207766 [details]
pkg version -v on host XXX
Comment 3 Miroslav Lachman 2019-09-24 13:26:23 UTC
Created attachment 207767 [details]
pkg version -v on host YYY
Comment 4 Miroslav Lachman 2019-09-24 13:26:59 UTC
Created attachment 207768 [details]
pkg changes 2019, host XXX
Comment 5 Miroslav Lachman 2019-09-24 13:27:43 UTC
Created attachment 207769 [details]
pkg changes 2019 host YYY
Comment 6 Miroslav Lachman 2019-09-24 13:29:06 UTC
FreeBSD hosts and guests are the same:
FreeBSD aaa.bbb.ccc 11.3-RELEASE-p3 FreeBSD 11.3-RELEASE-p3 #8: Wed Aug 21 01:03:29 CEST 2019     root@bbb.fff.lll:/usr/obj/usr/src/sys/GENERIC  amd64

Linux guest:
Linux docker-mq 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Both hosts are running VirtualBox for more than 1 year.

Host XXX (running just 1 VM with Zabbix server) started crashing last year

Jul 22 13:25:42 XXX kernel: pid 33223 (VBoxHeadless), uid 920: exited on signal 11
Oct 21 15:15:49 XXX kernel: pid 16858 (VBoxHeadless), uid 920: exited on signal 11
Nov 30 13:39:15 XXX kernel: pid 46647 (VBoxHeadless), uid 920: exited on signal 11
Dec 12 14:57:36 XXX kernel: pid 56079 (VBoxHeadless), uid 920: exited on signal 11

The other host YYY running 3 guests (FreeBSD Ipsec GW + Linux docker) crashed this April 1st.

Apr  1 10:13:50 neon kernel: pid 34673 (VBoxHeadless), uid 920: exited on signal 11
Apr 19 10:37:03 neon kernel: pid 34693 (VBoxHeadless), uid 920: exited on signal 11
May 27 04:08:25 neon kernel: pid 36780 (VBoxHeadless), uid 920: exited on signal 11

Extracted from old messages. Both machines were running 11.2, upgraded to 11.3 about 1 month ago.

I have separate log file for pkg: /var/log/pkg.log
Both attached.

There are no core dumps :(

Both machines will be updated in a few days. pkg version is attached too.
Comment 7 Miroslav Lachman 2019-10-06 13:24:10 UTC
I don't have any debug date, but it still crashes on virtualbox-ose-kmod-5.2.32_1
Strange enough it again crashed 2 VM guests on 2 different machines at the same time.

Host XXX with Zabbix in VM guest
Oct  6 11:57:39 XXX kernel: pid 40180 (VBoxHeadless), jid 0, uid 920: exited on signal 11

Host YYY with 3 VM guest, crashed was guest with Linux ubuntu-mini
Oct  6 11:57:39 neon kernel: pid 52545 (VBoxHeadless), jid 0, uid 920: exited on signal 11

I thing there must be something time dependent or uptime dependent.
Both VM guests were started at the same time = 2019-09-23 09:55 and crashed 2019-10-06 11:57:39