Bug 221877

Summary: devel/libvirt: segmentation fault with bhyve
Product: Ports & Packages Reporter: David Harrigan <dharrigan>
Component: Individual Port(s)Assignee: Roman Bogorodskiy <novel>
Status: Closed FIXED    
Severity: Affects Only Me CC: arved, w.schwarzenfeld
Priority: --- Keywords: needs-patch
Version: Latest   
Hardware: Any   
OS: Any   

Description David Harrigan 2017-08-28 09:25:05 UTC 
Hi,

Getting a repeated segmentation fault with libvirtd. See below

FreeBSD anton 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28 UTC 2017     root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

# pkg info libvirt-3.6.0 
libvirt-3.6.0
Name           : libvirt
Version        : 3.6.0

network configuration:

<network>
  <name>default</name>
  <uuid>3b981a1f-8870-11e7-bd3f-00259069ef52</uuid>
  <forward mode='bridge'/>
  <bridge name='bridge1'/>
</network>

domain configuration:

<domain type="bhyve">
  <name>generic</name>
  <uuid>ad086073-a314-4c99-b3d4-9d1f1d3ec2b3</uuid>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <vcpu>2</vcpu>
  <os>
    <type arch="x86_64">hvm</type>
    <boot dev="cdrom"/>
    <boot dev="hd"/>
  </os>
  <clock offset="utc"/>
  <on_reboot>destroy</on_reboot>
  <devices>
    <emulator>bhyve</emulator>
    <disk type="file" device="disk">
      <source file="/var/lib/libvirt/images/centos7.0.img"/>
      <target dev="hda" bus="ide"/>
    </disk>
    <disk type="file" device="cdrom">
      <source file="/var/lib/libvirt/images/CentOS-7-x86_64-Minimal-1611.iso"/>
      <target dev="hdb" bus="ide"/>
      <readonly/>
    </disk>
    <interface type="network">
      <source network="default"/>
      <mac address="00:16:3e:45:45:49"/>
    </interface>
    <graphics type="vnc" port="-1" keymap="en-gb"/>
    <console type="pty"/>
  </devices>
</domain>

Started as:

# virsh -c bhyve:///system
Welcome to virsh, the virtualisation interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # list --all
 Id    Name                           State
----------------------------------------------------

virsh # define /root/centos7.xml 
Domain generic defined from /root/centos7.xml

virsh # list --all
 Id    Name                           State
----------------------------------------------------
 -     generic                        shut off

virsh # start generic
error: Disconnected from bhyve:///system due to end of file
error: Failed to start domain generic
error: End of file while reading data: Input/output error

virsh # 


Stack below:

2017-08-28 08:28:16.070+0000: 34494046208: debug : virEventPollMakePollFDs:401 : Prepare n=4 w=5, f=11 e=1 d=0
2017-08-28 08:28:16.071+0000: 34494051328: debug : virAccessManagerCheckDomain:234 : manager=0x808020a50(name=stack) driver=bhyve domain=0x808c81000 perm=4
2017-08-28 08:28:16.071+0000: 34494046208: debug : virEventPollMakePollFDs:401 : Prepare n=5 w=6, f=12 e=1 d=0
2017-08-28 08:28:16.071+0000: 34494051328: debug : virAccessManagerCheckDomain:234 : manager=0x808020a20(name=none) driver=bhyve domain=0x808c81000 perm=4
2017-08-28 08:28:16.071+0000: 34494046208: debug : virEventPollCalculateTimeout:338 : Calculate expiry of 2 timers
2017-08-28 08:28:16.071+0000: 34494051328: info : virObjectUnref:327 : OBJECT_UNREF: obj=0x808020a50
2017-08-28 08:28:16.071+0000: 34494046208: debug : virEventPollCalculateTimeout:346 : Got a timeout scheduled for 1503908901070
2017-08-28 08:28:16.071+0000: 34494051328: info : virObjectRef:365 : OBJECT_REF: obj=0x80b140000
2017-08-28 08:28:16.071+0000: 34494046208: debug : virEventPollCalculateTimeout:359 : Schedule timeout then=1503908901070 now=1503908896071
2017-08-28 08:28:16.071+0000: 34494046208: debug : virEventPollCalculateTimeout:369 : Timeout at 1503908901070 due in 4999 ms
2017-08-28 08:28:16.071+0000: 34494046208: info : virEventPollRunOnce:640 : EVENT_POLL_RUN: nhandles=6 timeout=4999
2017-08-28 08:28:16.071+0000: 34494051328: debug : virDomainPCIAddressReserveAddrInternal:609 : PCI bus 0000:00 assigned isolation group 0 because of first device 0000:00:02.0
2017-08-28 08:28:16.071+0000: 34494051328: debug : virDomainPCIAddressReserveAddrInternal:626 : Reserving PCI address 0000:00:02.0 (aggregate='false')
2017-08-28 08:28:16.071+0000: 34494051328: debug : virDomainPCIAddressReserveAddrInternal:626 : Reserving PCI address 0000:00:03.0 (aggregate='false')
2017-08-28 08:28:16.071+0000: 34494051328: debug : virDomainPCIAddressReserveAddrInternal:626 : Reserving PCI address 0000:00:01.0 (aggregate='false')
[1]    31879 segmentation fault (core dumped)  /usr/local/sbin/libvirtd --pid-file=/var/run/libvirtd.pid


Happy to provide more information, just let me know what is required.

-=david=-
Comment 1 Roman Bogorodskiy freebsd_committer freebsd_triage 2018-03-06 17:54:47 UTC 
David, thanks for the detailed bug report, and sorry for the delay, I've been pretty busy recently. I hope to get to it this weekend.

> Happy to provide more information, just let me know what is required.

What's already there looks enough to get started, but if you have some spare time you could get a traceback from lldb. That would require rebuilding libvirt with -DWITH_DEBUG, reinstalling and reproducing this crash again to get a core file.

Once it's there:

lldb /usr/local/sbin/libvirtd -c /path/to/libvirtd.core and then "bt" in lldb prompt.
Comment 2 Roman Bogorodskiy freebsd_committer freebsd_triage 2018-03-06 18:00:00 UTC 
Oh, I've just noticed it's actually from Aug last year and wasn't auto-assigned because of a type in summary...

Anyway, needs validation.
Comment 3 Roman Bogorodskiy freebsd_committer freebsd_triage 2018-03-11 16:07:00 UTC 
Yeah, there's a bug. The bhyve driver expects network model to be specified, otherwise it crashes like in this case.

I've sent a fix: https://www.redhat.com/archives/libvir-list/2018-March/msg00515.html

I'll backport it to the port once it's reviewed and merged.

By the way, interface type 'network' is not support by the bhyve driver, 'bridge' should be used instead. For example:

      <interface type='bridge'>
        <model type='virtio'/>
        <source bridge="bridge1"/>
      </interface>
Comment 4 Roman Bogorodskiy freebsd_committer freebsd_triage 2018-03-11 16:17:48 UTC 
(In reply to Roman Bogorodskiy from comment #3)

BTW, there are some other issues in this sample xml. Working configuration would look something like:

<domain type="bhyve">
  <name>generic</name>
  <uuid>ad086073-a314-4c99-b3d4-9d1f1d3ec2b3</uuid>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <vcpu>2</vcpu>
  <os>
    <type arch="x86_64">hvm</type>
    <boot dev="cdrom"/>
    <boot dev="hd"/>
    <loader readonly="yes" type="pflash">/usr/local/share/uefi-firmware/BHYVE_UEFI.fd</loader>
  </os>
  <clock offset="utc"/>
  <on_reboot>destroy</on_reboot>
  <devices>
    <emulator>bhyve</emulator>
    <disk type="file" device="disk">
      <driver name='file' type='raw'/>
      <source file="/var/lib/libvirt/images/centos7.0.img"/>
      <target dev="hda" bus="sata"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name='file' type='raw'/>
      <source file="/var/lib/libvirt/images/CentOS-7-x86_64-Minimal-1611.iso"/>
      <target dev="hdb" bus="sata"/>
      <readonly/>
    </disk>
    <interface type="bridge">
      <mac address="00:16:3e:45:45:49"/>
      <model type='virtio'/>
      <source bridge="virbr0"/>
    </interface>
    <graphics type="vnc" port="-1" keymap="en-gb"/>
    <controller type='usb' model='nec-xhci'/>
    <input type='tablet' bus='usb'/>
  </devices>
</domain>
Comment 5 commit-hook freebsd_committer freebsd_triage 2018-03-13 14:38:54 UTC 
A commit references this bug:

Author: novel
Date: Tue Mar 13 14:38:12 UTC 2018
New revision: 464382
URL: https://svnweb.freebsd.org/changeset/ports/464382

Log:
  devel/libvirt: fix crash in bhyve driver

  Fix crash on domains with interfaces that have no model specified.

  PR:		221877
  Reported by:	David Harrigan

Changes:
  head/devel/libvirt/Makefile
  head/devel/libvirt/files/patch-src_bhyve_bhyve__command.c
Comment 6 Roman Bogorodskiy freebsd_committer freebsd_triage 2018-03-13 14:41:50 UTC 
Should be fixed now, thanks for report!