Bug 237124

Summary: [xen] Crash or freeze during boot with multiple network interfaces in HVM mode with i386 kernel
Product: Base System Reporter: freebsdbugs
Component: kernAssignee: freebsd-virtualization (Nobody) <virtualization>
Status: New ---    
Severity: Affects Some People    
Priority: ---    
Version: 12.0-RELEASE   
Hardware: i386   
OS: Any   

Description freebsdbugs 2019-04-08 21:28:43 UTC
While attempting to migrate some virtual machines from a Xen 3.4 host to Xen 4.8 host (both Linux), we discovered that some FreeBSD domU vms were either freezing or crashing during boot, once on the Xen 4.8 host.

After a bit of investigation it was determined that the same issue was causing both the crashes and freezes. In each case, the last kernel message seen would be:
xn0: backend features: feature-sg feature-gso-tcp4

After some more investigation, it was determined that booting with only 1 network interface worked fine (the problem VMs have 2 interfaces), and that the problem only seems to exist with the i386 kernels. I tried various versions of 11.0, 11.1, 11.2, and 12.0 RELEASE kernels, the only ones that would work with more than 1 network interface are the amd64 kernels.

So to summarize, to reproduce the bug:
Xen 4.8 host dom0
HVM domU VM with multiple network interfaces
Boot FreeBSD i386 kernel (even just booting from a FreeBSD iso encounters the bug)


Below is the VM's xen config file:
name = "FreeBSD-test"
maxmem = "1024"
memory = "1024"
vcpus = "2"
builder = "hvm"
kernel = "/usr/lib/xen/boot/hvmloader"
device_model_version = "qemu-xen-traditional"
device_model_override = "/usr/lib64/xen/bin/qemu-dm"
xen_platform_pci = "1"
boot = "d"
pae = "1"
acpi = "1"
apic = "1"
localtime = "0"
on_poweroff = "destroy"
on_reboot = "destroy"
on_crash = "destroy"
sdl = "0"
serial = "pty"
vnc = 1
vnclisten = "0.0.0.0"
vncconsole = 0
vncunused = 0
vncdisplay = 1
shadow_memory = "8"
disk = [ "file:/vms/FreeBSD-test.img,hda,w", "file:/iso/FreeBSD-12.0-RELEASE-i386-disc1.iso,hdc,r,devtype=cdrom"   ]
vif = [ "mac=00:16:3e:00:3d:42,bridge=xenbr0,model=e1000", "mac=00:16:3e:00:3e:42,bridge=xenbr1,model=e1000" ]
cpu_weight = "1024"