Bug 213333
| Summary: | FreeBSD 11-RELEASE fails to boot under KVM/Qemu | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Base System | Reporter: | Lex van Roon <r3boot> | ||||
| Component: | kern | Assignee: | freebsd-virtualization (Nobody) <virtualization> | ||||
| Status: | Closed DUPLICATE | ||||||
| Severity: | Affects Only Me | CC: | 0mp, abeesh, akelling, cam, d9uf01cnk97wfj9var7p, emaste, luciano, mainland, marvin0815, miguelcastellanos, pablo.ruiz, pi, sam, tsuroerusu | ||||
| Priority: | --- | ||||||
| Version: | 11.0-STABLE | ||||||
| Hardware: | amd64 | ||||||
| OS: | Any | ||||||
| See Also: | https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213155 | ||||||
| Attachments: |
|
||||||
I am also seeing this issue with KVM on OpenSUSE 42.1. I have also tried the same troubleshooting steps. My system information. Repository: openSUSE-Leap-42.1-Update Name: qemu-kvm Version: 2.3.1-19.3 Arch: x86_64 Vendor: openSUSE sharp@TwoRivers:~> uname -a Linux TwoRivers 4.1.34-33-default #1 SMP PREEMPT Thu Oct 20 08:03:29 UTC 2016 (fe18aba) x86_64 x86_64 x86_64 GNU/Linux sharp@TwoRivers:~> cat /proc/cpuinfo [192/948] processor : 0 vendor_id : AuthenticAMD cpu family : 21 model : 1 model name : AMD FX(tm)-8120 Eight-Core Processor stepping : 2 microcode : 0x6000629 cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 nodeid_msr topoext perfctr_core perfctr_nb ar at cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold vmmcall bugs : fxsave_leak sysret_ss_attrs bogomips : 6241.87 TLB size : 1536 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate cpb Found a work around. Seems to be an issue with copying the the host CPU topology and with the Opteron_G4 CPU arch too. Switching to Opteron_G3 or selecting Hypervisor_Default seems to mitigate this issue. Should note that this corrected my issue upgrading my servers from 10.3 to 11. I was also seeing issue with the install cd locking up too. I can confirm that by switching the hypervisor to Opteron_G3 I've got a working FreeBSD 11. I can confirm the EXACT same issue here. My server has dual Opteron 6348 CPU's. FreeBSD 10.3 and earlier all work correctly. My other system with a Xeon D-1541 has no such problem. To clarify my comment, using Opteron_G5 or Opteron_G4 causes boot to hang with FreeBSD 11.0-p6. Using Opteron_G3 works. I confirm the same problem with Opteron 6272 Processors, under Centos 7.2/KVM Hypervisor. Can anyone try booting FreeBSD 11.0 directly on the hardware? Unfortunately, I'm unable to. This may not be KVM related. I also tried 11-STABLE around a week back, the issue is still present there. (In reply to Cameron from comment #8) I tried on Intel I3 Vaio and on an dual Intel(R) Xeon(R) CPU E5410 @ 2.33GHz and it doesn't boot. I guess it may be a bios issue linked to UEFI beeing advertized but not properly implemented (Just a guess). 10.3-RELEASE boots flawlessly on the same hardware. Hope this helps, Luciano. I tried directly on the hardware as requested. FreeBSD-11.RELEASE booted correctly under the physical hardware, but fails to boot as guest on a qemu virtual machine under KVM (Centos 7.2) on the same hardware. I tested on a SuperMicro server with dual AMD Opteron 6272 processors, with a non UEFI BIOS. If you need more details please let me know. Thanks. Miguel I have a non-UEFI BIOS with a Supermicro H8DG6-F motherboard and dual 6348 CPU's. This issue doesn't appear to be UEFI related. I am also experiencing these issues with a non-UEFI based system. Motherboard: Asus M5A99X EVO, CPU: AMD FX-8120 I can also attest to this problem that everybody else is talking about. I was just trying to install FreeBSD 11.1-RELEASE in a VM using KVM on CentOS 7 when I ran into this when the installer refused to boot. The problem occurs both with "Copy host CPU configuration" as well as the "Opteron_G4" and "Opteron_G5" options. Opteron_G3 had it booting without an issue. This is affecting me too. I tried to identify the CPU feature flags that might be preventing boot on "Opteron_G4", by iteratively attempting boot with each single non-G3 feature flag disabled in KVM. Unfortunately none of those attempts worked, so it seems this must be due to a some combination of multiple CPU feature flags. Affecting me too. Started since pfsense 2.4 uses FreeBSD 11 I have the same issue with both 10.4 and 11.1 kernels---I cannot upgrade from 10.3. Setting hw.use_xsave=0 in loader.conf does not help. I've managed to work around this. There's 2 issues here: 1. Rather than using an Opteron_* model in kvm for the CPU, etc. Use kvm64. Others may work as well. I believe this alone will Just Work if you haven't been building from source. 2. Did you build everything/anything from source? You probably don't want to set 'CPUTYPE?=native', etc in /etc/make.conf (or perhaps you can find something "safe" to set it to?). If you are building from source AND had CPUTYPE set to something like Opteron_*, you'll probably need to rebuild your *current* working version before you can attempt to switch the CPU model to 'kvm64'. An Opteron optimized build won't even boot if you set the CPU model to 'kvm64' IIRC. First, unset CPUTYPE if you have it set and then rebuild _everything_. Rebuild world, kernel, and ports. Once you've confirmed that everything still works (you can reboot, your daemons still come up, etc), you should be able to switch your virtual CPU model to kvm64. Once your system runs with with kvm64 set, you can build the newer version of FreeBSD that you want to move to. To be safe, try the newer kernel first (as you should in general anyway) to confirm that it's working without surprises. Seeing as how I found that much more than the kernel and the base system were affected, this could very well be a compiler regression for the Opteron target. Anyway, hope this helps! Same on FreeBSD 11.1 amd64. Switching to kvm64 or Opteron_G3 "fixes" this issue for me. I've attached a patch that fixes this regression in bug #213155. *** This bug has been marked as a duplicate of bug 213155 *** |
Created attachment 175567 [details] Screenshot of failed boot of 11-RELEASE/amd64 on Debian/KVM Hypervisor: - Debian Jessie 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux - Qemu-kvm 1:2.1+dfsg-12+deb8u6 FreeBSD: - 11-RELEASE amd64 bootonly iso While booting a fresh virtual machine using 11-RELEASE/amd64, the kernel hangs right after it's loaded (see attached screenshot). This happens both with ACPI enabled and with ACPI disabled. I tried booting 10.3-RELEASE, which runs perfectly. I also operate multiple 10.[23] VM's on the same hypervisor, which all work. Do note, the spinner spins about 2~3 x before it freezes.