Bug 261198 - bhyve host panics with: spin lock 0xffffffff81eac800 (callout) helpanic: spin lock held too long
Summary: bhyve host panics with: spin lock 0xffffffff81eac800 (callout) helpanic: spin...
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: bhyve (show other bugs)
Version: CURRENT
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-virtualization (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-01-14 14:08 UTC by Tom Jones
Modified: 2022-01-16 10:38 UTC (History)
1 user (show)

See Also:


Attachments
Panic while rebooting guest (307.40 KB, image/jpeg)
2022-01-14 14:08 UTC, Tom Jones
no flags Details
Panic while running disk test (333.96 KB, image/jpeg)
2022-01-14 14:09 UTC, Tom Jones
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tom Jones freebsd_committer 2022-01-14 14:08:31 UTC
Created attachment 231002 [details]
Panic while rebooting guest

My FreeBSD bhyve host running 1 FreeBSD guest (both 14.0-CURRENT) is sporadically panicing with spinlock held too long. I have attached photos of the two panic messages I have seen.

The first panic happened when the guest was rebooting, the second when I was running a disk perf measurement in the vm. 

I will set up better remote access when I am a the machine tomorrow so I can interact with it once it has paniced. 

host: FreeBSD pokeitwithastick 14.0-CURRENT FreeBSD 14.0-CURRENT #8 main-n252356-01cad731921: Tue Jan 11 10:13:27 UTC 2022     tj@pokeitwithastick:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64
Comment 1 Tom Jones freebsd_committer 2022-01-14 14:09:06 UTC
Created attachment 231003 [details]
Panic while running disk test
Comment 2 Tom Jones freebsd_committer 2022-01-14 14:11:04 UTC
Host is a AMD Ryzen 7 3700X 8-Core Processor, full dmesg is here:

https://dmesgd.nycbug.org/index.cgi?do=view&id=6354
Comment 3 Vitaliy Gusev 2022-01-14 20:07:43 UTC
Tom, could you find guilty commit from the list below (manually or by usage 'git bisect') ? How regularly did you run your tests last two-thee months?


 $ git l sys/amd64/vmm/
 6171e026be11 bhyve: add support for MTRR
 c72e914cf108 vmm: vlapic resume can eat 100% CPU by vlapic_callout_handler
 e2650af157bc Make CPU_SET macros compliant with other implementations
 4c812fe61b7c vlapic: Schedule callouts on the local CPU
 de8554295b47 cpuset(9): Add CPU_FOREACH_IS(SET|CLR) and modify consumers to use it
 bcdc599dc2a1 Revert "cpuset(9): Add CPU_FOREACH_IS(SET|CLR) and modify consumers to use it"
 9068f6ea697b cpuset(9): Add CPU_FOREACH_IS(SET|CLR) and modify consumers to use it
Comment 4 Tom Jones freebsd_committer 2022-01-16 10:38:08 UTC
Yeah can do, but not until wednesday the com cable I have was 2mm and the headers on the motherboard are 2.54mm.