This one has been tough to capture but with the setup of a serial console and the ability to capture the output we see 14.0-CURRENT i386 do something very wrong : Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x56e5ddb5 fault code = supervisor read data, page not present instruction pointer = 0x20:0x1409fa0 stack pointer = 0x28:0xd2d7854 frame pointer = 0x28:0xd2d789c code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = resume, IOPL = 0 current process = 5 (rand_harvestq) trap number = 12 panic: page fault cpuid = 0 time = 1634357727 KDB: stack backtrace: db_trace_self_wrapper(d2d7814,10280000,d2d7814,c,0,...) at db_trace_self_wrapper+0x28/frame 0xd2d76a4 vpanic(1479d53,d2d76e0,d2d76e0,d2d7708,142f377,...) at vpanic+0x124/frame 0xd2d76c0 panic(1479d53,150c115,0,fffff,d2df69b,...) at panic+0x14/frame 0xd2d76d4 trap_fatal(d2d0000,7f,1,8,1511f2a,...) at trap_fatal+0x347/frame 0xd2d7708 trap_pfault(56e5ddb5,0,0) at trap_pfault+0x6f/frame 0xd2d773c trap(d2d7814,8,28,28,d2d798c,...) at trap+0x2d2/frame 0xd2d7808 calltrap() at 0xffc0319f/frame 0xd2d7808 --- trap 0xc, eip = 0x1409fa0, esp = 0xd2d7854, ebp = 0xd2d789c --- db_read_bytes(56e5ddb5,4,d2d78b0) at db_read_bytes+0x50/frame 0xd2d789c db_get_value(56e5ddb5,4,0) at db_get_value+0x22/frame 0xd2d78c8 db_backtrace(d2d79c4,a28588,0,ffffffff) at db_backtrace+0x313/frame 0xd2d7968 db_trace_self(151e400,d2d7984,d2d79c4,0,d2d798c,...) at db_trace_self+0x2b/frame 0xd2d7984 db_trace_self_wrapper(0,10280000,10280002,d2d7a48,202,...) at db_trace_self_wrapper+0x28/frame 0xd2d79c4 vpanic(151e486,d2d7a00,d2d7a00,d2d7ab0,12deb29,...) at vpanic+0x124/frame 0xd2d79e0 panic(151e486,15002a6,d2d8000,d2d8000,0,...) at panic+0x14/frame 0xd2d79f4 vm_fault(1e4f4ec,d2d8000,2,0,0) at vm_fault+0x1849/frame 0xd2d7ab0 vm_fault_trap(1e4f4ec,d2d8000,2,0,0,0) at vm_fault_trap+0x52/frame 0xd2d7ad8 trap_pfault(d2d8000,0,0) at trap_pfault+0x16a/frame 0xd2d7b24 trap(d2d7bf8,8,28,28,d2d8000,...) at trap+0x2d2/frame 0xd2d7bec calltrap() at 0xffc0319f/frame 0xd2d7bec --- trap 0xc, eip = 0x1404340, esp = 0xd2d7c38, ebp = 0xd2d7c60 --- random_nehemiah_read(65fbcbfe,7487cffd,937cb6b7,ae98cec2,f0294ab2,...) at random_nehemiah_read+0x60/frame 0xd2d7c60 Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x56e5ddb5 fault code = supervisor read data, page not present instruction pointer = 0x20:0x1409fa0 stack pointer = 0x28:0xd2d7534 frame pointer = 0x28:0xd2d757c code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = resume, IOPL = 0 current process = 5 (rand_harvestq) trap number = 12 panic: page fault cpuid = 0 time = 1634357727 KDB: stack backtrace: db_trace_self_wrapper(d2d74f4,10280000,d2d74f4,c,0,...) at db_trace_self_wrapper+0x28/frame 0xd2d7384 vpanic(1479d53,d2d73c0,d2d73c0,d2d73e8,142f377,...) at vpanic+0x124/frame 0xd2d73a0 panic(1479d53,150c115,0,fffff,d2df39b,...) at panic+0x14/frame 0xd2d73b4 trap_fatal(d2d0000,7f,1,8,1511f2a,...) at trap_fatal+0x347/frame 0xd2d73e8 trap_pfault(56e5ddb5,0,0) at trap_pfault+0x6f/frame 0xd2d741c trap(d2d74f4,8,28,28,d2d766c,...) at trap+0x2d2/frame 0xd2d74e8 calltrap() at 0xffc0319f/frame 0xd2d74e8 --- trap 0xc, eip = 0x1409fa0, esp = 0xd2d7534, ebp = 0xd2d757c --- db_read_bytes(56e5ddb5,4,d2d7590) at db_read_bytes+0x50/frame 0xd2d757c db_get_value(56e5ddb5,4,0) at db_get_value+0x22/frame 0xd2d75a8 db_backtrace(d2d76a4,a28588,0,ffffffff) at db_backtrace+0x313/frame 0xd2d7648 db_trace_self(1479d01,d2d7664,d2d76a4,d2d7854,d2d766c,...) at db_trace_self+0x2b/frame 0xd2d7664 db_trace_self_wrapper(d2d7814,10280000,d2d7814,c,0,...) at db_trace_self_wrapper+0x28/frame 0xd2d76a4 vpanic(1479d53,d2d76e0,d2d76e0,d2d7708,142f377,...) at vpanic+0x124/frame 0xd2d76c0 panic(1479d53,150c115,0,fffff,d2df69b,...) at panic+0x14/frame 0xd2d76d4 trap_fatal(d2d0000,7f,1,8,1511f2a,...) at trap_fatal+0x347/frame 0xd2d7708 trap_pfault(56e5ddb5,0,0) at trap_pfault+0x6f/frame 0xd2d773c trap(d2d7814,8,28,28,d2d798c,...) at trap+0x2d2/frame 0xd2d7808 calltrap() at 0xffc0319f/frame 0xd2d7808 --- trap 0xc, eip = 0x1409fa0, esp = 0xd2d7854, ebp = 0xd2d789c --- db_read_bytes(56e5ddb5,4,d2d78b0) at db_read_bytes+0x50/frame 0xd2d789c db_get_value(56e5ddb5,4,0) at db_get_value+0x22/frame 0xd2d78c8 db_backtrace(d2d79c4,a28588,0,ffffffff) at db_backtrace+0x313/frame 0xd2d7968 db_trace_self(151e400,d2d7984,d2d79c4,0,d2d798c,...) at db_trace_self+0x2b/frame 0xd2d7984 db_trace_self_wrapper(0,10280000,10280002,d2d7a48,202,...) at db_trace_self_wrapper+0x28/frame 0xd2d79c4 vpanic(151e486,d2d7a00,d2d7a00,d2d7ab0,12deb29,...) at vpanic+0x124/frame 0xd2d79e0 panic(151e486,15002a6,d2d8000,d2d8000,0,...) at panic+0x14/frame 0xd2d79f4 vm_fault(1e4f4ec,d2d8000,2,0,0) at vm_fault+0x1849/frame 0xd2d7ab0 vm_fault_trap(1e4f4ec,d2d8000,2,0,0,0) at vm_fault_trap+0x52/frame 0xd2d7ad8 trap_pfault(d2d8000,0,0) at trap_pfault+0x16a/frame 0xd2d7b24 trap(d2d7bf8,8,28,28,d2d8000,...) at trap+0x2d2/frame 0xd2d7bec calltrap() at 0xffc0319f/frame 0xd2d7bec --- trap 0xc, eip = 0x1404340, esp = 0xd2d7c38, ebp = 0xd2d7c60 --- random_nehemiah_read(65fbcbfe,7487cffd,937cb6b7,ae98cec2,f0294ab2,...) at random_nehemiah_read+0x60/frame 0xd2d7c60 This repeats over and over and over for a while until the machine simply resets/reboots and we need to fsck the root filesystem. There is no coredump. Here is what I see at boot time : /boot/kernel/kernel text=0xfecb4 text=0xb492a0 - text=0x5f66a0 data=0xb0 data=0x10df8c+0x332204 - syms=[0x4+0xece00+0x4+0x1847a6 Loading configured modules... /etc/hostid size=0x25 /boot/kernel/cryptodev.ko text=0x20a0 text=0x2113 data=0x228+0x8 syms=[0x4+0x8c0/+0x4+0xa94] /boot/entropy size=0x1000 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb ---<<BOOT>>--- MP Configuration Table version 1.4 found at 0x4f0c00 APIC: Using the MADT enumerator. Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.0-CURRENT #0 main-n249945-67bceb38f46: Thu Oct 7 19:43:11 UTC 2021 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/i386.i386/sys/GENERIC i386 FreeBSD clang version 12.0.1 (git@github.com:llvm/llvm-project.git llvmorg-12.0.1-0-gfed41342a82f) WARNING: WITNESS option enabled, expect reduced performance. VT(vga): resolution 640x480 Preloaded elf kernel "/boot/kernel/kernel" at 0x218d000. Preloaded hostuuid "/etc/hostid" at 0x2195b08. Preloaded elf module "/boot/kernel/cryptodev.ko" at 0x2195b48. Preloaded boot_entropy_cache "/boot/entropy" at 0x2195fe8. Preloaded TSLOG data "TSLOG" at 0x2196034. Calibrating TSC clock ... TSC clock: 400008539 Hz CPU: VIA Esther processor 1200MHz (400.01-MHz 686-class CPU) Origin="CentaurHauls" Id=0x6a9 Family=0x6 Model=0xa Stepping=9 Features=0xa7c9baff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,APIC,SEP,MTRR,PGE,CMOV,PAT,CLFLUSH,ACPI,MMX,FXSR,SSE,SSE2,TM,PBE> Features2=0x181<SSE3,EST,TM2> AMD Features=0x100000<NX> VIA Padlock Features=0x3fcc<RNG,AES,AES-CTR,SHA1,SHA256,RSA> real memory = 1073741824 (1024 MB) Physical memory chunk(s): 0x0000000000001000 - 0x000000000009dfff, 643072 bytes (157 pages) 0x0000000000100000 - 0x00000000007fffff, 7340032 bytes (1792 pages) 0x0000000002300000 - 0x000000003aca0fff, 949620736 bytes (231841 pages) avail memory = 955727872 (911 MB) MADT: Found CPU APIC ID 0 ACPI ID 0: enabled SMP: Added CPU 0 (AP) Event timer "LAPIC" quality 100 LAPIC: ipi_wait() us multiplier 3 (r 11200290 tsc 400008539) ACPI APIC Table: <CN700 AWRDACPI> APIC: CPU 0 has ACPI ID 0 Pentium Pro MTRR support enabled bios32: Found BIOS32 Service Directory header at 0x4fa4e0 bios32: Entry = 0xfa960 (4fa960) Rev = 0 Len = 1 pcibios: PCI BIOS entry at 0xf0000+0xa9b0 pnpbios: Found PnP BIOS data at 0x4fb420 pnpbios: Entry = f0000:b450 Rev = 1.0 Other BIOS signatures found: random: registering fast source VIA Nehemiah Padlock RNG random: fast provider: "VIA Nehemiah Padlock RNG" random: read 4096 bytes from preloaded cache random: unblocking device. VIMAGE (virtualized network stack) enabled hostuuid: using a566e725-28ad-11ec-9bb3-003018b029b6 ULE: setup cpu 0 ACPI: RSDP 0x00000000000F6E10 000014 (v00 CN700 ) ACPI: RSDT 0x000000003BEE3040 00002C (v01 CN700 AWRDACPI 42302E31 AWRD 00000000) ACPI: FACP 0x000000003BEE30C0 000074 (v01 CN700 AWRDACPI 42302E31 AWRD 00000000) ACPI: DSDT 0x000000003BEE3180 004FC3 (v01 CN700 AWRDACPI 00001000 MSFT 0100000E) ACPI: FACS 0x000000003BEE0000 000040 ACPI: APIC 0x000000003BEE81C0 00005A (v01 CN700 AWRDACPI 42302E31 AWRD 00000000) MADT: Found IO APIC ID 2, Interrupt 0 at 0xfec00000 ioapic0: ver 0x03 maxredir 0x17 ioapic0: Routing external 8259A's -> intpin 0 MADT: Interrupt override: source 0, irq 2 ioapic0: Routing IRQ 0 -> intpin 2 MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level ioapic0: intpin 9 polarity: low lapic0: Routing NMI -> LINT1 lapic0: LINT1 trigger: edge lapic0: LINT1 polarity: high ioapic0 <Version 0.3> irqs 0-23 lapic: Divisor 2, Frequency 50001087 Hz cpu0 BSP: ID: 0x00000000 VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 Timecounter "TSC" frequency 400008539 Hz quality 800 random: entropy device external interface wlan: <802.11 Link Layer> snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=2 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 crypto: <crypto core> kbd: new array size 4 kbd1 at kbdmux0 mem: <memory> null: <full device, null device, zero device> io: <I/O> [ath_hal] loaded vtvga0: <VT VGA driver> smbios0: <System Management BIOS> at iomem 0xf0be0-0xf0bfe smbios0: Version: 2.3, BCD Revision: 2.3 crypto: assign cryptosoft0 driver id 0, flags 0x6000000 aesni0: No AES or SHA support. acpi0: <CN700 AWRDACPI> ACPI: Actual Package length (8) is larger than NumElements field (2), truncated ACPI: Actual Package length (8) is larger than NumElements field (2), truncated ACPI: 1 ACPI AML tables successfully acquired and loaded ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 pci_open(1): mode 1 addr port (0x0cf8) is 0x80000760 pci_open(1a): mode1res=0x80000000 (0x80000000) pci_cfgcheck: device 0 [class=060000] [hdr=80] is there (id=03141106) pcibios: BIOS version 2.10 acpi0: Power Button (fixed) acpi0: wakeup code va 0xd468000 pa 0x1000 acpi0: reservation of f0000, 8000 (3) failed acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 3bde0000 (3) failed cpu0: <ACPI CPU> on acpi0 cpu0: switching to generic Cx mode attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 ioapic0: routing intpin 2 (ISA IRQ 0) to lapic 0 vector 49 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: <AT realtime clock> port 0x70-0x73 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 50 Event timer "RTC" frequency 32768 Hz quality 0 ACPI timer: -> 10 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 3 4 6 7 10 11 12 Validation 0 255 N 0 3 4 6 7 10 11 12 After Disable 0 255 N 0 3 4 6 7 10 11 12 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 3 4 6 7 10 11 12 Validation 0 11 N 0 3 4 6 7 10 11 12 After Disable 0 255 N 0 3 4 6 7 10 11 12 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 3 4 6 7 10 11 12 Validation 0 10 N 0 3 4 6 7 10 11 12 After Disable 0 255 N 0 3 4 6 7 10 11 12 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 6 7 10 11 12 Validation 0 255 N 0 3 4 6 7 10 11 12 After Disable 0 255 N 0 3 4 6 7 10 11 12 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 6 7 10 11 12 Validation 0 255 N 0 3 4 6 7 10 11 12 After Disable 0 255 N 0 3 4 6 7 10 11 12 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 6 7 10 11 12 Validation 0 255 N 0 3 4 6 7 10 11 12 After Disable 0 255 N 0 3 4 6 7 10 11 12 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 6 7 10 11 12 Validation 0 255 N 0 3 4 6 7 10 11 12 After Disable 0 255 N 0 3 4 6 7 10 11 12 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 6 7 10 11 12 Validation 0 255 N 0 3 4 6 7 10 11 12 After Disable 0 255 N 0 3 4 6 7 10 11 12 pci_link8: Index IRQ Rtd Ref IRQs Initial Probe 0 20 N 0 20 Validation 0 20 N 0 20 After Disable 0 255 N 0 20 pci_link9: Index IRQ Rtd Ref IRQs Initial Probe 0 21 N 0 21 Validation 0 21 N 0 21 After Disable 0 255 N 0 21 pci_link10: Index IRQ Rtd Ref IRQs Initial Probe 0 22 N 0 22 Validation 0 22 N 0 22 After Disable 0 255 N 0 22 pci_link11: Index IRQ Rtd Ref IRQs Initial Probe 0 23 N 0 23 Validation 0 23 N 0 23 After Disable 0 255 N 0 23 acpi_button0: <Power Button> on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pcib0: decoding 4 range 0-0xcf7 pcib0: decoding 4 range 0xd00-0xffff pcib0: decoding 3 range 0xa0000-0xbffff pcib0: decoding 3 range 0xc0000-0xdffff pcib0: decoding 3 range 0x3bf00000-0xfebfffff pci0: <ACPI PCI bus> on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x1106, dev=0x0314, revid=0x00 domain=0, bus=0, slot=0, func=0 class=06-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x2230, cachelnsz=0 (dwords) lattimer=0x08 (240 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 map[10]: type Prefetchable Memory, range 32, base rxe8000000, size 27, enabled pcib0: allocated type 3 (0xe8000000-0xefffffff) for rid 10 of pci0:0:0:0 found-> vendor=0x1106, dev=0x1314, revid=0x00 domain=0, bus=0, slot=0, func=1 class=06-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0200, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1106, dev=0x2314, revid=0x00 domain=0, bus=0, slot=0, func=2 class=06-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0200, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1106, dev=0x3208, revid=0x00 domain=0, bus=0, slot=0, func=3 class=06-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0200, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1106, dev=0x4314, revid=0x00 domain=0, bus=0, slot=0, func=4 class=06-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0200, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1106, dev=0x7314, revid=0x00 domain=0, bus=0, slot=0, func=7 class=06-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0200, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x1106, dev=0xb198, revid=0x00 domain=0, bus=0, slot=1, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0007, statreg=0x0230, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D1 D3 current D0 secbus=1, subbus=1 found-> vendor=0x10ec, dev=0x8167, revid=0x10 domain=0, bus=0, slot=9, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x02b0, cachelnsz=1 (dwords) lattimer=0x20 (960 ns), mingnt=0x20 (8000 ns), maxlat=0x40 (16000 ns) intpin=a, irq=10 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type I/O Port, range 32, base rxf200, size 8, enabled pcib0: allocated type 4 (0xf200-0xf2ff) for rid 10 of pci0:0:9:0 map[14]: type Memory, range 32, base rxfdfff000, size 8, enabled pcib0: allocated type 3 (0xfdfff000-0xfdfff0ff) for rid 14 of pci0:0:9:0 pcib0: matched entry for 0.9.INTA pcib0: slot 9 INTA hardwired to IRQ 18 found-> vendor=0x1106, dev=0x3149, revid=0x80 domain=0, bus=0, slot=15, func=0 class=01-01-8f, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x20 (960 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=11 powerspec 2 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base rxff00, size 3, enabled pcib0: allocated type 4 (0xff00-0xff07) for rid 10 of pci0:0:15:0 map[14]: type I/O Port, range 32, base rxfe00, size 2, enabled pcib0: allocated type 4 (0xfe00-0xfe03) for rid 14 of pci0:0:15:0 map[18]: type I/O Port, range 32, base rxfd00, size 3, enabled pcib0: allocated type 4 (0xfd00-0xfd07) for rid 18 of pci0:0:15:0 map[1c]: type I/O Port, range 32, base rxfc00, size 2, enabled pcib0: allocated type 4 (0xfc00-0xfc03) for rid 1c of pci0:0:15:0 map[20]: type I/O Port, range 32, base rxfb00, size 4, enabled pcib0: allocated type 4 (0xfb00-0xfb0f) for rid 20 of pci0:0:15:0 map[24]: type I/O Port, range 32, base rxf400, size 8, enabled pcib0: allocated type 4 (0xf400-0xf4ff) for rid 24 of pci0:0:15:0 pcib0: matched entry for 0.15.INTB (src \_SB_.PCI0.ALKA:0) pcib0: slot 15 INTB routed to irq 20 via \_SB_.PCI0.ALKA found-> vendor=0x1106, dev=0x0571, revid=0x06 domain=0, bus=0, slot=15, func=1 class=01-01-8a, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x20 (960 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=255 powerspec 2 supports D0 D3 current D0 pcib0: allocated type 4 (0x1f0-0x1f7) for rid 10 of pci0:0:15:1 pcib0: allocated type 4 (0x3f6-0x3f6) for rid 14 of pci0:0:15:1 pcib0: allocated type 4 (0x170-0x177) for rid 18 of pci0:0:15:1 pcib0: allocated type 4 (0x376-0x376) for rid 1c of pci0:0:15:1 map[20]: type I/O Port, range 32, base rxfa00, size 4, enabled pcib0: allocated type 4 (0xfa00-0xfa0f) for rid 20 of pci0:0:15:1 found-> vendor=0x1106, dev=0x3227, revid=0x00 domain=0, bus=0, slot=17, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x0087, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) powerspec 2 supports D0 D3 current D0 agp0: <VIA 3314 (P4M800CE) host to PCI bridge> on hostb0 WARNING: Device "agp" is Giant locked and may be deleted before FreeBSD 14.0. agp0: allocating GATT for aperture of size 128M agp0: aperture size is 128M pcib1: <PCI-PCI bridge> at device 1.0 on pci0 pcib0: allocated type 4 (0xe000-0xefff) for rid 1c of pcib1 pcib0: allocated type 3 (0xfb000000-0xfcffffff) for rid 20 of pcib1 pcib0: allocated type 3 (0xf4000000-0xf7ffffff) for rid 24 of pcib1 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pcib1: I/O decode 0xe000-0xefff pcib1: memory decode 0xfb000000-0xfcffffff pcib1: prefetched decode 0xf4000000-0xf7ffffff pcib1: special decode VGA pci1: <PCI bus> on pcib1 pcib1: allocated bus range (1-1) for rid 0 of pci1 pci1: domain=0, physical bus=1 found-> vendor=0x1106, dev=0x3344, revid=0x01 domain=0, bus=1, slot=0, func=0 class=03-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x0230, cachelnsz=0 (dwords) lattimer=0x20 (960 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) intpin=a, irq=5 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Prefetchable Memory, range 32, base rxf4000000, size 26, enabled pcib1: allocated prefetch range (0xf4000000-0xf7ffffff) for rid 10 of pci0:1:0:0 map[14]: type Memory, range 32, base rxfb000000, size 24, enabled pcib1: allocated memory range (0xfb000000-0xfbffffff) for rid 14 of pci0:1:0:0 pcib0: matched entry for 0.1.INTA pcib0: slot 1 INTA hardwired to IRQ 16 pcib1: slot 0 INTA is routed to irq 16 vgapci0: <VGA-compatible display> mem 0xf4000000-0xf7ffffff,0xfb000000-0xfbffffff irq 16 at device 0.0 on pci1 vgapci0: Boot video device re0: <RealTek 8169SC/8110SC Single-chip Gigabit Ethernet> port 0xf200-0xf2ff mem 0xfdfff000-0xfdfff0ff irq 18 at device 9.0 on pci0 re0: MSI count : 0 re0: MSI-X count : 0 re0: Chip rev. 0x18000000 re0: MAC rev. 0x00000000 miibus0: <MII bus> on re0 rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus0 rgephy0: OUI 0x00e04c, model 0x0011, rev. 2 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow re0: Using defaults for TSO: 65518/35/2048 re0: bpf attached re0: Ethernet address: 00:30:18:b0:29:b6 ioapic0: routing intpin 18 (PCI IRQ 18) to lapic 0 vector 51 atapci0: <VIA 6420 SATA150 controller> port 0xff00-0xff07,0xfe00-0xfe03,0xfd00-0xfd07,0xfc00-0xfc03,0xfb00-0xfb0f,0xf400-0xf4ff irq 20 at device 15.0 on pci0 ioapic0: routing intpin 20 (PCI IRQ 20) to lapic 0 vector 52 ata2: <ATA channel> at channel 0 on atapci0 ata3: <ATA channel> at channel 1 on atapci0 atapci1: <VIA 8237 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfa00-0xfa0f at device 15.1 on pci0 ata0: <ATA channel> at channel 0 on atapci1 ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 0 vector 53 ata1: <ATA channel> at channel 1 on atapci1 ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 0 vector 54 isab0: <PCI-ISA bridge> at device 17.0 on pci0 isa0: <ISA bus> on isab0 acpi_tz0: <Thermal Zone> on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (9600,n,8,1) ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 55 uart0: fast interrupt uart0: PPS capture mode: DCD atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 atkbd: the current kbd controller command byte 0065 atkbd: keyboard ID 0x41ab (2) kbdc: RESET_KBD return code:00fa kbdc: RESET_KBD status:00aa kbd0 at atkbd0 kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x1d0000 ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 56 atkbd0: [GIANT-LOCKED] psm0: unable to allocate IRQ ahc_isa_identify 0: ioport 0xc00 alloc failed ahc_isa_identify 1: ioport 0x1c00 alloc failed ahc_isa_identify 2: ioport 0x2c00 alloc failed ahc_isa_identify 3: ioport 0x3c00 alloc failed ahc_isa_identify 4: ioport 0x4c00 alloc failed ahc_isa_identify 5: ioport 0x5c00 alloc failed ahc_isa_identify 6: ioport 0x6c00 alloc failed ahc_isa_identify 7: ioport 0x7c00 alloc failed ahc_isa_identify 8: ioport 0x8c00 alloc failed ahc_isa_identify 9: ioport 0x9c00 alloc failed ahc_isa_identify 10: ioport 0xac00 alloc failed ahc_isa_identify 11: ioport 0xbc00 alloc failed ahc_isa_identify 12: ioport 0xcc00 alloc failed ahc_isa_identify 13: ioport 0xdc00 alloc failed ahc_isa_identify 14: ioport 0xec00 alloc failed pcib0: allocated type 3 (0xb0000-0xb07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb0800-0xb0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb1000-0xb17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb1800-0xb1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb2000-0xb27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb2800-0xb2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb3000-0xb37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb3800-0xb3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb4000-0xb47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb4800-0xb4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb5000-0xb57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb5800-0xb5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb6000-0xb67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb6800-0xb6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb7000-0xb77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb7800-0xb7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb8000-0xb87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb8800-0xb8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb9000-0xb97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb9800-0xb9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xba000-0xba7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xba800-0xbafff) for rid 0 of orm0 pcib0: allocated type 3 (0xbb000-0xbb7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbb800-0xbbfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbc000-0xbc7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbc800-0xbcfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbd000-0xbd7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbd800-0xbdfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe000-0xbe7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe800-0xbefff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf000-0xbf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf800-0xbffff) for rid 0 of orm0 pcib0: allocated type 3 (0xc0000-0xc07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc0800-0xc0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc1000-0xc17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc1800-0xc1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc2000-0xc27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc2800-0xc2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc3000-0xc37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc3800-0xc3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc4000-0xc47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc4800-0xc4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc5000-0xc57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc5800-0xc5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc6000-0xc67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc6800-0xc6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc7000-0xc77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc7800-0xc7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc8000-0xc87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc8800-0xc8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xc9000-0xc97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xc9800-0xc9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xca000-0xca7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xca800-0xcafff) for rid 0 of orm0 pcib0: allocated type 3 (0xcb000-0xcb7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xcb800-0xcbfff) for rid 0 of orm0 pcib0: allocated type 3 (0xcc000-0xcc7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xcc800-0xccfff) for rid 0 of orm0 pcib0: allocated type 3 (0xcd000-0xcd7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xcd800-0xcdfff) for rid 0 of orm0 pcib0: allocated type 3 (0xce000-0xce7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xce800-0xcefff) for rid 0 of orm0 pcib0: allocated type 3 (0xcf000-0xcf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xcf800-0xcffff) for rid 0 of orm0 pcib0: allocated type 3 (0xd0000-0xd07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xd0000-0xd4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xd5000-0xd57ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd5800-0xd5fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd6000-0xd67ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd6800-0xd6fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd7000-0xd77ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd7800-0xd7fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd8000-0xd87ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd8800-0xd8fff) for rid 1 of orm0 pcib0: allocated type 3 (0xd9000-0xd97ff) for rid 1 of orm0 pcib0: allocated type 3 (0xd9800-0xd9fff) for rid 1 of orm0 pcib0: allocated type 3 (0xda000-0xda7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xda800-0xdafff) for rid 1 of orm0 pcib0: allocated type 3 (0xdb000-0xdb7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdb800-0xdbfff) for rid 1 of orm0 pcib0: allocated type 3 (0xdc000-0xdc7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdc800-0xdcfff) for rid 1 of orm0 pcib0: allocated type 3 (0xdd000-0xdd7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdd800-0xddfff) for rid 1 of orm0 pcib0: allocated type 3 (0xde000-0xde7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xde800-0xdefff) for rid 1 of orm0 pcib0: allocated type 3 (0xdf000-0xdf7ff) for rid 1 of orm0 pcib0: allocated type 3 (0xdf800-0xdffff) for rid 1 of orm0 pnp_identify: Trying Read_Port at 203 pnp_identify: Trying Read_Port at 243 pnp_identify: Trying Read_Port at 283 pnp_identify: Trying Read_Port at 2c3 pnp_identify: Trying Read_Port at 303 pnp_identify: Trying Read_Port at 343 pnp_identify: Trying Read_Port at 383 pnp_identify: Trying Read_Port at 3c3 PNP Identify complete isa_probe_children: disabling PnP devices ata: ata0 already exists; skipping it ata: ata1 already exists; skipping it atkbdc: atkbdc0 already exists; skipping it atrtc: atrtc0 already exists; skipping it attimer: attimer0 already exists; skipping it sc: sc0 already exists; skipping it uart: uart0 already exists; skipping it isa_probe_children: probing non-PnP devices orm0: <ISA Option ROM> at iomem 0xd0000-0xd4fff pnpid ORM0000 on isa0 sc0 failed to probe on isa0 vga0 failed to probe on isa0 pcib0: allocated type 4 (0x3f0-0x3f5) for rid 0 of fdc0 pcib0: allocated type 4 (0x3f7-0x3f7) for rid 1 of fdc0 fdc0 failed to probe at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 ppc0: parallel port not found. ppc0 failed to probe at irq 7 on isa0 pcib0: allocated type 4 (0x2f8-0x2f8) for rid 0 of uart1 uart1 failed to probe at port 0x2f8 irq 3 on isa0 isa_probe_children: probing PnP devices AcpiOsExecute: task queue not started est0: <Enhanced SpeedStep Frequency Control> on cpu0 est0: current setting not found in table Device configuration finished. procfs registered Timecounters tick every 1.000 msec lo0: bpf attached vlan: initialized, using hash tables with chaining crypto: <crypto device> tcp_init: net.inet.tcp.tcbhashsize auto tuned to 8192 IPsec: Initialized Security Association Processing. usb_needs_explore_all: no devclass AcpiOsExecute: enqueue 1 pending tasks ata2: SATA connect time=0ms status=00000113 WARNING: WITNESS option enabled, expect reduced performance. ata2: reset tp1 mask=01 ostat0=50 ostat1=00 Trying to mount root from ufs:/dev/ada0s1a [rw]... Root mount waiting for: CAM ata2: stat0=0x50 err=0x01 lsb=0x00 msb=0x00 ata2: reset tp2 stat0=50 stat1=00 devices=0x1 ata3: SATA connect timeout status=00000000 Root mount waiting for: CAM ata0: reset tp1 mask=03 ostat0=60 ostat1=70 ata0: stat0=0x20 err=0x20 lsb=0x20 msb=0x20 ata0: stat1=0x30 err=0x30 lsb=0x30 msb=0x30 ata0: reset tp2 stat0=20 stat1=30 devices=0x0 ata1: reset tp1 mask=00 ostat0=ff ostat1=ff pass0 at ata2 bus 0 scbus0 target 0 lun 0 pass0: <TOSHIBA MK3265GSX GJ003A> ATA8-ACS SATA 1.x device pass0: Serial Number 801NB7THB pass0: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) ada0 at ata2 bus 0 scbus0 target 0 lun 0 ada0: <TOSHIBA MK3265GSX GJ003A> ATA8-ACS SATA 1.x device ada0: Serial Number 801NB7THB ada0: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) ada0: 305245MB (625142448 512 byte sectors) GEOM: new disk ada0 mountroot: waiting for device /dev/ada0s1a... WARNING: / was not properly dismounted atrtc0: providing initial system time start_init: trying /sbin/init Enter root password, or ^D to go multi-user Password: Enter full pathname of shell or RETURN for /bin/sh: then of course comes the fsck of the root filesystem and in no time at all we are running again. root@:/ # fsck -y /dev/ada0s1a ** /dev/ada0s1a ** SU+J Recovering /dev/ada0s1a Journal timestamp does not match fs mount time ** Skipping journal, falling through to full fsck ** Last Mounted on / ** Root file system ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 135909 files, 831357 used, 73818561 free (969 frags, 9227199 blocks, 0.0% fragmentation) ***** FILE SYSTEM IS CLEAN ***** root@:/ # exit Setting hostuuid: a566e725-28ad-11ec-9bb3-003018b029b6. Setting hostid: 0x14a15242. Fast boot: skipping disk checks. Mounting local filesystems:. Setting up harvesting: PURE_NEHEMIAH,[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: . Setting hostname: esther. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/perl5/5.32/mach/CORE lo0: link state changed to UP re0: link state changed to DOWN Starting Network: lo0 re0. lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUMre0: link state changed to UP ,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE> ether 00:30:18:b0:29:b6 inet 172.16.35.59 netmask 0xffffffc0 broadcast 172.16.35.63 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> Starting devd. add host 127.0.0.1: gateway lo0 fib 0: route already in table add net default: gateway 172.16.35.1 add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Updating motd:. Clearing /tmp. Creating and/or trimming log files. Updating /var/run/os-release done. Starting syslogd. No core dumps found. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting sendmail_submit. Starting sendmail_msp_queue. Starting cron. Starting background file system checks in 60 seconds. Sat Oct 16 04:25 FreeBSD/i386 (esther) (ttyu0) login: I have tried different types of load to trigger the panic. It seems almost at random. Network load sometimes works. Begin a buildworld seems fine for fifteen minutes and then a panic. -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional
What is the source line for random_nehemiah_read+0x60?
(In reply to Konstantin Belousov from comment #1) https://cgit.freebsd.org/src/tree/sys/dev/random/nehemiah.c#n96 What are you asking really ?
(In reply to Dennis Clarke from comment #2) I am asking for the line number for the instruction at random_nehemiah_read+0x60.
(In reply to Konstantin Belousov from comment #3) I see no way to get that information. The machine never seems to stop these repeat panics and it always resets. I think I saw it once and only once stop at the kernel debugger. There is no coredump even though there is plenty of swap. If I ever see the kernel debugger prompt then I will try a "dump" and "reset" but that seems unlikely.
You do not need neither kernel debugger nor dump. Load your kernel.full in gdb, then do "list *random_nehemiah_read+0x60'.
(In reply to Konstantin Belousov from comment #5) I do not see a file called "kernel.full" anywhere. This machine was installed from a snapshot install media about a week ago. I am trying to install gdb but the machine panic'd and reset during the pkg install and I am trying to get to a point where some sort of information can be achieved.
(In reply to Konstantin Belousov from comment #5) I am going to move the single hard disk over to another machine and then get a buildworld/buildkernel going. Then I can migrate the disk back to this odd ball motherboard where I get a panic every hour.
Build was easy on another machine : esther# uname -apKU FreeBSD esther 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n250102-d95c0a12a2d: Mon Oct 18 05:58:15 GMT 2021 root@esther:/usr/obj/usr/src/i386.i386/sys/GENERIC i386 i386 1400038 1400038 I also have that kernel.full : esther# ls -lapb /usr/obj/usr/src/i386.i386/sys/GENERIC/kernel.full -rwxr-xr-x 1 root wheel 86441196 Oct 18 05:58 /usr/obj/usr/src/i386.i386/sys/GENERIC/kernel.full esther# So now I wait and see if the panic happens again.
Finally I can follow up here. The kernel ( and buildworld ) were done with another machine and then I was able to move the hard disk back to the troublesome VIA Eden Esther motherboard. After letting the machine boot : esther# esther# uname -apKU FreeBSD esther 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n250102-d95c0a12a2d: Mon Oct 18 05:58:15 GMT 2021 root@esther:/usr/obj/usr/src/i386.i386/sys/GENERIC i386 i386 1400038 1400038 esther# I left it to sit idle for a good long while and of course we caught a panic and even a coredump. esther# esther# ls -lap /var/crash/ total 109816 drwxr-x--- 2 root wheel 512 Oct 18 18:24 ./ drwxr-xr-x 24 root wheel 512 Oct 18 19:20 ../ -rw-r--r-- 1 root wheel 2 Oct 18 18:24 bounds -rw-r--r-- 1 root wheel 84 Oct 18 18:24 core.txt.0 -rw------- 1 root wheel 516 Oct 18 18:24 info.0 lrwxr-xr-x 1 root wheel 6 Oct 18 18:24 info.last -> info.0 -rw-r--r-- 1 root wheel 5 Oct 7 21:44 minfree -rw------- 1 root wheel 121741312 Oct 18 18:24 vmcore.0 lrwxr-xr-x 1 root wheel 8 Oct 18 18:24 vmcore.last -> vmcore.0 esther# Regardless lets get what you asked for : esther# cd esther# setenv TERM 'dumb' esther# gdb -q /usr/obj/usr/src/i386.i386/sys/GENERIC/kernel.full Reading symbols from /usr/obj/usr/src/i386.i386/sys/GENERIC/kernel.full... (gdb) list *random_nehemiah_read+0x60 0x1404240 is in random_nehemiah_read (/usr/src/sys/dev/random/nehemiah.c:69). 64 { 65 uint32_t retval = 0; 66 uint32_t rate = 0; 67 68 #ifdef __GNUCLIKE_ASM 69 __asm __volatile( 70 "movl $0,%%edx\n\t" 71 "xstore" 72 : "=a" (retval), "+d" (rate), "+D" (buf) 73 : (gdb) for the sake of looking at more lines : (gdb) list - 54 .rs_source = RANDOM_PURE_NEHEMIAH, 55 .rs_read = random_nehemiah_read 56 }; 57 58 static struct fpu_kern_ctx *fpu_ctx_save; 59 60 /* This H/W source never stores more than 8 bytes in one go */ 61 /* ARGSUSED */ 62 static __inline size_t 63 VIA_RNG_store(void *buf) (gdb) list 64 { 65 uint32_t retval = 0; 66 uint32_t rate = 0; 67 68 #ifdef __GNUCLIKE_ASM 69 __asm __volatile( 70 "movl $0,%%edx\n\t" 71 "xstore" 72 : "=a" (retval), "+d" (rate), "+D" (buf) 73 : (gdb) list 74 : "memory" 75 ); 76 #endif 77 if (rate == 0) 78 return (retval&0x1f); 79 return (0); 80 } 81 82 static void 83 random_nehemiah_init(void) (gdb) quit esther# Hope this helps however I can xz compress that coredump and upload it however I worry that a coredump will contain security data such as a root password.
I need simple but consistent data from you, and I do not need the core dump. I need the exact content from console on the panic, like what you provided before, but for the kernel that you built anew. Then, for the same kernel, I need either source line for the last top frame before trap, or (this would probably easier to describe) a disassembly of random_nehemiah_read(). You do not need to wait for the kernel panic to get the second thing: % gdb kernel.full (gdb) disassemble random_nehemiah_read But again, I need both panic info and disassembly _from then same kernel binary_.
Well, the most recent comment and data that I provided was from a panic on the same new kernel that I built in the last 48 hours. esther# uname -apKU FreeBSD esther 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n250102-d95c0a12a2d: Mon Oct 18 05:58:15 GMT 2021 root@esther:/usr/obj/usr/src/i386.i386/sys/GENERIC i386 i386 1400038 1400038 esther# Why did I bother to build a new kernel? Because there was no way to follow your instructions when there is no such thing as "kernel.full" previously. Also no way to fetch it and then use gdb. Why? There was no way to install gdb. The whole system was so unstable that it did a panic in the middle of "pkg install gdb" and that mess landed me in a situation where the pkg sqlite database held a pid lock record on the process when the panic happened. It has been a ton of fun just getting to this point. This is what I saw on the console when the panic happened in the middle of trying to do "pkg install gdb" : Last login: Mon Oct 18 01:58:17 on ttyu0 Oct 18 18:37:35 esther login[753]: ROOT LOGIN (root) ON ttyu0 FreeBSD 14.0-CURRENT (GENERIC) #0 main-n250102-d95c0a12a2d: Mon Oct 18 05:58:15 GMT 2021 Welcome to FreeBSD! Release Notes, Errata: https://www.FreeBSD.org/releases/ Security Advisories: https://www.FreeBSD.org/security/ FreeBSD Handbook: https://www.FreeBSD.org/handbook/ FreeBSD FAQ: https://www.FreeBSD.org/faq/ Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/ FreeBSD Forums: https://forums.FreeBSD.org/ Documents installed with the system are in the /usr/local/share/doc/freebsd/ directory, or can be installed later with: pkg install en-freebsd-doc For other languages, replace "en" with a language code like de or fr. Show the version of FreeBSD installed: freebsd-version ; uname -a Please include that output and any error messages when posting questions. Introduction to manual pages: man man FreeBSD directory layout: man hier To change this login announcement, see motd(5). You have new mail. esther# esther# ls -lap /var/crash total 109816 drwxr-x--- 2 root wheel 512 Oct 18 18:24 ./ drwxr-xr-x 24 root wheel 512 Oct 18 18:23 ../ -rw-r--r-- 1 root wheel 2 Oct 18 18:24 bounds -rw-r--r-- 1 root wheel 84 Oct 18 18:24 core.txt.0 -rw------- 1 root wheel 516 Oct 18 18:24 info.0 lrwxr-xr-x 1 root wheel 6 Oct 18 18:24 info.last -> info.0 -rw-r--r-- 1 root wheel 5 Oct 7 21:44 minfree -rw------- 1 root wheel 121741312 Oct 18 18:24 vmcore.0 lrwxr-xr-x 1 root wheel 8 Oct 18 18:24 vmcore.last -> vmcore.0 esther# esther# cat /var/crash/core.txt.0 Unable to find a kernel debugger. Please install the devel/gdb port or gdb package. esther# esther# pkg update --force Updating FreeBSD repository catalogue... Fetching meta.conf: 100% 163 B 0.2kB/s 00:01 Fetching packagesite.pkg: 100% 6 MiB 3.2MB/s 00:02 Processing entries: 100% FreeBSD repository update completed. 30438 packages processed. All repositories are up to date. esther# esther# date -u Mon Oct 18 18:41:42 UTC 2021 esther# pkg audit 0 problem(s) in 0 installed package(s) found. esther# esther# pkg install gdb Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. Checking integrity... done (0 conflicting) The following 4 package(s) will be affected (of 0 checked): New packages to be INSTALLED: boost-libs: 1.72.0_5 gdb: 11.1 libiconv: 1.16 source-highlight: 3.1.9_1 Number of packages to be installed: 4 The process will require 218 MiB more space. Proceed with this action? [y/N]: y [1/4] Installing boost-libs-1.72.0_5... [1/4] Extracting boost-libs-1.72.0_5: 98%panic: vm_fault_lookup: fault on nofault entry, addr: 0xd2d8000 cpuid = 0 time = 1634582864 KDB: stack backtrace: db_trace_self_wrapper(0,10283000,10283002,d2d7a48,202,...) at db_trace_self_wrapper+0x28/frame 0xd2d79bc vpanic(151e45c,d2d79f8,d2d79f8,d2d7ab0,12de9b2,...) at vpanic+0x124/frame 0xd2d79d8 panic(151e45c,150024a,d2d8000,d2d8000,0,...) at panic+0x14/frame 0xd2d79ec vm_fault(1e4f5ec,d2d8000,2,0,0) at vm_fault+0x1862/frame 0xd2d7ab0 vm_fault_trap(1e4f5ec,d2d8000,2,0,0,0) at vm_fault_trap+0x52/frame 0xd2d7ad8 trap_pfault(d2d8000,0,0) at trap_pfault+0x16a/frame 0xd2d7b24 trap(d2d7bf8,8,28,28,d2d8000,...) at trap+0x2d2/frame 0xd2d7bec calltrap() at 0xffc0319f/frame 0xd2d7bec --- trap 0xc, eip = 0x1404240, esp = 0xd2d7c38, ebp = 0xd2d7c60 --- random_nehemiah_read(b2d5278a,f90b3d82,3e782ecf,9908ee70,a799ef37,...) at random_nehemiah_read+0x60/frame 0xd2d7c60 Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xb2b9d8d1 fault code = supervisor read data, page not present instruction pointer = 0x20:0x1409ea0 stack pointer = 0x28:0xd2d784c frame pointer = 0x28:0xd2d7894 code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = resume, IOPL = 0 current process = 5 (rand_harvestq) trap number = 12 panic: page fault cpuid = 0 time = 1634582864 KDB: stack backtrace: db_trace_self_wrapper(d2d780c,10283000,d2d780c,c,0,...) at db_trace_self_wrapper+0x28/frame 0xd2d769c vpanic(1479c1e,d2d76d8,d2d76d8,d2d7700,142f277,...) at vpanic+0x124/frame 0xd2d76b8 panic(1479c1e,150c09f,0,fffff,d2df69b,...) at panic+0x14/frame 0xd2d76cc trap_fatal(d2d0000,7f,1,8,1511eb4,...) at trap_fatal+0x347/frame 0xd2d7700 trap_pfault(b2b9d8d1,0,0) at trap_pfault+0x6f/frame 0xd2d7734 trap(d2d780c,8,28,28,d2d7984,...) at trap+0x2d2/frame 0xd2d7800 calltrap() at 0xffc0319f/frame 0xd2d7800 --- trap 0xc, eip = 0x1409ea0, esp = 0xd2d784c, ebp = 0xd2d7894 --- . . . I will attach the remainder of the 4000+ lines in a txt file.
Created attachment 228851 [details] kernel panic as seen on the serial console As promised here is the entire console output.
Here is the disassembly : dclarke@esther:~ $ uname -apKU FreeBSD esther 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n250102-d95c0a12a2d: Mon Oct 18 05:58:15 GMT 2021 root@esther:/usr/obj/usr/src/i386.i386/sys/GENERIC i386 i386 1400038 1400038 dclarke@esther:~ $ dclarke@esther:~ $ dclarke@esther:~ $ ls -lap /usr/obj/usr/src/i386.i386/sys/GENERIC/kernel.full -rwxr-xr-x 1 root wheel 86441196 Oct 18 05:58 /usr/obj/usr/src/i386.i386/sys/GENERIC/kernel.full dclarke@esther:~ $ dclarke@esther:~ $ TERM=dumb gdb -q /usr/obj/usr/src/i386.i386/sys/GENERIC/kernel.full Reading symbols from /usr/obj/usr/src/i386.i386/sys/GENERIC/kernel.full... (gdb) disassemble random_nehemiah_read Dump of assembler code for function random_nehemiah_read: 0x014041e0 <+0>: push %ebp 0x014041e1 <+1>: mov %esp,%ebp 0x014041e3 <+3>: push %ebx 0x014041e4 <+4>: push %edi 0x014041e5 <+5>: push %esi 0x014041e6 <+6>: and $0xfffffff8,%esp 0x014041e9 <+9>: sub $0x18,%esp 0x014041ec <+12>: mov %fs:0x0,%eax 0x014041f2 <+18>: push $0x0 0x014041f4 <+20>: push 0x1e5bb30 0x014041fa <+26>: mov %eax,0xc(%esp) 0x014041fe <+30>: push %eax 0x014041ff <+31>: call 0x1416e90 <fpu_kern_enter> 0x01404204 <+36>: add $0xc,%esp 0x01404207 <+39>: cmpl $0x0,0xc(%ebp) 0x0140420b <+43>: je 0x1404277 <random_nehemiah_read+151> 0x0140420d <+45>: mov 0x8(%ebp),%esi 0x01404210 <+48>: mov 0xc(%ebp),%ebx 0x01404213 <+51>: jmp 0x1404235 <random_nehemiah_read+85> 0x01404215 <+53>: nop 0x01404216 <+54>: nop 0x01404217 <+55>: nop 0x01404218 <+56>: nop 0x01404219 <+57>: nop 0x0140421a <+58>: nop 0x0140421b <+59>: nop 0x0140421c <+60>: nop 0x0140421d <+61>: nop 0x0140421e <+62>: nop 0x0140421f <+63>: nop 0x01404220 <+64>: push %edi 0x01404221 <+65>: lea 0xc(%esp),%eax 0x01404225 <+69>: push %eax 0x01404226 <+70>: push %esi 0x01404227 <+71>: call 0x142caf8 <memcpy> 0x0140422c <+76>: add $0xc,%esp 0x0140422f <+79>: add %edi,%esi 0x01404231 <+81>: sub %edi,%ebx 0x01404233 <+83>: je 0x1404277 <random_nehemiah_read+151> 0x01404235 <+85>: xor %edx,%edx 0x01404237 <+87>: lea 0x8(%esp),%edi --Type <RET> for more, q to quit, c to continue without paging-- 0x0140423b <+91>: mov $0x0,%edx 0x01404240 <+96>: repz xstore-rng 0x01404244 <+100>: and $0x1f,%eax 0x01404247 <+103>: test %edx,%edx 0x01404249 <+105>: mov $0x0,%ecx 0x0140424e <+110>: cmovne %ecx,%eax 0x01404251 <+113>: mov %ebx,%edi 0x01404253 <+115>: cmp %ebx,%eax 0x01404255 <+117>: jae 0x1404220 <random_nehemiah_read+64> 0x01404257 <+119>: xor %edx,%edx 0x01404259 <+121>: lea 0x8(%esp),%edi 0x0140425d <+125>: mov $0x0,%edx 0x01404262 <+130>: repz xstore-rng 0x01404266 <+134>: mov %eax,%edi 0x01404268 <+136>: and $0x1f,%edi 0x0140426b <+139>: test %edx,%edx 0x0140426d <+141>: mov $0x0,%eax 0x01404272 <+146>: cmovne %eax,%edi 0x01404275 <+149>: jmp 0x1404220 <random_nehemiah_read+64> 0x01404277 <+151>: push 0x1e5bb30 0x0140427d <+157>: push 0x8(%esp) 0x01404281 <+161>: call 0x1417100 <fpu_kern_leave> 0x01404286 <+166>: add $0x8,%esp 0x01404289 <+169>: mov 0xc(%ebp),%eax 0x0140428c <+172>: lea -0xc(%ebp),%esp 0x0140428f <+175>: pop %esi 0x01404290 <+176>: pop %edi 0x01404291 <+177>: pop %ebx 0x01404292 <+178>: pop %ebp 0x01404293 <+179>: ret End of assembler dump. (gdb) list *random_nehemiah_read 0x14041e0 is in random_nehemiah_read (/usr/src/sys/dev/random/nehemiah.c:99). 94 } 95 96 /* It is specifically allowed that buf is a multiple of sizeof(long) */ 97 static u_int 98 random_nehemiah_read(void *buf, u_int c) 99 { 100 uint8_t *b; 101 size_t count, ret; 102 uint64_t tmp; 103 (gdb) 104 fpu_kern_enter(curthread, fpu_ctx_save, FPU_KERN_NORMAL); 105 b = buf; 106 for (count = c; count > 0; count -= ret) { 107 ret = MIN(VIA_RNG_store(&tmp), count); 108 memcpy(b, &tmp, ret); 109 b += ret; 110 } 111 fpu_kern_leave(curthread, fpu_ctx_save); 112 113 return (c); (gdb) 114 } 115 116 static int 117 nehemiah_modevent(module_t mod, int type, void *unused) 118 { 119 int error = 0; 120 121 switch (type) { 122 case MOD_LOAD: 123 if (via_feature_rng & VIA_HAS_RNG) { (gdb) quit dclarke@esther:~ $
Try this please diff --git a/sys/dev/random/nehemiah.c b/sys/dev/random/nehemiah.c index ba0dfb200ba0..3ad18005c935 100644 --- a/sys/dev/random/nehemiah.c +++ b/sys/dev/random/nehemiah.c @@ -68,7 +68,7 @@ VIA_RNG_store(void *buf) #ifdef __GNUCLIKE_ASM __asm __volatile( "movl $0,%%edx\n\t" - "xstore" + ".byte 0x0f, 0xa7, 0xc0" : "=a" (retval), "+d" (rate), "+D" (buf) : : "memory"
(In reply to Konstantin Belousov from comment #14) looks very similar to : https://github.com/torvalds/linux/blob/2f111a6fd5b5297b4e92f53798ca086f7c7d33a4/drivers/char/hw_random/via-rng.c#L77 done! System is running fine and I am trying to stress it a bit. We won't really know unless I can stress it and let it be for a day or so but usually the thing has a panic and reset within a few hours. Or sooner!
(In reply to Konstantin Belousov from comment #14) The little machine is chugging away on a buildworld and seems to be nice and stable. However it sets a new record for "slow" and that is fine with me. In fact the git clone was hours : esther# pwd /usr/src esther# cd .. esther# rm -rf src esther# /usr/bin/time -p git clone anongit@git.freebsd.org:src.git src Cloning into 'src'... The authenticity of host 'git.freebsd.org (96.47.72.109)' can't be established. ED25519 key fingerprint is SHA256:y1ljKrKMD3lDObRUG3xJ9gXwEIuqnh306tSyFd1tuZE. Matching host key fingerprint found in DNS. This key is not known by any other names Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added 'git.freebsd.org' (ED25519) to the list of known hosts. remote: Enumerating objects: 4015024, done. remote: Counting objects: 100% (379021/379021), done. remote: Compressing objects: 100% (27310/27310), done. remote: Total 4015024 (delta 373279), reused 351711 (delta 351711), pack-reused 3636003 Receiving objects: 100% (4015024/4015024), 1.34 GiB | 692.00 KiB/s, done. Resolving deltas: 100% (3179795/3179795), done. Checking objects: 100% (8388608/8388608), done. Updating files: 100% (87898/87898), done. real 10272.67 user 8911.64 sys 904.85 esther# esther# cat /root/sys_dev_random_nehemiah.patch --- sys/dev/random/nehemiah.c.orig 2021-10-18 02:07:54.279416000 +0000 +++ sys/dev/random/nehemiah.c 2021-10-21 00:28:09.571500000 +0000 @@ -68,7 +68,7 @@ #ifdef __GNUCLIKE_ASM __asm __volatile( "movl $0,%%edx\n\t" - "xstore" + ".byte 0x0f, 0xa7, 0xc0" : "=a" (retval), "+d" (rate), "+D" (buf) : : "memory" esther# I figure the buildworld will run until sometime in November. Still a bit faster than QEMU and RISC-V emulation.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=362c6d8dec6dfbedc5717e45769042279a1dd3df commit 362c6d8dec6dfbedc5717e45769042279a1dd3df Author: Konstantin Belousov <kib@FreeBSD.org> AuthorDate: 2021-10-20 22:42:05 +0000 Commit: Konstantin Belousov <kib@FreeBSD.org> CommitDate: 2021-10-22 23:31:16 +0000 nehemiah: manually assemble xstore(-rng) It seems that clang IAS erronously adds repz prefix which should not be there. Cpu would try to store around %ecx bytes of random, while we only expect a word. PR: 259218 Reported and tested by: Dennis Clarke <dclarke@blastwave.org> Sponsored by: The FreeBSD Foundation MFC after: 1 week sys/dev/random/nehemiah.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=5306ca9a8e2f2b3bfa7d1ebc784dedbd5422fdae commit 5306ca9a8e2f2b3bfa7d1ebc784dedbd5422fdae Author: Konstantin Belousov <kib@FreeBSD.org> AuthorDate: 2021-10-20 22:42:05 +0000 Commit: Konstantin Belousov <kib@FreeBSD.org> CommitDate: 2021-10-28 00:35:54 +0000 nehemiah: manually assemble xstore(-rng) PR: 259218 (cherry picked from commit 362c6d8dec6dfbedc5717e45769042279a1dd3df) sys/dev/random/nehemiah.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Follow up. The little VIA Eden Esther 7F4K1G2E-LF board is running fine for a week now. At the very least it is walking briskly. Managed to perform a buildworld and buildkernel. The buildworld required a bit more than 7 days and 8 hours : esther# /usr/bin/time -p /usr/bin/nice -n +18 make buildworld . . . cc -target i386-unknown-freebsd14.0 --sysroot=/usr/obj/usr/src/i386.i386/tmp -B/usr/obj/usr/src/i386.i386/tmp/usr/bin -O2 -pipe -fno-common -g -gz=zlib -std=gnu99 -Wno-format-zero-length -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -o shared_shadow_inval_test.full shared_shadow_inval_test.o -lprivateatf-c objcopy --only-keep-debug shared_shadow_inval_test.full shared_shadow_inval_test.debug objcopy --strip-debug --add-gnu-debuglink=shared_shadow_inval_test.debug shared_shadow_inval_test.full shared_shadow_inval_test ===> tests/sys/vmm (all) echo '#! /usr/libexec/atf-sh' > vmm_cred_jail.tmp cat /usr/src/tests/sys/vmm/vmm_cred_jail.sh >>vmm_cred_jail.tmp chmod +x vmm_cred_jail.tmp mv vmm_cred_jail.tmp vmm_cred_jail ===> etc (all) ===> etc/sendmail (all) rm -f freebsd.cf m4 -D_CF_DIR_=/usr/src/contrib/sendmail/cf/ -D_NO_MAKEINFO_ /usr/src/contrib/sendmail/cf/m4/cf.m4 /usr/src/etc/sendmail/freebsd.mc > freebsd.cf chmod 444 freebsd.cf rm -f freebsd.submit.cf m4 -D_CF_DIR_=/usr/src/contrib/sendmail/cf/ -D_NO_MAKEINFO_ /usr/src/contrib/sendmail/cf/m4/cf.m4 /usr/src/etc/sendmail/freebsd.submit.mc > freebsd.submit.cf chmod 444 freebsd.submit.cf 83844.82 real 75910.18 user 5317.98 sys -------------------------------------------------------------- >>> World build completed on Fri Oct 29 09:14:59 GMT 2021 >>> World built in 634489 seconds, ncpu: 1 -------------------------------------------------------------- real 634499.23 user 599593.08 sys 21475.68 You have new mail. esther# At times it seems to get "stuck" while doing installworld and hitting the SIGINFO ( CTRL - T ) on the console would report : load: 11.40 cmd: install 14225 [runnable] 2562.09r 0.00u 0.25s 0% 1368k mi_switch+0x1ca critical_exit_preempt+0x5d __stop_set_sysinit_set+0xe3c01df7 ll+0x7 Then it would seem to gather its wits and resume. esther# uname -apKU FreeBSD esther 14.0-CURRENT FreeBSD 14.0-CURRENT #2 main-n250207-3b5331dd8d4-dirty: Fri Oct 29 16:54:01 GMT 2021 root@esther:/usr/obj/usr/src/i386.i386/sys/GENERIC i386 i386 1400039 1400039 esther# Seems perfectly stable and slow. -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional
As a follow up the machine seems to "freeze" when asked to perform any sort of load. There is no output at the console. The machine does not respond to anything other than a ping. I even tried a SIGINFO on the serial console and nothing happens. The only action I can take is to pull out the power.
Further follow up I did try to run some pkg commands on that box and within a few minutes it would freeze up however a SIGINFO to the serial console reveals ever increasing load : Loading kernel... /boot/kernel/kernel text=0xfecb0 text=0xb49980 text=0x5f6920 data=0xb0 data=0x10e08c+0x332244 syms=[0x4+0xed0a0+0x4+0x184b6c] Loading configured modules... /boot/kernel/cryptodev.ko text=0x20a0 text=0x2113 data=0x228+0x8 syms=[0x4+0x8c0+0x4+0xa94] /boot/entropy size=0x1000 /etc/hostid size=0x25 GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb ---<<BOOT>>--- MP Configuration Table version 1.4 found at 0x4f0c00 APIC: Using the MADT enumerator. Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.0-CURRENT #2 main-n250207-3b5331dd8d4-dirty: Fri Oct 29 16:54:01 GMT 2021 root@esther:/usr/obj/usr/src/i386.i386/sys/GENERIC i386 FreeBSD clang version 12.0.1 (git@github.com:llvm/llvm-project.git llvmorg-12.0.1-0-gfed41342a82f) WARNING: WITNESS option enabled, expect reduced performance. . . . Setting up harvesting: PURE_NEHEMIAH,[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: . Setting hostname: esther. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/perl5/5.32/mach/CORE lo0: link state changed to UP re0: link state changed to UP re0: link state changed to DOWN re0: link state changed to UP Starting Network: lo0 re0. lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE> ether 00:30:18:b0:29:b6 inet 172.16.35.59 netmask 0xffffffc0 broadcast 172.16.35.63 media: Ethernet autoselect (1000baseT <full-duplex,master>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> Starting devd. add host 127.0.0.1: gateway lo0 fib 0: route already in table add net default: gateway 172.16.35.1 add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Updating motd:. Clearing /tmp. Creating and/or trimming log files. Updating /var/run/os-release done. Starting syslogd. No core dumps found. Mounting late filesystems:. Security policy loaded: MAC/ntpd (mac_ntpd) Starting ntpd. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting sendmail_submit. Starting sendmail_msp_queue. Starting cron. Starting background file system checks in 60 seconds. Sun Nov 7 05:10 FreeBSD/i386 (esther) (ttyu0) login: root Password: Nov 7 05:11:08 esther login[729]: ROOT LOGIN (root) ON ttyu0 Last login: Sun Nov 7 01:52:30 on ttyu0 FreeBSD 14.0-CURRENT (GENERIC) #2 main-n250207-3b5331dd8d4-dirty: Fri Oct 29 16:54:01 GMT 2021 Welcome to FreeBSD! Release Notes, Errata: https://www.FreeBSD.org/releases/ Security Advisories: https://www.FreeBSD.org/security/ FreeBSD Handbook: https://www.FreeBSD.org/handbook/ FreeBSD FAQ: https://www.FreeBSD.org/faq/ Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/ FreeBSD Forums: https://forums.FreeBSD.org/ Documents installed with the system are in the /usr/local/share/doc/freebsd/ directory, or can be installed later with: pkg install en-freebsd-doc For other languages, replace "en" with a language code like de or fr. Show the version of FreeBSD installed: freebsd-version ; uname -a Please include that output and any error messages when posting questions. Introduction to manual pages: man man FreeBSD directory layout: man hier To change this login announcement, see motd(5). You have mail. esther# ls log readme.jitterbug readme.pkg readme.src esther# cat log/readme.pkg from Steve Wills pkg upgrade -Ffy pkg upgrade -fy pkg autoremove -y pkg clean -ya esther# esther# esther# pkg upgrade -Ffy Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. Checking for upgrades (69 candidates): 100% Processing candidates (69 candidates): 100% Checking integrity... done (0 conflicting) The following 69 package(s) will be affected (of 0 checked): Installed packages to be UPGRADED: boost-libs: 1.72.0_5 -> 1.72.0_6 git: 2.33.0 -> 2.33.1 libnghttp2: 1.45.1 -> 1.46.0 Installed packages to be REINSTALLED: bitmap-1.0.9 ca_root_nss-3.71 curl-7.79.1 expat-2.4.1 fontconfig-2.13.94_1,1 freetype2-2.11.0 gdb-11.1_1 gettext-runtime-0.21 gettext-tools-0.21 gmp-6.2.1 icu-70.1,1 indexinfo-0.3.1 libICE-1.0.10,1 libSM-1.2.3,1 libX11-1.7.2,1 libXau-1.0.9 libXaw-1.0.14,2 libXdmcp-1.1.3 libXext-1.3.4,1 libXft-2.3.3 libXmu-1.1.3,1 libXpm-3.5.13 libXrender-0.9.10_2 libXt-1.2.1,1 libffi-3.3_1 libiconv-1.16 libpthread-stubs-0.4 libssh2-1.9.0_3,3 libtextstyle-0.21 libxcb-1.14_1 mpc-1.2.1 mpdecimal-2.5.1 mpfr-4.1.0_1 p5-Authen-SASL-2.16_1 p5-CGI-4.53 p5-Clone-0.45 p5-Digest-HMAC-1.04 p5-Encode-Locale-1.05 p5-Error-0.17029 p5-GSSAPI-0.28_1 p5-HTML-Parser-3.76_1 p5-HTML-Tagset-3.20_1 p5-HTTP-Date-6.05 p5-HTTP-Message-6.33 p5-IO-HTML-1.004 p5-IO-Socket-INET6-2.72_1 p5-IO-Socket-SSL-2.072 p5-LWP-MediaTypes-6.04 p5-Mozilla-CA-20211001 p5-Net-SSLeay-1.90 p5-Socket6-0.29 p5-TimeDate-2.33,1 p5-URI-5.10 pcre2-10.37 perl5-5.32.1_1 pkg-1.17.3 png-1.6.37_1 python38-3.8.12_1 readline-8.1.1 source-highlight-3.1.9_1 vim-8.2.3458 xauth-1.1 xbitmaps-1.1.2 xlogo-1.0.5 xorgproto-2021.4 xterm-369 Number of packages to be upgraded: 3 Number of packages to be reinstalled: 66 esther# pkg upgrade -fy Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. Checking for upgrades (69 candidates): 100% Processing candidates (69 candidates): 100% Checking integrity... done (0 conflicting) The following 69 package(s) will be affected (of 0 checked): Installed packages to be UPGRADED: boost-libs: 1.72.0_5 -> 1.72.0_6 git: 2.33.0 -> 2.33.1 libnghttp2: 1.45.1 -> 1.46.0 Installed packages to be REINSTALLED: bitmap-1.0.9 ca_root_nss-3.71 curl-7.79.1 expat-2.4.1 fontconfig-2.13.94_1,1 freetype2-2.11.0 gdb-11.1_1 gettext-runtime-0.21 gettext-tools-0.21 gmp-6.2.1 icu-70.1,1 indexinfo-0.3.1 libICE-1.0.10,1 libSM-1.2.3,1 libX11-1.7.2,1 libXau-1.0.9 libXaw-1.0.14,2 libXdmcp-1.1.3 libXext-1.3.4,1 libXft-2.3.3 libXmu-1.1.3,1 libXpm-3.5.13 libXrender-0.9.10_2 libXt-1.2.1,1 libffi-3.3_1 libiconv-1.16 libpthread-stubs-0.4 libssh2-1.9.0_3,3 libtextstyle-0.21 libxcb-1.14_1 mpc-1.2.1 mpdecimal-2.5.1 mpfr-4.1.0_1 p5-Authen-SASL-2.16_1 p5-CGI-4.53 p5-Clone-0.45 p5-Digest-HMAC-1.04 p5-Encode-Locale-1.05 p5-Error-0.17029 p5-GSSAPI-0.28_1 p5-HTML-Parser-3.76_1 p5-HTML-Tagset-3.20_1 p5-HTTP-Date-6.05 p5-HTTP-Message-6.33 p5-IO-HTML-1.004 p5-IO-Socket-INET6-2.72_1 p5-IO-Socket-SSL-2.072 p5-LWP-MediaTypes-6.04 p5-Mozilla-CA-20211001 p5-Net-SSLeay-1.90 p5-Socket6-0.29 p5-TimeDate-2.33,1 p5-URI-5.10 pcre2-10.37 perl5-5.32.1_1 pkg-1.17.3 png-1.6.37_1 python38-3.8.12_1 readline-8.1.1 source-highlight-3.1.9_1 vim-8.2.3458 xauth-1.1 xbitmaps-1.1.2 xlogo-1.0.5 xorgproto-2021.4 xterm-369 Number of packages to be upgraded: 3 Number of packages to be reinstalled: 66 [1/69] Reinstalling xorgproto-2021.4... [1/69] Extracting xorgproto-2021.4: 100% [2/69] Reinstalling perl5-5.32.1_1... [2/69] Extracting perl5-5.32.1_1: 100% [3/69] Reinstalling libpthread-stubs-0.4... [3/69] Extracting libpthread-stubs-0.4: 100% [4/69] Reinstalling libXdmcp-1.1.3... [4/69] Extracting libXdmcp-1.1.3: 100% [5/69] Reinstalling libXau-1.0.9... [5/69] Extracting libXau-1.0.9: 100% [6/69] Reinstalling p5-TimeDate-2.33,1... [6/69] Extracting p5-TimeDate-2.33,1: 100% [7/69] Reinstalling libxcb-1.14_1... [7/69] Extracting libxcb-1.14_1: 100% [8/69] Reinstalling libICE-1.0.10,1... [8/69] Extracting libICE-1.0.10,1: 100% [9/69] Reinstalling png-1.6.37_1... [9/69] Extracting png-1.6.37_1: 100% [10/69] Reinstalling p5-URI-5.10... [10/69] Extracting p5-URI-5.10: 100% [11/69] Reinstalling p5-LWP-MediaTypes-6.04... [11/69] Extracting p5-LWP-MediaTypes-6.04: 100% [12/69] Reinstalling p5-IO-HTML-1.004... [12/69] Extracting p5-IO-HTML-1.004: 100% [13/69] Reinstalling p5-HTTP-Date-6.05... [13/69] Extracting p5-HTTP-Date-6.05: 100% [14/69] Reinstalling p5-Encode-Locale-1.05... [14/69] Extracting p5-Encode-Locale-1.05: 100% [15/69] Reinstalling p5-Clone-0.45... [15/69] Extracting p5-Clone-0.45: 100% [16/69] Reinstalling libX11-1.7.2,1... [16/69] Extracting libX11-1.7.2,1: 100% [17/69] Reinstalling libSM-1.2.3,1... [17/69] Extracting libSM-1.2.3,1: 100% [18/69] Reinstalling indexinfo-0.3.1... [18/69] Extracting indexinfo-0.3.1: 100% [19/69] Reinstalling p5-Socket6-0.29... [19/69] Extracting p5-Socket6-0.29: 100% [20/69] Reinstalling p5-HTTP-Message-6.33... [20/69] Extracting p5-HTTP-Message-6.33: 100% [21/69] Reinstalling p5-HTML-Tagset-3.20_1... [21/69] Extracting p5-HTML-Tagset-3.20_1: 100% [22/69] Reinstalling libXt-1.2.1,1... [22/69] Extracting libXt-1.2.1,1: 100% [23/69] Reinstalling libXext-1.3.4,1... [23/69] Extracting libXext-1.3.4,1: 100% [24/69] Reinstalling icu-70.1,1... [24/69] Extracting icu-70.1,1: 100% [25/69] Reinstalling gettext-runtime-0.21... [25/69] Extracting gettext-runtime-0.21: 100% [26/69] Reinstalling freetype2-2.11.0... [26/69] Extracting freetype2-2.11.0: 100% [27/69] Reinstalling expat-2.4.1... [27/69] Extracting expat-2.4.1: 100% [28/69] Reinstalling readline-8.1.1... [28/69] Extracting readline-8.1.1: 100% [29/69] Reinstalling p5-Net-SSLeay-1.90... [29/69] Extracting p5-Net-SSLeay-1.90: 100% [30/69] Reinstalling p5-Mozilla-CA-20211001... [30/69] Extracting p5-Mozilla-CA-20211001: 100% [31/69] Reinstalling p5-IO-Socket-INET6-2.72_1... [31/69] Extracting p5-IO-Socket-INET6-2.72_1: 100% [32/69] Reinstalling p5-HTML-Parser-3.76_1... [32/69] Extracting p5-HTML-Parser-3.76_1: 100% [33/69] Reinstalling p5-GSSAPI-0.28_1... [33/69] Extracting p5-GSSAPI-0.28_1: 100% [34/69] Reinstalling p5-Digest-HMAC-1.04... [34/69] Extracting p5-Digest-HMAC-1.04: 100% [35/69] Reinstalling mpdecimal-2.5.1... [35/69] Extracting mpdecimal-2.5.1: 100% [36/69] Reinstalling libssh2-1.9.0_3,3... [36/69] Extracting libssh2-1.9.0_3,3: 100% [37/69] Upgrading libnghttp2 from 1.45.1 to 1.46.0... [37/69] Extracting libnghttp2-1.46.0: 100% [38/69] Reinstalling libffi-3.3_1... [38/69] Extracting libffi-3.3_1: 100% [39/69] Reinstalling libXrender-0.9.10_2... [39/69] Extracting libXrender-0.9.10_2: 100% [40/69] Reinstalling libXpm-3.5.13... [40/69] Extracting libXpm-3.5.13: 100% [41/69] Reinstalling libXmu-1.1.3,1... [41/69] Extracting libXmu-1.1.3,1: 100% [42/69] Reinstalling gmp-6.2.1... [42/69] Extracting gmp-6.2.1: 100% [43/69] Reinstalling fontconfig-2.13.94_1,1... [43/69] Extracting fontconfig-2.13.94_1,1: 100% Running fc-cache to build fontconfig cache... Font directories: /usr/local/share/fonts /usr/local/lib/X11/fonts /usr/local/share/fonts: skipping, no such directory /usr/local/lib/X11/fonts: skipping, no such directory /var/db/fontconfig: cleaning cache directory fc-cache: succeeded [44/69] Reinstalling ca_root_nss-3.71... [44/69] Extracting ca_root_nss-3.71: 100% [45/69] Upgrading boost-libs from 1.72.0_5 to 1.72.0_6... [45/69] Extracting boost-libs-1.72.0_6: 75% load: 10.29 cmd: pkg 747 [runnable] 1253.05r 124.04u 131.78s 0% 275308k mi_switch+0x1ca critical_exit_preempt+0x5d uma_zfree_arg+0x44f bucket_free+0x6a cache_alloc+0x45a cache_alloc_retry+0x18 uma_zalloc_arg+0x159 vm_page_alloc_noobj_domain+0x97 vm_page_alloc_noobj+0x3b allocbuf+0x6c7 geteblk+0x78 softdep_process_journal+0x1e8 jwait+0xac softdep_update_inodeblock+0x18f ffs_update+0x291 ffs_truncate+0x541 ufs_setattr+0x674 VOP_SETATTR_APV+0x2d load: 12.13 cmd: pkg 747 [runnable] 1820.46r 124.04u 131.78s 0% 275308k mi_switch+0x1ca critical_exit_preempt+0x5d uma_zfree_arg+0x44f bucket_free+0x6a cache_alloc+0x45a cache_alloc_retry+0x18 uma_zalloc_arg+0x159 vm_page_alloc_noobj_domain+0x97 vm_page_alloc_noobj+0x3b allocbuf+0x6c7 geteblk+0x78 softdep_process_journal+0x1e8 jwait+0xac softdep_update_inodeblock+0x18f ffs_update+0x291 ffs_truncate+0x541 ufs_setattr+0x674 VOP_SETATTR_APV+0x2d load: 12.29 cmd: pkg 747 [runnable] 2746.55r 124.04u 131.78s 0% 275308k mi_switch+0x1ca critical_exit_preempt+0x5d uma_zfree_arg+0x44f bucket_free+0x6a cache_alloc+0x45a cache_alloc_retry+0x18 uma_zalloc_arg+0x159 vm_page_alloc_noobj_domain+0x97 vm_page_alloc_noobj+0x3b allocbuf+0x6c7 geteblk+0x78 softdep_process_journal+0x1e8 jwait+0xac softdep_update_inodeblock+0x18f ffs_update+0x291 ffs_truncate+0x541 ufs_setattr+0x674 VOP_SETATTR_APV+0x2d load: 12.30 cmd: pkg 747 [runnable] 9713.15r 124.04u 131.78s 0% 275308k mi_switch+0x1ca critical_exit_preempt+0x5d uma_zfree_arg+0x44f bucket_free+0x6a cache_alloc+0x45a cache_alloc_retry+0x18 uma_zalloc_arg+0x159 vm_page_alloc_noobj_domain+0x97 vm_page_alloc_noobj+0x3b allocbuf+0x6c7 geteblk+0x78 softdep_process_journal+0x1e8 jwait+0xac softdep_update_inodeblock+0x18f ffs_update+0x291 ffs_truncate+0x541 ufs_setattr+0x674 VOP_SETATTR_APV+0x2d load: 12.30 cmd: pkg 747 [runnable] 45533.10r 124.04u 131.78s 0% 275308k mi_switch+0x1ca critical_exit_preempt+0x5d uma_zfree_arg+0x44f bucket_free+0x6a cache_alloc+0x45a cache_alloc_retry+0x18 uma_zalloc_arg+0x159 vm_page_alloc_noobj_domain+0x97 vm_page_alloc_noobj+0x3b allocbuf+0x6c7 geteblk+0x78 softdep_process_journal+0x1e8 jwait+0xac softdep_update_inodeblock+0x18f ffs_update+0x291 ffs_truncate+0x541 ufs_setattr+0x674 VOP_SETATTR_APV+0x2d load: 12.41 cmd: pkg 747 [runnable] 46715.76r 124.04u 131.78s 0% 275308k mi_switch+0x1ca critical_exit_preempt+0x5d uma_zfree_arg+0x44f bucket_free+0x6a cache_alloc+0x45a cache_alloc_retry+0x18 uma_zalloc_arg+0x159 vm_page_alloc_noobj_domain+0x97 vm_page_alloc_noobj+0x3b allocbuf+0x6c7 geteblk+0x78 softdep_process_journal+0x1e8 jwait+0xac softdep_update_inodeblock+0x18f ffs_update+0x291 ffs_truncate+0x541 ufs_setattr+0x674 VOP_SETATTR_APV+0x2d At this point I have to pull power as the system will never recover.
Can we close this as "fixed" in the compiler?
(In reply to Dennis Clarke from comment #21) I suspect that this hang is fixed by https://cgit.freebsd.org/src/commit/?id=d7acbe481d17ccb81c2b879b9731c83b018f3094 based on the fact that the dmesg indicates the kernel is in the affected range of revisions, and based on the kernel stacks printed by the SIGINFO handler.
A commit in branch stable/12 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=b2e306164faa10ab0be957378a5344f45f7288de commit b2e306164faa10ab0be957378a5344f45f7288de Author: Konstantin Belousov <kib@FreeBSD.org> AuthorDate: 2021-10-20 22:42:05 +0000 Commit: Konstantin Belousov <kib@FreeBSD.org> CommitDate: 2021-12-09 00:31:53 +0000 nehemiah: manually assemble xstore(-rng) PR: 259218 (cherry picked from commit 362c6d8dec6dfbedc5717e45769042279a1dd3df) sys/dev/random/nehemiah.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)