I get kernel panic on my freebsd 13 system: panic: vm_fault_lookup: fault on nofault entry, addr: 0 cpuid = 1 time = 1626721843 KDB: stack backtrace: #0 0x104318f at kdb_backtrace+0x4f #1 0xffc71a at vpanic+0x11a #2 0xffc5f4 at panic+0x14 #3 0x1257943 at vm_fault+0x1753 #4 0x1256127 at vm_fault_trap+0x87 #5 0x139f276 at trap_pfault+0x146 #6 0x139e841 at trap+0x381 #7 0xffc0319f at ip_divert_event_tag+0xe29cc977 #8 0x28 at ll+0x7 Uptime: 5h57m28s Automatic reboot in 15 seconds - press a key on the console to abort Rebooting... Intervals between panic events vary from several minutes to several tens of hours. I haven't seen my system running for more than two days. Version info & HW: FreeBSD 13.0-RELEASE CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz MB: INTEL D945GLF2 Video: agp0: <Intel 82945G (945G GMCH) SVGA controller> on vgapci0 Audio controller: hdac0: <Intel 82801G HDA Controller> mem 0x883a0000-0x883a3fff irq 22 at device 27.0 on pci0 Eth adapters: re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0x2000-0x20ff mem 0x88200000-0x88200fff,0x88000000-0x8800ffff irq 16 at device 0.0 on pci1 rl0: <RealTek 8139 10/100BaseTX> port 0x1000-0x10ff mem 0x88100000 -0x881000ff irq 21 at device 0.0 on pci2 HDD Controller: atapci0: <Intel ICH7 SATA300 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30a0-0x30af irq 19 at device 31.2 on pci0 HDD: ada0: <WDC WD5000AAKX-603CA0 18.01H18> ATA8-ACS SATA 3.x device Loaded modules: # kldstat Id Refs Address Size Name 1 10 0x800000 177e46c kernel 2 1 0x1d200000 6000 ichsmb.ko 3 1 0x1d206000 4000 smbus.ko 4 2 0x1d20a000 28000 ipfw.ko 5 1 0x1d232000 5000 ipdivert.ko
Do you have the latest 13.0 patches from freebsd-update? There was an ipdivert bug fix that could be related.
(In reply to Mark Johnston from comment #1) I am not sure if I have these. How can I check? Today I built a new kernel to test if this will change something - it had no effect.
I just managed to apply patches up to p3. freebsd-update output said: The following files will be updated as part of updating to 13.0-RELEASE-p3: /bin/freebsd-version /boot/kernel/if_vlan.ko /boot/kernel/ipdivert.ko /boot/kernel/ipfw.ko /boot/kernel/kernel /boot/kernel/linux.ko /boot/kernel/mpt.ko /boot/kernel/virtio_pci.ko /lib/libcasper.so.1 /usr/bin/bc /usr/bin/dc /usr/lib/debug/boot/kernel/if_vlan.ko.debug /usr/lib/debug/boot/kernel/ipdivert.ko.debug /usr/lib/debug/boot/kernel/ipfw.ko.debug /usr/lib/debug/boot/kernel/kernel.debug /usr/lib/debug/boot/kernel/linux.ko.debug /usr/lib/debug/boot/kernel/mpt.ko.debug /usr/lib/debug/boot/kernel/virtio_pci.ko.debug /usr/lib/libradius.a /usr/lib/libradius.so.4 /usr/lib/libradius_p.a /usr/src/contrib/bc/src/program.c /usr/src/lib/libc++/Makefile /usr/src/lib/libcasper/libcasper/libcasper_impl.c /usr/src/lib/libcasper/libcasper/libcasper_impl.h /usr/src/lib/libcasper/libcasper/service.c /usr/src/lib/libcasper/libcasper/zygote.c /usr/src/lib/libradius/radlib.c /usr/src/sys/amd64/amd64/support.S /usr/src/sys/amd64/linux/linux_support.s /usr/src/sys/amd64/linux32/linux32_support.s /usr/src/sys/compat/linux/linux_futex.c /usr/src/sys/conf/newvers.sh /usr/src/sys/dev/mpt/mpt_cam.c /usr/src/sys/dev/pms/freebsd/driver/ini/src/agdef.h /usr/src/sys/dev/pms/freebsd/driver/ini/src/agtiapi.c /usr/src/sys/dev/virtio/pci/virtio_pci_legacy.c /usr/src/sys/net/if_vlan.c /usr/src/sys/netinet/ip_divert.c /usr/src/sys/netpfil/ipfw/ip_fw_pfil.c /usr/src/tools/build/mk/OptionalObsoleteFiles.inc # uname -r -> 13.0-RELEASE-p3 I will report in a few days if patching did ellimnate my kernel panics.
(In reply to Deyan from comment #2) Which branch did you build the kernel from?
Hmm, ipdivert might be a red herring, I didn't notice the large offset before: #7 0xffc0319f at ip_divert_event_tag+0xe29cc977 Assuming that you are in fact using divert sockets though (e.g., some ipfw rule diverts packets to natd), it is worth ruling out that problem since it can cause random panics.
(In reply to Mark Johnston from comment #5) I have such a rule in ipfw: ${fw} 500 divert natd ip from any to any in via ${oif} my natd.conf has these lines: interface ${natd_interface} dynamic yes unregistered_only yes redirect_port tcp ... ... multiple lines as the one above follow
(In reply to Mark Johnston from comment #4) I suppose it was the 13.0 RELEASE I followed this manual: https://docs.freebsd.org/doc/6.1-RELEASE/usr/share/doc/handbook/kernelconfig-building.html Procedure 2. Building a Kernel the “New” Way
Have there been any crashes on 13.0-p3?
Hi, no crashes after my last comment so I assume this issue is no more present on p3. You may eventually close this ticket if no deep-digging to find the root cause is necessary.
(In reply to Deyan from comment #9) Thanks. I can't find another PR to dup this to, so I'll just mark it fixed. I suspect the panic was addressed by https://cgit.freebsd.org/src/commit/?id=a1fadf7de25b973a308b86d04c4ada4fa8be193f Since that can cause fairly random memory corruption I don't think it's worth digging deeper into this particular instance.
^Triage: Assign to committer that (tentatively/probably :)) resolved, tracke merges (In reply to Mark Johnston from comment #10) Did stable/12,11 need/receive the merge? I believe epoch stuff was 12, or 12/13 only? Set '-' for the ones that didn't, ta!
(In reply to Kubilay Kocak from comment #11) The bug is only in 13.0.