I found dialing multiple ppp profiles may make kernel panic. And I found the panic rate is related to PCIe-to-SATA adapter. Without the adapter, the panic rate is low, maybe less than 10%. With the adapter, the panic rate is pretty high, maybe more than 90%. ahci0: <ASMedia ASM1061 AHCI SATA controller> port 0xcf00-0xcf07,0xce00-0xce03,0xcd00-0xcd07,0xcc00-0xcc03,0xcb00-0xcb1f mem 0xfbeff000-0xfbeff1ff irq 16 at device 0.0 on pci1 ahci0: AHCI v1.20 with 2 6Gbps ports, Port Multiplier supported (I have tested other adapters (different brand) and the panic rate is still high) # kgdb kernel.debug /usr/crash/vmcore.9 ..skip some lines... <118>Starting PPP profile: hinet_static hi1 hi2 WARNING: attempt to domain_add(netgraph) after domainfinalize() Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 04 fault virtual address = 0x378 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80934881 stack pointer = 0x28:0xfffffe045f3d7930 frame pointer = 0x28:0xfffffe045f3d79b0 code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 656 (ppp) trap number = 12 panic: page fault cpuid = 2 KDB: stack backtrace: #0 0xffffffff80988d50 at kdb_backtrace+0x60 #1 0xffffffff8094a865 at panic+0x155 #2 0xffffffff80dce2ef at trap_fatal+0x38f #3 0xffffffff80dce63c at trap_pfault+0x33c #4 0xffffffff80dcdc7a at trap+0x47a #5 0xffffffff80db1102 at calltrap+0x8 #6 0xffffffff809347ba at __mtx_lock_flags+0x5a #7 0xffffffff81e2a5d3 at ng_attach_common+0x73 #8 0xffffffff81e2a673 at ngc_attach+0x43 #9 0xffffffff809c7fbf at socreate+0x1af #10 0xffffffff809cfe77 at sys_socket+0xf7 #11 0xffffffff80dced0b at amd64_syscall+0x3fb #12 0xffffffff80db13eb at Xfast_syscall+0xfb ..skip some lines... #0 doadump (textdump=<value optimized out>) at pcpu.h:219 219 __asm("movq %%gs:%1,%0" : "=r" (td) (kgdb) bt #0 doadump (textdump=<value optimized out>) at pcpu.h:219 #1 0xffffffff8094a3e8 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:449 #2 0xffffffff8094a8a4 in panic (fmt=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:756 #3 0xffffffff80dce2ef in trap_fatal (frame=<value optimized out>, eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:882 #4 0xffffffff80dce63c in trap_pfault (frame=0xfffffe045f72a880, usermode=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:693 #5 0xffffffff80dcdc7a in trap (frame=0xfffffe045f72a880) at /usr/src/sys/amd64/amd64/trap.c:457 #6 0xffffffff80db1102 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:231 #7 0xffffffff80934881 in __mtx_lock_sleep (c=0xffffffff81e2bc40, tid=18446735278230332560, opts=0, file=0x200 <Address 0x200 out of bounds>, line=5005456) at /usr/src/sys/kern/kern_mutex.c:430 #8 0xffffffff809347ba in __mtx_lock_flags (c=<value optimized out>, opts=613803152, file=0x0, line=512) at /usr/src/sys/kern/kern_mutex.c:223 #9 0xffffffff81e2a5d3 in ng_attach_common (so=0xfffff800248052b8, type=2) at /usr/src/sys/modules/netgraph/socket/../../../netgraph/ng_socket.c:605 #10 0xffffffff81e2a673 in ngc_attach (so=0xfffff800248052b8, proto=<value optimized out>, td=<value optimized out>) at /usr/src/sys/modules/netgraph/socket/../../../netgraph/ng_socket.c:538 #11 0xffffffff809c7fbf in socreate (dom=<value optimized out>, aso=0xfffffe045f72aab0, type=<value optimized out>, proto=2, cred=0xfffff8001a5bd300, td=0xfffff8002495e490) at /usr/src/sys/kern/uipc_socket.c:458 #12 0xffffffff809cfe77 in sys_socket (td=0xfffff8002495e490, uap=<value optimized out>) at /usr/src/sys/kern/uipc_syscalls.c:265 #13 0xffffffff80dced0b in amd64_syscall (td=0xfffff8002495e490, traced=0) at subr_syscall.c:133 #14 0xffffffff80db13eb in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:390 #15 0x0000000801e0fbca in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) up #1 0xffffffff8094a3e8 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:449 449 doadump(TRUE); (kgdb) up #2 0xffffffff8094a8a4 in panic (fmt=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:756 756 kern_reboot(bootopt); (kgdb) up #3 0xffffffff80dce2ef in trap_fatal (frame=<value optimized out>, eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:882 882 panic("%s", trap_msg[type]); (kgdb) up #4 0xffffffff80dce63c in trap_pfault (frame=0xfffffe045f72a880, usermode=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:693 693 trap_fatal(frame, eva); (kgdb) up #5 0xffffffff80dcdc7a in trap (frame=0xfffffe045f72a880) at /usr/src/sys/amd64/amd64/trap.c:457 457 (void) trap_pfault(frame, FALSE); (kgdb) up #6 0xffffffff80db1102 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:231 231 call trap Current language: auto; currently asm (kgdb) up #7 0xffffffff80934881 in __mtx_lock_sleep (c=0xffffffff81e2bc40, tid=18446735278230332560, opts=0, file=0x200 <Address 0x200 out of bounds>, line=5005456) at /usr/src/sys/kern/kern_mutex.c:430 430 owner = (struct thread *)(v & ~MTX_FLAGMASK); Current language: auto; currently minimal (kgdb) up #8 0xffffffff809347ba in __mtx_lock_flags (c=<value optimized out>, opts=613803152, file=0x0, line=512) at /usr/src/sys/kern/kern_mutex.c:223 223 __mtx_lock(m, curthread, opts, file, line); Fix: I found adding sleep between ppp dialing can work around this issue. --- a/rc.d/ppp +++ b/rc.d/ppp @@ -86,6 +86,7 @@ ppp_start() for _p in $_ppp_profile; do echo -n " $_p" ppp_start_profile $_p + sleep 2 done echo "." How-To-Repeat: Add multiple ppp profiles into rc.conf and reboot.
For bugs matching the following criteria: Status: In Progress Changed: (is less than) 2014-06-01 Reset to default assignee and clear in-progress tags. Mail being skipped