Description Dimitry Andric 2017-12-30 16:14:34 UTC

Created attachment 189224 [details]
/var/crash/core.txt.7

This is an i386 -CURRENT box, 12.0-CURRENT #0 r326912M: Sun Dec 17 23:24:21 CET 2017 to be precise, with a GENERIC kernel.  Building world from sources hosted on NFS (the NFS server is another FreeBSD 12.0-CURRENT VM, but an amd64 one which doesn't panic) usually triggers a panic fairly quickly:

Unread portion of the kernel message buffer:
panic: Assertion os.cidx != os.pidx_tail failed at /usr/src/sys/net/mp_ring.c:478
cpuid = 2
time = 1514640944
KDB: stack backtrace:
db_trace_self_wrapper(c12eb2d2,725f706d,2e676e69,37343a63,d3000a38,...) at db_trace_self_wrapper+0x2a/frame 0xd32b79c0
kdb_backtrace(c12e5bc6,5a479630,0,d32b7a8c,b,...) at kdb_backtrace+0x2d/frame 0xd32b7a28
vpanic(c124bb03,d32b7a8c,c124bb03,d32b7a8c,d32b7a8c,...) at vpanic+0x133/frame 0xd32b7a5c
kassert_panic(c124bb03,c12feba2,c12feb68,1de,126,...) at kassert_panic+0xd9/frame 0xd32b7a80
ifmp_ring_check_drainage(d63f9000,20,d63f6898,d31ed81c,d63f6800,...) at ifmp_ring_check_drainage+0xfa/frame 0xd32b7ac8
_task_fn_tx(d63f6800,0,c12eadc0,146,0,...) at _task_fn_tx+0x80/frame 0xd32b7af0
gtaskqueue_run_locked(d31ed800,d31ed81c,c1234d41,0,0,...) at gtaskqueue_run_locked+0x157/frame 0xd32b7b34
gtaskqueue_thread_loop(d3283c24,d32b7ba8,c12dde15,410,0,...) at gtaskqueue_thread_loop+0xa7/frame 0xd32b7b6c
fork_exit(c0c80300,d3283c24,d32b7ba8) at fork_exit+0x7e/frame 0xd32b7b94
fork_trampoline() at fork_trampoline+0x8/frame 0xd32b7b94
--- trap 0, eip = 0, esp = 0xd32b7be0, ebp = 0 ---

#0  __curthread () at ./machine/pcpu.h:216
#1  doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:349
#2  0xc057803e in db_dump (dummy=-1060628262, dummy2=false, dummy3=-1, dummy4=0xd32b77b4 "") at /usr/src/sys/ddb/db_command.c:574
#3  0xc0577e0b in db_command (last_cmdp=<optimized out>, cmd_table=<optimized out>, dopager=<optimized out>) at /usr/src/sys/ddb/db_command.c:481
#4  0xc0577b70 in db_command_loop () at /usr/src/sys/ddb/db_command.c:534
#5  0xc057ae5d in db_trap (type=<optimized out>, code=<optimized out>) at /usr/src/sys/ddb/db_main.c:250
#6  0xc0c8208e in kdb_trap (type=<optimized out>, code=<optimized out>, tf=0xd32b79dc) at /usr/src/sys/kern/subr_kdb.c:660
#7  0xc11f8629 in trap (frame=<optimized out>) at /usr/src/sys/i386/i386/trap.c:677
#8  <signal handler called>
#9  kdb_enter (why=0xc124c8d7 "panic", msg=<optimized out>) at /usr/src/sys/kern/subr_kdb.c:443
#10 0xc0c3b860 in vpanic (fmt=<optimized out>, ap=<optimized out>) at /usr/src/sys/kern/kern_shutdown.c:804
#11 0xc0c3b709 in kassert_panic (fmt=0xc124bb03 "Assertion %s failed at %s:%d") at /usr/src/sys/kern/kern_shutdown.c:701
#12 0xc0d57a1a in ifmp_ring_check_drainage (r=0xd63f9000, budget=<optimized out>) at /usr/src/sys/net/mp_ring.c:478
#13 0xc0d51520 in _task_fn_tx (context=<optimized out>) at /usr/src/sys/net/iflib.c:3643
#14 0xc0c806d7 in gtaskqueue_run_locked (queue=0xd31ed800) at /usr/src/sys/kern/subr_gtaskqueue.c:329
#15 0xc0c803a7 in gtaskqueue_thread_loop (arg=0xd3283c24) at /usr/src/sys/kern/subr_gtaskqueue.c:504
#16 0xc0bf9d4e in fork_exit (callout=0xc0c80300 <gtaskqueue_thread_loop>, arg=<optimized out>, frame=<optimized out>) at /usr/src/sys/kern/kern_fork.c:1048
#17 <signal handler called>
(kgdb) frame 12
#12 0xc0d57a1a in ifmp_ring_check_drainage (r=0xd63f9000, budget=<optimized out>) at /usr/src/sys/net/mp_ring.c:478
478		MPASS(os.cidx != os.pidx_tail);	/* implied by STALLED */
(kgdb) print os
$1 = <optimized out>
(kgdb) print /x r->state
$6 = 0x304c004c104c1
(kgdb) print /x *((union ring_state *)&r->state)
$9 = {{pidx_head = 0x4c1, pidx_tail = 0x4c1, cidx = 0x4c0, flags = 0x3}, state = 0x304c004c104c1}
(kgdb) up
#13 0xc0d51520 in _task_fn_tx (context=<optimized out>) at /usr/src/sys/net/iflib.c:3643
3643		ifmp_ring_check_drainage(txq->ift_br, TX_BATCH_SIZE);
(kgdb) print *txq->ift_br
$13 = {state = 849647690122433, size = 2048, cookie = 0xd63f6800, mt = 0xc18cf664 <M_IFLIB>,
  drain = 0xc0d53590 <iflib_txq_drain>, can_drain = 0xc0d55520 <iflib_txq_can_drain>, enqueues = 0xc5d6b2c0,
  drops = 0xc5d6b2b8, starts = 0xc5d6b2b0, stalls = 0xc5d6b2a8, restarts = 0xc5d6b2a0,
  abdications = 0xc5d6b298, items = 0xd63f9080}

As far as I can see, in ifmp_ring_check_drainage(), it copies r->state into os.state, and clearly r->state has a cidx value unequal to pidx_tail.  But where that r->state originally comes from, I don't know...

The vmcore.7 file is available upon request.