Bug 275523

This panic was on RISC-V hardware (real hardware), but the panic may not be RISC-V only.  Hardware does pass 48h of memtester (so far).

Kernel core file and symbol table available on request, Full core.txt available at:

[2:6:306]dgilbert@ump:/var/crash> uname -a
FreeBSD ump.daveg.ca 15.0-CURRENT FreeBSD 15.0-CURRENT #1 main-n266101-608da65de955-dirty: Wed Oct 25 02:49:32 EDT 2023     root@ump.daveg.ca:/usr/obj/usr/src/riscv.riscv64/sys/GENERIC riscv

https://termbin.com/y9g4

Unread portion of the kernel message buffer:
panic: Bad link elm 0xffffffd050ee24f8 next->prev != elm
cpuid = 0
time = 1699626334
KDB: stack backtrace:
db_trace_self() at db_trace_self
db_trace_self_wrapper() at db_trace_self_wrapper+0x36
kdb_backtrace() at kdb_backtrace+0x2c
vpanic() at vpanic+0x116
panic() at panic+0x26
do_osd_del() at do_osd_del+0x378
osd_del() at osd_del+0x5c
khelp_destroy_osd() at khelp_destroy_osd+0x78
tcp_discardcb() at tcp_discardcb+0x96
tcp_usr_detach() at tcp_usr_detach+0x4e
sorele_locked() at sorele_locked+0xce
tcp_close() at tcp_close+0x1d0
tcp_timer_2msl() at tcp_timer_2msl+0x132
tcp_timer_enter() at tcp_timer_enter+0x11e
softclock_call_cc() at softclock_call_cc+0x112
softclock_thread() at softclock_thread+0x9e
fork_exit() at fork_exit+0x68
fork_trampoline() at fork_trampoline+0xa
KDB: enter: panic

get_curthread () at /usr/src/sys/riscv/include/pcpu.h:71
71		__asm __volatile("ld %0, 0(tp)" : "=&r"(td));
(kgdb) #0  get_curthread () at /usr/src/sys/riscv/include/pcpu.h:71
#1  doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:405
#2  0xffffffc0000e2dda in db_dump (dummy=<optimized out>, 
    dummy2=<optimized out>, dummy3=<optimized out>, dummy4=<optimized out>)
    at /usr/src/sys/ddb/db_command.c:591
#3  0xffffffc0000e2bea in db_command (last_cmdp=<optimized out>, 
    cmd_table=<optimized out>, dopager=true)
    at /usr/src/sys/ddb/db_command.c:504
#4  0xffffffc0000e296a in db_command_loop ()
    at /usr/src/sys/ddb/db_command.c:551
#5  0xffffffc0000e5c32 in db_trap (type=<optimized out>, 
    type@entry=<error reading variable: value is not available>, 
    code=<optimized out>, 
    code@entry=<error reading variable: value is not available>)
    at /usr/src/sys/ddb/db_main.c:268
#6  0xffffffc00033825c in kdb_trap (type=3, code=0, tf=<optimized out>)
    at /usr/src/sys/kern/subr_kdb.c:790
#7  0xffffffc0005b1d88 in do_trap_supervisor (frame=0xffffffc0043bc690)
    at /usr/src/sys/riscv/riscv/trap.c:359
#8  <signal handler called>
#9  kdb_enter (why=<optimized out>, msg=<optimized out>)
    at /usr/src/sys/kern/subr_kdb.c:556
#10 0xffffffc0002f44d2 in vpanic (
    fmt=0xffffffc00062846a "Bad link elm %p next->prev != elm", 
    ap=0xffffffc0043bc848) at /usr/src/sys/kern/kern_shutdown.c:958
#11 0xffffffc0002f42ce in panic (
    fmt=0x12 <error: Cannot access memory at address 0x12>)
    at /usr/src/sys/kern/kern_shutdown.c:894
#12 0xffffffc0002d5b56 in do_osd_del (type=2, osd=0xffffffd050ee24f8, slot=1, 
    list_locked=0) at /usr/src/sys/kern/kern_osd.c:346
#13 0xffffffc0002d61f2 in osd_del (type=2, osd=0x1, slot=37)
    at /usr/src/sys/kern/kern_osd.c:311
#14 0xffffffc0002be3da in khelp_remove_osd (
    h=0xffffffc0007c4b30 <ertt_helper>, hosd=0xffffffd050ee24f8)
    at /usr/src/sys/kern/kern_khelp.c:223
#15 khelp_destroy_osd (hosd=0xffffffd050ee24f8)
    at /usr/src/sys/kern/kern_khelp.c:203
#16 0xffffffc00045878a in tcp_discardcb (tp=0xffffffd050ee2000)
    at /usr/src/sys/netinet/tcp_subr.c:2416
#17 0xffffffc000463d44 in tcp_usr_detach (so=0xffffffd33d624000)
    at /usr/src/sys/netinet/tcp_usrreq.c:217
#18 0xffffffc000386060 in sofree (so=0xffffffd33d624000)
    at /usr/src/sys/kern/uipc_socket.c:1211
#19 sorele_locked (so=0xffffffd33d624000)
    at /usr/src/sys/kern/uipc_socket.c:1238
#20 0xffffffc0004586ca in tcp_close (tp=<optimized out>)
    at /usr/src/sys/netinet/tcp_subr.c:2541
#21 0xffffffc0004627ae in tcp_timer_2msl (tp=0xffffffd050ee2000)
    at /usr/src/sys/netinet/tcp_timer.c:373
#22 0xffffffc00046130c in tcp_timer_enter (xtp=0xffffffd050ee2000)
    at /usr/src/sys/netinet/tcp_timer.c:880
#23 0xffffffc00030f58e in softclock_call_cc (c=0xffffffd050ee2198, 
    cc=0xffffffc003e6a0c0, direct=0) at /usr/src/sys/kern/kern_timeout.c:719
#24 0xffffffc000310b4c in softclock_thread (arg=0xffffffc003e6a0c0)
    at /usr/src/sys/kern/kern_timeout.c:858
#25 0xffffffc0002b1c9c in fork_exit (
    callout=0xffffffc000310aaa <softclock_thread>, arg=0xffffffc003e6a0c0, 
    frame=0xffffffc0043bcc50) at /usr/src/sys/kern/kern_fork.c:1160
#26 0xffffffc0005b1aee in fork_trampoline ()
    at /usr/src/sys/riscv/riscv/swtch.S:370
Backtrace stopped: frame did not save the PC
(kgdb)