Bug 210724 - panic on vnet destroy on recent FreeBSD 11-ALPHA5: vnet or rw_wlock_hard related
Summary: panic on vnet destroy on recent FreeBSD 11-ALPHA5: vnet or rw_wlock_hard related
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Bjoern A. Zeeb
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-06-30 15:06 UTC by Oleg Ginzburg
Modified: 2018-01-10 07:54 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Oleg Ginzburg 2016-06-30 15:06:13 UTC
FreeBSD 11.0-ALPHA5 r302206

When destroy vnet-based interfaces, FreeBSD panic with Fatal trap 12:
--
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x3b0
fault code              = supervisor read data, page not present
instruction pointer     = 0x20:0xffffffff80ad4c8a
stack pointer           = 0x28:0xfffffe03df5376d0
frame pointer           = 0x28:0xfffffe03df537750
code segment            = base rx0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 0 (thread taskq)
trap number             = 12
panic: page fault
cpuid = 0
Uptime: 5h2m21s
Dumping 2715 out of 16225 MB: (CTRL-C to abort)  (CTRL-C to abort)  (CTRL-C to abort)  (CTRL-C to abort)  (CTRL-C to abort)  (CTRL-C to abort)  (CTRL-C to abort)  (CTRL-C to abort)  (CTRL-C to abort) ..1% (CTRL-C to abort)  (CTRL-C to abort)  (CTRL-C to abort) ..11% (CTRL-C to abort)  (CTRL-C to abort)  (CTRL-C to abort) ..21%..31%..41%..51%..61%..71% (CTRL-C to abort)  (CTRL-C to abort)  (CTRL-C to abort) ..81%..91%

Reading symbols from /boot/kernel/zfs.ko...done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...done.
Loaded symbols for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/pf.ko...done.
Loaded symbols for /boot/kernel/pf.ko
Reading symbols from /boot/kernel/coretemp.ko...done.
Loaded symbols for /boot/kernel/coretemp.ko
Reading symbols from /boot/kernel/aesni.ko...done.
Loaded symbols for /boot/kernel/aesni.ko
Reading symbols from /boot/kernel/sem.ko...done.
Loaded symbols for /boot/kernel/sem.ko
Reading symbols from /boot/kernel/cpuctl.ko...done.
Loaded symbols for /boot/kernel/cpuctl.ko
Reading symbols from /boot/kernel/cc_htcp.ko...done.
Loaded symbols for /boot/kernel/cc_htcp.ko
Reading symbols from /boot/kernel/ipfw_nat.ko...done.
Loaded symbols for /boot/kernel/ipfw_nat.ko
Reading symbols from /boot/kernel/ipfw.ko...done.
Loaded symbols for /boot/kernel/ipfw.ko
Reading symbols from /boot/kernel/libalias.ko...done.
Loaded symbols for /boot/kernel/libalias.ko
Reading symbols from /boot/kernel/linprocfs.ko...done.
Loaded symbols for /boot/kernel/linprocfs.ko
Reading symbols from /boot/kernel/linux_common.ko...done.
Loaded symbols for /boot/kernel/linux_common.ko
Reading symbols from /boot/kernel/linsysfs.ko...done.
Loaded symbols for /boot/kernel/linsysfs.ko
Reading symbols from /boot/kernel/fdescfs.ko...done.
Loaded symbols for /boot/kernel/fdescfs.ko
Reading symbols from /boot/kernel/if_bridge.ko...done.
Loaded symbols for /boot/kernel/if_bridge.ko
Reading symbols from /boot/kernel/bridgestp.ko...done.
Loaded symbols for /boot/kernel/bridgestp.ko
Reading symbols from /boot/kernel/if_tap.ko...done.
Loaded symbols for /boot/kernel/if_tap.ko
Reading symbols from /boot/kernel/vmm.ko...done.
Loaded symbols for /boot/kernel/vmm.ko
Reading symbols from /boot/kernel/nmdm.ko...done.
Loaded symbols for /boot/kernel/nmdm.ko
Reading symbols from /boot/modules/vboxdrv.ko...done.
Loaded symbols for /boot/modules/vboxdrv.ko
Reading symbols from /boot/kernel/linux64.ko...done.
Loaded symbols for /boot/kernel/linux64.ko
Reading symbols from /boot/modules/nvidia.ko...done.
Loaded symbols for /boot/modules/nvidia.ko
Reading symbols from /boot/modules/cuse4bsd.ko...done.
Loaded symbols for /boot/modules/cuse4bsd.ko
Reading symbols from /boot/kernel/uhid.ko...done.
Loaded symbols for /boot/kernel/uhid.ko
Reading symbols from /boot/kernel/snd_uaudio.ko...done.
Loaded symbols for /boot/kernel/snd_uaudio.ko
Reading symbols from /boot/kernel/ums.ko...done.
Loaded symbols for /boot/kernel/ums.ko
Reading symbols from /boot/modules/vboxnetflt.ko...done.
Loaded symbols for /boot/modules/vboxnetflt.ko
Reading symbols from /boot/kernel/netgraph.ko...done.
Loaded symbols for /boot/kernel/netgraph.ko
Reading symbols from /boot/kernel/ng_ether.ko...done.
Loaded symbols for /boot/kernel/ng_ether.ko
Reading symbols from /boot/modules/vboxnetadp.ko...done.
Loaded symbols for /boot/modules/vboxnetadp.ko
Reading symbols from /boot/kernel/linux.ko...done.
Loaded symbols for /boot/kernel/linux.ko
Reading symbols from /boot/kernel/ctl.ko...done.
Loaded symbols for /boot/kernel/ctl.ko
Reading symbols from /boot/kernel/iscsi.ko...done.
Loaded symbols for /boot/kernel/iscsi.ko
Reading symbols from /boot/kernel/nullfs.ko...done.
Loaded symbols for /boot/kernel/nullfs.ko
Reading symbols from /boot/kernel/if_epair.ko...done.
Loaded symbols for /boot/kernel/if_epair.ko
#0  doadump (textdump=<value optimized out>) at pcpu.h:221
221             __asm("movq %%gs:%1,%0" : "=r" (td)

----------------

(kgdb) list *0xffffffff80ad4c8a
0xffffffff80ad4c8a is in __rw_wlock_hard (/usr/src/sys/kern/kern_rwlock.c:793).
788                      * running on another CPU, spin until the owner stops
789                      * running or the state of the lock changes.
790                      */
791                     v = rw->rw_lock;
792                     owner = (struct thread *)RW_OWNER(v);
793                     if (!(v & RW_LOCK_READ) && TD_IS_RUNNING(owner)) {
794                             if (LOCK_LOG_TEST(&rw->lock_object, 0))
795                                     CTR3(KTR_LOCK, "%s: spinning on %p held by %p",
796                                         __func__, rw, owner);
797                             KTR_STATE1(KTR_SCHED, "thread", sched_tdname(curthread),
Current language:  auto; currently minimal


----------------


(kgdb) backtrace
#0  doadump (textdump=<value optimized out>) at pcpu.h:221
#1  0xffffffff80ad96f9 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366
#2  0xffffffff80ad9c71 in vpanic (fmt=<value optimized out>, ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:759
#3  0xffffffff80ad9ae3 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:690
#4  0xffffffff80fb5cd1 in trap_fatal (frame=0xfffffe03df537620, eva=944) at /usr/src/sys/amd64/amd64/trap.c:841
#5  0xffffffff80fb5f81 in trap_pfault (frame=0x0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:716
#6  0xffffffff80fb5460 in trap (frame=0xfffffe03df537620) at /usr/src/sys/amd64/amd64/trap.c:442
#7  0xffffffff80f98ca1 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236
#8  0xffffffff80ad4c8a in __rw_wlock_hard (c=<value optimized out>, tid=<value optimized out>, file=<value optimized out>, line=<value optimized out>)
    at /usr/src/sys/kern/kern_rwlock.c:792
#9  0xffffffff824ddc61 in ifaddr_change () from /boot/kernel/ipfw_nat.ko
#10 0xffffffff80c54284 in in_difaddr_ioctl (data=<value optimized out>, ifp=<value optimized out>, td=<value optimized out>) at /usr/src/sys/netinet/in.c:638
#11 0xffffffff80c5331f in in_control (so=<value optimized out>, cmd=<value optimized out>, data=<value optimized out>, ifp=0xfffff803106a9000, td=0x0)
    at /usr/src/sys/netinet/in.c:248
#12 0xffffffff80c54b2a in in_ifscrub_all () at /usr/src/sys/netinet/in.c:921
#13 0xffffffff80c64e7d in ip_destroy (unused=<value optimized out>) at /usr/src/sys/netinet/ip_input.c:399
#14 0xffffffff80c069dc in vnet_sysuninit () at /usr/src/sys/net/vnet.c:596
#15 0xffffffff80c0688a in vnet_destroy (vnet=0xfffff803240fa240) at /usr/src/sys/net/vnet.c:282
#16 0xffffffff80a99678 in prison_deref (pr=<value optimized out>, flags=0) at /usr/src/sys/kern/kern_jail.c:2693
#17 0xffffffff80b34a2a in taskqueue_run_locked (queue=<value optimized out>) at /usr/src/sys/kern/subr_taskqueue.c:465
#18 0xffffffff80b35938 in taskqueue_thread_loop (arg=<value optimized out>) at /usr/src/sys/kern/subr_taskqueue.c:719
#19 0xffffffff80a904d5 in fork_exit (callout=0xffffffff80b35850 <taskqueue_thread_loop>, arg=0xffffffff81dac430, frame=0xfffffe03df537ac0) at /usr/src/sys/kern/kern_fork.c:1038
#20 0xffffffff80f991de in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:611
#21 0x0000000000000000 in ?? ()
Comment 1 commit-hook freebsd_committer freebsd_triage 2016-06-30 17:17:07 UTC
A commit references this bug:

Author: bz
Date: Thu Jun 30 17:16:22 UTC 2016
New revision: 302300
URL: https://svnweb.freebsd.org/changeset/base/302300

Log:
  In case of the global eventhandler make sure the current VNET
  is still operational before doing any work;  otherwise we might
  run into, e.g., released locks.

  PR:		210724
  Reported by:	olevole olevole.ru
  Sponsored by:	The FreeBSD Foundation

Changes:
  projects/vnet/sys/netpfil/ipfw/ip_fw_nat.c
Comment 2 commit-hook freebsd_committer freebsd_triage 2016-06-30 19:33:25 UTC
A commit references this bug:

Author: bz
Date: Thu Jun 30 19:32:46 UTC 2016
New revision: 302302
URL: https://svnweb.freebsd.org/changeset/base/302302

Log:
  In case of the global eventhandler make sure the current VNET
  is still operational before doing any work;  otherwise we might
  run into, e.g., destroyed locks.

  PR:		210724
  Reported by:	olevole olevole.ru
  Sponsored by:	The FreeBSD Foundation
  MFC after:	2 weeks
  Obtained from:	projects/vnet
  Approved by:	re (gjb)

Changes:
  head/sys/netpfil/ipfw/ip_fw_nat.c
Comment 3 Oleg Ginzburg 2018-01-10 07:54:08 UTC
problem is solved