Bug 209466 - Dummynet Kernel Panic
Summary: Dummynet Kernel Panic
Status: Closed DUPLICATE of bug 222617
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 10.3-RELEASE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: Eugene Grosbein
URL:
Keywords: crash, needs-qa
Depends on:
Blocks:
 
Reported: 2016-05-12 15:51 UTC by Alexander
Modified: 2017-09-28 11:23 UTC (History)
5 users (show)

See Also:
koobs: mfc-stable10?
koobs: mfc-stable11?


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander 2016-05-12 15:51:57 UTC
I have only one pipe:
00900 21009791 25817826989 pipe tablearg ip from any to table(60) out
00905 13633970  2823363333 pipe tablearg ip from table(61) to any in
table 60 have about 200-300 lines:

Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x4000000388
fault code              = supervisor read data, page not present
instruction pointer     = 0x20:0xffffffff8094f1c9
stack pointer           = 0x28:0xfffffe011e26b960
frame pointer           = 0x28:0xfffffe011e26ba00
code segment            = base rx0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 0 (dummynet)
trap number             = 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff8098e390 at kdb_backtrace+0x60
#1 0xffffffff80951066 at vpanic+0x126
#2 0xffffffff80950f33 at panic+0x43
#3 0xffffffff80d55f7b at trap_fatal+0x36b
#4 0xffffffff80d5627d at trap_pfault+0x2ed
#5 0xffffffff80d558fa at trap+0x47a
#6 0xffffffff80d3b8d2 at calltrap+0x8
#7 0xffffffff80a84b67 at ip_input+0x4a7
#8 0xffffffff80a24092 at netisr_dispatch_src+0x62
#9 0xffffffff81a75f7c at dummynet_send+0x10c
#10 0xffffffff81a75b93 at dummynet_task+0x293
#11 0xffffffff8099f1c5 at taskqueue_run_locked+0xe5
#12 0xffffffff8099fc58 at taskqueue_thread_loop+0xa8
#13 0xffffffff8091a4ea at fork_exit+0x9a
#14 0xffffffff80d3be0e at fork_trampoline+0xe
Uptime: 5h23m9s
Dumping 371 out of 4062 MB:..5%..13%..22%..31%..44%..52%..61%..74%..82%..91%

Reading symbols from /boot/kernel/if_igb.ko.symbols...done.
Loaded symbols for /boot/kernel/if_igb.ko.symbols
Reading symbols from /boot/kernel/accf_http.ko.symbols...done.
Loaded symbols for /boot/kernel/accf_http.ko.symbols
Reading symbols from /boot/kernel/coretemp.ko.symbols...done.
Loaded symbols for /boot/kernel/coretemp.ko.symbols
Reading symbols from /boot/kernel/netgraph.ko.symbols...done.
Loaded symbols for /boot/kernel/netgraph.ko.symbols
Reading symbols from /boot/kernel/ng_ipfw.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_ipfw.ko.symbols
Reading symbols from /boot/kernel/ipfw.ko.symbols...done.
Loaded symbols for /boot/kernel/ipfw.ko.symbols
Reading symbols from /boot/kernel/ng_car.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_car.ko.symbols
Reading symbols from /boot/kernel/dummynet.ko.symbols...done.
Loaded symbols for /boot/kernel/dummynet.ko.symbols
Reading symbols from /boot/kernel/uhid.ko.symbols...done.
Loaded symbols for /boot/kernel/uhid.ko.symbols
Reading symbols from /boot/kernel/ums.ko.symbols...done.
Loaded symbols for /boot/kernel/ums.ko.symbols
Reading symbols from /boot/kernel/ng_socket.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_socket.ko.symbols
Reading symbols from /boot/kernel/ng_mppc.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_mppc.ko.symbols
Reading symbols from /boot/kernel/rc4.ko.symbols...done.
Loaded symbols for /boot/kernel/rc4.ko.symbols
Reading symbols from /boot/kernel/ng_ether.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_ether.ko.symbols
Reading symbols from /boot/kernel/ng_pppoe.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_pppoe.ko.symbols
Reading symbols from /boot/kernel/ng_tee.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_tee.ko.symbols
Reading symbols from /boot/kernel/ng_pptpgre.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_pptpgre.ko.symbols
Reading symbols from /boot/kernel/ng_ksocket.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_ksocket.ko.symbols
Reading symbols from /boot/kernel/ng_iface.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_iface.ko.symbols
Reading symbols from /boot/kernel/ng_ppp.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_ppp.ko.symbols
Reading symbols from /boot/kernel/ng_netflow.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_netflow.ko.symbols
Reading symbols from /boot/kernel/ng_tcpmss.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_tcpmss.ko.symbols
Reading symbols from /boot/kernel/ng_bpf.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_bpf.ko.symbols
Reading symbols from /boot/kernel/ng_vjc.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_vjc.ko.symbols
#0  doadump (textdump=<value optimized out>) at pcpu.h:219
219     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) #0  doadump (textdump=<value optimized out>) at pcpu.h:219
#1  0xffffffff80950cc2 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:486
#2  0xffffffff809510a5 in vpanic (fmt=<value optimized out>,
    ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:889
#3  0xffffffff80950f33 in panic (fmt=0x0)
    at /usr/src/sys/kern/kern_shutdown.c:818
#4  0xffffffff80d55f7b in trap_fatal (frame=<value optimized out>,
    eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:858
#5  0xffffffff80d5627d in trap_pfault (frame=0xfffffe011e26b8b0,
    usermode=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:681
#6  0xffffffff80d558fa in trap (frame=0xfffffe011e26b8b0)
    at /usr/src/sys/amd64/amd64/trap.c:447
#7  0xffffffff80d3b8d2 in calltrap ()
    at /usr/src/sys/amd64/amd64/exception.S:236
#8  0xffffffff8094f1c9 in __rw_rlock (c=0xfffff80066e52c38, file=0x0, line=0)
    at /usr/src/sys/kern/kern_rwlock.c:430
#9  0xffffffff80a84b67 in ip_input (m=0xfffff80033b42e00)
    at /usr/src/sys/netinet/ip_input.c:593
#10 0xffffffff80a24092 in netisr_dispatch_src (proto=<value optimized out>,
    source=<value optimized out>, m=0x0) at /usr/src/sys/net/netisr.c:976
#11 0xffffffff81a75f7c in dummynet_send (m=<value optimized out>)
    at /usr/src/sys/modules/dummynet/../../netpfil/ipfw/ip_dn_io.c:665
#12 0xffffffff81a75b93 in dummynet_task (context=<value optimized out>,
    pending=<value optimized out>)
    at /usr/src/sys/modules/dummynet/../../netpfil/ipfw/ip_dn_io.c:625
#13 0xffffffff8099f1c5 in taskqueue_run_locked (queue=0xfffff8000c4b1400)
    at /usr/src/sys/kern/subr_taskqueue.c:342
#14 0xffffffff8099fc58 in taskqueue_thread_loop (arg=<value optimized out>)
    at /usr/src/sys/kern/subr_taskqueue.c:563
#15 0xffffffff8091a4ea in fork_exit (
    callout=0xffffffff8099fbb0 <taskqueue_thread_loop>,
    arg=0xffffffff81a7a5b8, frame=0xfffffe011e26bc00)
    at /usr/src/sys/kern/kern_fork.c:1027
#16 0xffffffff80d3be0e in fork_trampoline ()
    at /usr/src/sys/amd64/amd64/exception.S:611
#17 0x0000000000000000 in ?? ()
Current language:  auto; currently minimal
(kgdb)

------------------------------------------------------------------------
ps -axlww

UID   PID  PPID CPU PRI NI    VSZ               RSS MWCHAN   STAT TT        TIME COMMAND
  0     0     0   0  -8  0      0 18005602445710912 -        DLs   -    10:45.79 [kernel]
  0     1     0   0  41  0   9492 17979214968265728 wait     DLs   -     0:00.01 [init]
  0     2     0   0 -16  0      0 18005602445710912 -        DL    -     0:02.37 [cam]
  0     3     0   0 -16  0      0 18005602445710912 waiting_ DL    -     0:00.01 [sctp_iterator]
  0     4     0   0 -16  0      0 18005602445710912 umarcl   DL    -     0:00.67 [pagedaemon]
  0     5     0   0 -16  0      0 18005602445710912 psleep   DL    -     0:00.00 [vmdaemon]
  0     6     0   0 155  0      0 18005602445710912 pgzero   DL    -     0:00.00 [pagezero]
  0     7     0   0  20  0      0 18005602445710912 sdflush  DL    -     0:00.69 [bufdaemon]
  0     8     0   0  16  0      0 18005602445710912 syncer   DL    -     0:01.59 [syncer]
  0     9     0   0 -16  0      0 18005602445710912 vlruwt   DL    -     0:00.07 [vnlru]
  0    10     0   0 -16  0      0 18005602445710912 audit_wo DL    -     0:00.00 [audit]
  0    11     0   0 155  0      0 18005602445710912 -        RL    -  3679:32.13 [idle]
  0    12     0   0 -84  0      0 18005602445710912 -        WL    -    80:44.17 [intr]
  0    13     0   0 -16  0      0 18005602445710912 sleep    DL    -    13:17.51 [ng_queue]
  0    14     0   0  -8  0      0 18005602445710912 -        DL    -     0:00.05 [geom]
  0    15     0   0 -16  0      0 18005602445710912 -        DL    -     2:40.33 [rand_harvestq]
  0    16     0   0 -68  0      0 18005602445710912 -        DL    -     0:00.64 [usb]
  0   624     1   0  52  0  16740 17979214311598592 select   Ds    -     0:00.00 [moused]
  0   641     1   0  20  0  13628 17979214975268864 select   Ds    -     0:00.38 [devd]
  0   884     1   0  21  0  42916 17979214311591424 wait     D     -     0:00.00 [syslog-ng]
  0  1011     1   0  20  0  63972 17979214311602176 nanslp   Ds    -     2:44.94 [mpd5]
101  1146     1   0  20  0  27820 17979214968234752 select   Ds    -     0:20.25 [zebra]
101  1152     1   0  20  0  30184 17979214968243968 select   Ds    -     0:03.57 [ospfd]
  0  1229     1   0  20  0  57984 17979214977036544 piperd   Ds    -     0:00.02 [perl]
  0  1230  1229   0  20  0  12364 17979214989044224 kqread   D     -     0:00.68 [tail]
  0  1245     1   0  20  0  70944 17979214972429056 select   D     -     3:17.91 [snmpd]
  0  1349     1   0  30  0  61312 17979214989026048 select   Ds    -     0:00.08 [sshd]
  0  2378     1   0  52  0  16624 17979214972409088 nanslp   Ds    -     0:00.29 [cron]
  0  2425     1   0  52  0  14520 17979217567197696 ttyin    Ds+   -     0:00.00 [getty]
  0  2426     1   0  22  0  47756 17979225354212608 wait     Ds    -     0:00.01 [login]
  0  2427     1   0  52  0  14520 17979214975250688 ttyin    Ds+   -     0:00.00 [getty]
  0  2428     1   0  52  0  14520 17979214989051648 ttyin    Ds+   -     0:00.00 [getty]
  0  2429     1   0  52  0  14520 17979214975301632 ttyin    Ds+   -     0:00.00 [getty]
  0  2430     1   0  52  0  14520 17979214989038848 ttyin    Ds+   -     0:00.00 [getty]
  0  2431     1   0  52  0  14520 17979214968252928 ttyin    Ds+   -     0:00.00 [getty]
  0  2432     1   0  52  0  14520 17979225355519744 ttyin    Ds+   -     0:00.00 [getty]
  0  2500   884   0  20  0  93980 17979214975412736 kqread   Ds    -     1:12.10 [syslog-ng]
  0  2501  2426   0  20  0  23600 17979225371473920 ttyin    D+    -     0:00.03 [csh]
: 80  3091 10679   0  20  0 319096 17979214975247104 select   D     -     0:00.35 [httpd]
 80  3096 10679   0  20  0 323192 17979214972458240 lockf    D     -     0:00.43 [httpd]
 80  3114 10679   0  20  0 323192 17979214975325440 lockf    D     -     0:00.27 [httpd]
 80  3115 10679   0  20  0 323192 17979214989031424 select   D     -     0:00.38 [httpd]
 80  4169 10679   0  20  0 323192 17979225362905600 lockf    D     -     0:00.22 [httpd]
 80  4171 10679   0  20  0 323192 17979214971218432 lockf    D     -     0:00.29 [httpd]
 80  6112 10679   0  20  0 323192 17979214975329024 lockf    D     -     0:00.22 [httpd]
 80  6929 10679   0  20  0 323192 17979217988464896 select   D     -     0:00.17 [httpd]
 80  7990 10679   0  20  0 319096 17979217567239680 lockf    D     -     0:00.00 [httpd]
  0 10679     1   0  20  0 319096 17979214977042176 select   Ds    -     0:01.85 [httpd]
136 16175     1   0  20  0  24896 17979221004258816 select   Ds    -     0:15.97 [dhcpd]
  0 17151     1   0  20  0  26124 17979220984199168 select   Ds    -     0:10.18 [ntpd]
  0 24699     1   0  21  0  17096 17979214985778432 wait     D     -     0:00.00 [sh]
  0 24700 24699   0  20  0  49736 17979225355532288 select   D     -     0:09.07 [utm5_rfw]
  0 54046     1   0  20  0  72728                 0 select   D     -     0:02.34 [perl]
 80 93850 10679   0  20  0 323192 17979214989013248 select   D     -     0:01.08 [httpd]
 80 93851 10679   0  20  0 323192 17979217988424960 lockf    D     -     0:01.71 [httpd]
 80 96813 10679   0  20  0 323192 17979217567194112 kqread   D     -     0:00.67 [httpd]
 80 97870 10679   0  20  0 323192 17979221004285952 select   D     -     0:01.03 [httpd]
 80 99947 10679   0  20  0 323192 17979214985787392 lockf    D     -     0:00.81 [httpd]

------------------------------------------------------------------------
vmstat -s

1650846807 cpu context switches
656478034 device interrupts
  7354698 software interrupts
 82071065 traps
191950933 system calls
       16 kernel threads created
   206913  fork() calls
   100676 vfork() calls
        0 rfork() calls
        0 swap pager pageins
        0 swap pager pages paged in
        0 swap pager pageouts
        0 swap pager pages paged out
     5524 vnode pager pageins
    30437 vnode pager pages paged in
    13322 vnode pager pageouts
    18574 vnode pager pages paged out
        0 page daemon wakeups
        0 pages examined by the page daemon
     3708 pages reactivated
  9311059 copy-on-write faults
     3262 copy-on-write optimized faults
 52999331 zero fill pages zeroed
        0 zero fill pages prezeroed
     1222 intransit blocking page faults
 64843608 total VM faults taken
     5698 page faults requiring I/O
        0 pages affected by kernel thread creation
  7739150 pages affected by  fork()
  6163994 pages affected by vfork()
        0 pages affected by rfork()
        0 pages cached
 72361672 pages freed
        0 pages freed by daemon
        0 pages freed by exiting processes
    15982 pages active
    57566 pages inactive
       35 pages in VM cache
   104673 pages wired down
   828233 pages free
     4096 bytes per page
 36429462 total name lookups
          cache hits (93% pos + 5% neg) system 0% per-directory
          deletions 0%, falsehits 0%, toolong 0%

------------------------------------------------------------------------
vmstat -m

         Type InUse MemUse HighUse Requests  Size(s)
    CAM queue    11     4K       -       45  16,32,512
         cdev     4     1K       -        4  256
     acpitask     1    16K       -        1  16384
     filedesc    74   205K       -   308603  16,32,2048,4096,8192,16384
        sigio     1     1K       -        8  64
      kdtrace   427    96K       -   375417  64,256
         kenv   107    12K       -      115  16,32,64,128,8192
       kqueue    39    39K       -     8461  256,512,2048
    proc-args    27     2K       -   216096  16,32,64,128,256
         hdac     1     1K       -        1  1024
        hhook     2     1K       -        2  256
      ithread   194    31K       -      194  32,128,256
       KTRACE   100    13K       -      100  128
       linker   447   302K       -     1332  16,32,64,128,256,512,1024,2048,4096,8192
      acpisem    26     4K       -       26  128
        lockf   120    10K       -   522560  64,128,256,512
   loginclass     3     1K       -     1804  64
       devbuf  6400 13265K       -     7344  16,32,64,128,256,512,1024,2048,4096,8192,32768,65536
         temp   487    33K       -  2058328  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768
       ip6opt   163    41K       -      745  256
       ip6ndp  1125   218K       -     2373  64,128,256
       module   509    64K       -      509  128
     mtx_pool     2    16K       -        2  8192
      ata_pci     2     1K       -        2  64
     pmchooks     1     1K       -        1  128
         pgrp    26     4K       -     2087  128
      session    24     3K       -     1974  128
         proc     2    32K       -        2  16384
      subproc   275   341K       -   307823  512,4096
         cred    94    15K       -  2174442  64,256
       feeder     7     1K       -        7  32
       plimit    17     5K       -     1960  256
      uidinfo     5     5K       -      323  128,4096
CAM dev queue     5     1K       -        5  64
       sysctl     0     0K       -   136704  16,32,64
    sysctloid  8998   451K       -     9203  16,32,64,128
    sysctltmp     0     0K       -    93553  16,32,64,128
      tidhash     1    32K       -        1  32768
      callout    13  4232K       -       13
        umtx  1044   131K       -     1044  128
     p1003.1b     1     1K       -        1  16
         SWAP     2   549K       -        2  64
          bus  1572   152K       -    19720  16,32,64,128,256,512,1024,2048
       bus-sc   184   685K       -    13416  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
      devstat     6    13K       -        6  32,4096
 eventhandler   100     8K       -      100  64,128
         kobj   329  1316K       -      522  4096
      Per-cpu     1     1K       -        1  32
       DEVFS3   134    34K       -      169  256
       DEVFS1   112    56K       -      131  512
         rman   331    38K       -      763  16,32,128
         sbuf     0     0K       -     2086  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
   DEVFS_RULE    55    26K       -       55  64,512
        DEVFS    25     1K       -       26  16,128
    taskqueue    59     9K       -       59  16,32,256
       Unitno    27     2K       -   146255  32,64
         vmem     3   104K       -        8  1024,2048,4096,8192,16384,32768,65536
     ioctlops     0     0K       -   459977  256,512,1024
       select   352    44K       -      352  128
          iov     0     0K       -  8967062  16,32,64,128,256,512
          msg     4    30K       -        4  2048,4096,8192,16384
          sem     4   106K       -        4  2048,4096
          shm     1    32K       -        1  32768
          tty    18    18K       -       28  1024
          pts     1     1K       -       11  256
         accf     1     1K       -        1  64
     mbuf_tag   123    31K       - 194650553  32,64,128,256
        shmfd     1     8K       -        1  8192
       soname     9     1K       -  7211860  16,32,128
          pcb   136   667K       -   126844  16,32,64,128,1024,2048,8192
       DEVFSP     2     1K       -        6  64
     vfscache     1  2048K       -        1
   cl_savebuf     0     0K       -     1816  64
     vfs_hash     1  1024K       -        1
       vnodes     1     1K       -        8  32,256
        mount    16     1K       -       86  16,32,64,128,256
  vnodemarker     0     0K       -     1254  512
          BPF   219    36K       -      850  16,64,128,512,4096
        ifnet   216   432K       -      834  128,256,512,1024,2048
       ifaddr  2313   804K       -     6633  32,64,256,512,2048,4096
  ether_multi  2277   104K       -     9398  16,32,64
        clone     8     1K       -        8  128
       arpcom     4     1K       -        4  16
      lltable  1759   548K       -    36212  256,512
      acpidev    64     4K       -       64  64
NFSD V4client     1     1K       -        1  256
 NFSD lckfile     1     1K       -        1  256
     routetbl  1865   145K       -    89885  32,64,128,256,512
         igmp   215    54K       -      829  256
   in_mfilter    28    28K       -       55  1024
     in_multi   214    54K       -      828  256
  ip_moptions    56     9K       -      110  64,256
    sctp_a_it     0     0K       -     1390  16
     sctp_vrf     1     1K       -        1  64
     sctp_ifa   213    27K       -      859  128
     sctp_ifn   186    24K       -      827  128
    sctp_iter     0     0K       -     1390  256
    hostcache     1    32K       -        1  32768
     syncache     1    64K       -        1  65536
  in6_mfilter   161   161K       -      736  1024
    in6_multi  1845   259K       -     7735  32,256
 ip6_moptions   322    46K       -     1472  32,256
          mld   215    27K       -      829  128
          rpc     2    16K       -        2  8192
audit_evclass   187     6K       -      231  32
      pagedep     3   129K       -      815  256
     inodedep    36  1042K       -    18362  512
    bmsafemap     4     9K       -     8113  256,8192
       newblk    53  2061K       -    31934  256
     indirdep     4     1K       -     1632  128,32768
     freefrag     6     1K       -     7439  128
     freeblks     4     1K       -     5369  256
     freefile     3     1K       -     4132  64
       diradd    21     3K       -     6739  128
        mkdir     0     0K       -      494  128
       dirrem     1     1K       -     4693  128
    newdirblk     0     0K       -      262  64
     freework     5     1K       -     7461  16,128
      freedep     0     0K       -       43  64
      jaddref     0     0K       -     7233  128
      jremref     0     0K       -     4741  128
       jmvref     0     0K       -      131  128
      jnewblk     0     0K       -    31933  128
    jfreefrag     0     0K       -     7439  128
         jseg    37     5K       -     8950  128
      jsegdep    88     6K       -    51346  64
        sbdep     0     0K       -      631  64
     savedino     0     0K       -     5678  256
      jblocks     2     1K       -        2  128,256
      softdep     1     1K       -        1  512
  ufs_dirhash   102    21K       -      138  16,32,64,128,256,512
    ufs_quota     1  1024K       -        1
    ufs_mount     3    21K       -        3  512,4096,16384
    vm_pgdata     2   513K       -        2  128
      UMAHash     7    11K       -       16  512,1024,2048,4096
 NFSD session     1     1K       -        1  1024
      memdesc     1     4K       -        1  4096
     atkbddev     2     1K       -        2  64
    pfs_nodes    21     6K       -       21  256
         GEOM    49     9K       -      624  16,32,64,128,256,512,1024,2048,8192,16384
      CAM SIM     5     2K       -        5  256
      CAM XPT    23     2K       -       90  16,32,64,128,256,512,1024,2048,65536
    raid_data     0     0K       -       90  32,128,256
      CAM DEV     6    12K       -       17  2048
      CAM CCB     0     0K       -   101877  2048
     pci_link    16     2K       -       16  64,128
    acpi_perf    12     3K       -       12  256
     CAM path     7     1K       -       54  32
       kbdmux     8    22K       -        8  16,512,1024,2048,16384
          LED     8     1K       -        8  16,128
       apmdev     1     1K       -        1  128
  madt_table     0     0K       -        1  4096
          USB    66   172K       -       69  16,32,128,256,512,1024,4096,8192,32768,65536
       USBdev    61    12K       -       61  32,64,128,256,512,4096
       qpidrv     2     1K       -        2  16
      io_apic     2     4K       -        2  2048
     acpiintr     1     1K       -        1  64
       acpica  5062   556K       -    76778  16,32,64,128,256,512,1024,2048,16384
          MCA    25     4K       -       25  128
      entropy  1026    65K       -   121406  32,64,4096
   CAM periph     4     1K       -       20  16,32,64,128,256
          msi    25     4K       -       25  128
     nexusdev     4     1K       -        4  16
       isadev     9     2K       -        9  128
     netgraph  1348  4298K       -    34048  16,64,128,256,512,1024,4096,8192
 netgraph_msg     0     0K       -  1168565  64,128,256,512,1024,65536
netgraph_hook  5012   627K       -   107640  128
netgraph_node  1775   508K       -    47113  128,256,512,1024,2048,4096,8192,16384,32768
  IpFw/IpAcct    35    38K       -       64  16,32,64,128,256,1024,32768
     ipfw_tbl  1768   442K       -    92181  256
     dummynet   213    91K       -   473910  256,512,4096
     dummynet    35    35K       -       36  512,1024
netgraph_path     0     0K       -   656127  16,32
netgraph_sock   222    12K       -    44271  32,128,256,512,1024,2048,4096
netgraph_mppc     0     0K       -        1  1024
netgraph_pppoe    88    23K       -    13289  64,512,16384
netgraph_ksock   128    16K       -     5719  128
netgraph_iface   210    27K       -      824  128
 netgraph_ppp   210  3360K       -      824  16384
 netflow_hash     2  6144K       -        2
netflow_general     1     1K       -        1  256
 netgraph_bpf  2508   314K       -    13168  128

------------------------------------------------------------------------
vmstat -z

ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               384,      0,     109,       1,     109,   0,   0
UMA Zones:             2176,      0,     109,       0,     109,   0,   0
UMA Slabs:               80,      0,    3153,      47,    3982,   0,   0
UMA RCntSlabs:           88,      0,   10563,      12,   10563,   0,   0
UMA Hash:               256,      0,       2,      13,       9,   0,   0
4 Bucket:                32,      0,     162,    3213,   70352,   0,   0
6 Bucket:                48,      0,      71,    3747,    8048,   0,   0
8 Bucket:                64,      0,      43,    3863,   80450,  11,   0
12 Bucket:               96,      0,     115,    1935,   42164,   0,   0
16 Bucket:              128,      0,     152,    1491,   66439,   0,   0
32 Bucket:              256,      0,     220,    1265,   17637,  52,   0
64 Bucket:              512,      0,     313,     599,   57623,2900,   0
128 Bucket:            1024,      0,     474,     310,  873268,14506,   0
256 Bucket:            2048,      0,     500,     150,  966943,   3,   0
vmem btag:               56,      0,    9002,    2713,   15706,  83,   0
VM OBJECT:              256,      0,    7083,    2007, 5358842,   0,   0
RADIX NODE:             144,      0,   13308,   13017,10619110,   0,   0
MAP:                    240,      0,       3,      61,       3,   0,   0
KMAP ENTRY:             128,      0,      21,    1002,      21,   0,   0
MAP ENTRY:              128,      0,    6069,    8873,13393029,   0,   0
VMSPACE:                448,      0,      42,     480,  307589,   0,   0
fakepg:                 104,      0,       0,       0,       0,   0,   0
mt_zone:               4112,      0,     383,       0,     383,   0,   0
16:                      16,      0,    4053,    3979, 9637049,   0,   0
32:                      32,      0,    7993,    4507,61239264,   0,   0
64:                      64,      0,    9202,    3756, 9776988,   0,   0
128:                    128,      0,   19509,    1633, 1545603,   0,   0
256:                    256,      0,    9264,    1746,135536345,   0,   0
512:                    512,      0,    2830,     394, 1008114,   0,   0
1024:                  1024,      0,     441,     171, 1312983,   0,   0
2048:                  2048,      0,     366,      94,  672934,   0,   0
4096:                  4096,      0,     891,      39,  431948,   0,   0
8192:                  8192,      0,      64,      14,    1508,   0,   0
16384:                16384,      0,     228,      16,    3081,   0,   0
32768:                32768,      0,      44,      19,    2622,   0,   0
65536:                65536,      0,      29,      18,     805,   0,   0
SLEEPQUEUE:              80,      0,     523,    1120,     523,   0,   0
64 pcpu:                  8,      0,   15094,    3210,   48250,   0,   0
Files:                   80,      0,     511,    1889, 5948902,   0,   0
TURNSTILE:              136,      0,     523,     397,     523,   0,   0
rl_entry:                40,      0,     360,    2740,     360,   0,   0
umtx pi:                 96,      0,       0,       0,       0,   0,   0
MAC labels:              40,      0,       0,       0,       0,   0,   0
PROC:                  1264,      0,      57,     159,  307605,   0,   0
THREAD:                1192,      0,     368,     154,   67810,   0,   0
cpuset:                  72,      0,     162,     223,     182,   0,   0
audit_record:          1248,      0,       0,       0,       0,   0,   0
mbuf_packet:            256, 1610385,   16502,    4253,412105658,   0,   0
mbuf:                   256, 1610385,       1,    5068,512417776,   0,   0
mbuf_cluster:          2048, 251622,   20746,      30,   20746,   0,   0
mbuf_jumbo_page:       4096, 125811,       0,     175,   36233,   0,   0
mbuf_jumbo_9k:         9216, 111831,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384,  83872,       0,       0,       0,   0,   0
mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   0
NetGraph items:          72,  65565,       5,    4397,43786674,   0,   0
NetGraph data items:     72,  65565,       1,    4649,120349620,   0,   0
ttyinq:                 160,      0,     120,     680,     960,   0,   0
DMAR_MAP_ENTRY:         120,      0,       0,       0,       0,   0,   0
ttyoutq:                256,      0,      64,     611,     505,   0,   0
g_bio:                  248,      0,       0,    1408,  305784,   0,   0
nvme_request:           128,      0,       0,       0,       0,   0,   0
vtnet_tx_hdr:            24,      0,       0,       0,       0,   0,   0
FPU_save_area:          512,      0,       0,       0,       0,   0,   0
VNODE:                  472,      0,    5824,     208,    9983,   0,   0
VNODEPOLL:              112,      0,       1,     139,       1,   0,   0
BUF TRIE:               144,      0,    1015,   25661,   70178,   0,   0
S VFS Cache:            108,      0,    4986,    1419,  243229,   0,   0
STS VFS Cache:          148,      0,       0,       0,       0,   0,   0
L VFS Cache:            328,      0,     163,     365,   39295,   0,   0
LTS VFS Cache:          368,      0,       0,       0,       0,   0,   0
NAMEI:                 1024,      0,       0,     208,10354087,   0,   0
DIRHASH:               1024,      0,     178,      98,     220,   0,   0
NCLNODE:                528,      0,       0,       0,       0,   0,   0
Mountpoints:            816,      0,       2,      13,       2,   0,   0
pipe:                   744,      0,      12,     213,  207130,   0,   0
procdesc:               128,      0,       0,       0,       0,   0,   0
ksiginfo:               112,      0,     207,    1928,   18980,   0,   0
itimer:                 352,      0,       0,       0,       0,   0,   0
KNOTE:                  128,      0,      78,    1813, 5462513,   0,   0
socket:                 696, 129995,     540,     205, 2264273,   0,   0
unpcb:                  240, 130000,      17,     719,   13387,   0,   0
ipq:                     56,   7881,       0,     568,     424,   0,   0
udp_inpcb:              392, 130000,     200,     380, 2072269,   0,   0
udpcb:                   24, 130093,     200,    2973, 2072269,   0,   0
tcp_inpcb:              392, 130000,     277,     543,   47411,   0,   0
tcpcb:                 1032, 129996,     157,     131,   47411,   0,   0
tcptw:                   88,  26010,     120,    2040,   36087,   0,   0
syncache:               160,  15375,       0,     750,   18190,   0,   0
hostcache:              136,  15370,      91,     663,     377,   0,   0
sackhole:                32,      0,       0,    3125,     208,   0,   0
tcpreass:                40,  15800,       0,    2800,      95,   0,   0
sctp_ep:               1424, 129996,       0,       0,       0,   0,   0
sctp_asoc:             2416,  40000,       0,       0,       0,   0,   0
sctp_laddr:              48,  80012,       0,    3071,    1504,   0,   0
sctp_raddr:             728,  80000,       0,       0,       0,   0,   0
sctp_chunk:             136, 400026,       0,       0,       0,   0,   0
sctp_readq:             104, 400026,       0,       0,       0,   0,   0
sctp_stream_msg_out:    104, 400026,       0,       0,       0,   0,   0
sctp_asconf:             40, 400000,       0,       0,       0,   0,   0
sctp_asconf_ack:         48, 400060,       0,       0,       0,   0,   0
udplite_inpcb:          392, 130000,       0,       0,       0,   0,   0
ripcb:                  392, 130000,     130,     430,  119706,   0,   0
rtentry:                200,      0,    1593,     667,    4871,   0,   0
IPFW dynamic rule:      120,   4125,       0,       0,       0,   0,   0
selfd:                   56,      0,     767,    4061,173043367,   0,   0
SWAPMETA:               288, 503256,       0,       0,       0,   0,   0
FFS inode:              176,      0,    5785,     507,    9930,   0,   0
FFS1 dinode:            128,      0,       0,       0,       0,   0,   0
FFS2 dinode:            256,      0,    5785,     500,    9924,   0,   0
NetFlow IPv4 cache:      88, 1048606,   11247,    9275, 2574468,   0,   0
NetFlow IPv6 cache:     128, 1048606,       0,       0,       0,   0,   0


------------------------------------------------------------------------
vmstat -i

interrupt                          total       rate
irq18: ehci0 uhci5                     2          0
irq19: uhci2 uhci4*               101839       1755
irq21: uhci1                          25          0
irq23: uhci3 ehci1                  1183         20
cpu0:timer                      25956883     447532
irq256: igb0:que 0              51892835     894704
irq257: igb0:que 1              34695322     598195
irq258: igb0:que 2              33894028     584379
irq259: igb0:que 3              37781705     651408
irq260: igb0:que 4              31470588     542596
irq261: igb0:que 5              34916830     602014
irq262: igb0:que 6              39887801     687720
irq263: igb0:que 7              42086634     725631
irq264: igb0:link                      2          0
irq265: igb1:que 0              57575117     992674
irq266: igb1:que 1              38316880     660635
irq267: igb1:que 2              39444444     680076
irq268: igb1:que 3              45242454     780042
irq269: igb1:que 4              38463048     663156
irq270: igb1:que 5              38252323     659522
irq271: igb1:que 6              46709519     805336
irq272: igb1:que 7              45745453     788714
irq273: igb1:link                      2          0
cpu1:timer                      21723881     374549
cpu7:timer                      15617115     269260
cpu4:timer                      14258757     245840
cpu10:timer                     15120605     260700
cpu3:timer                      22458755     387219
cpu8:timer                      17601094     303467
cpu2:timer                      22168500     382215
cpu11:timer                     16212014     279517
cpu5:timer                      13677497     235818
cpu6:timer                      14882604     256596
cpu9:timer                      15578167     268589
Total                          871733906   15029894
------------------------------------------------------------------------
pstat -T

511/129992 files
0M/3851M swap space

------------------------------------------------------------------------
pstat -s

Device          512-blocks     Used    Avail Capacity
/dev/ada0p3        7888616        0  7888616     0%

------------------------------------------------------------------------
iostat

iostat: kvm_read(_tk_nin): invalid address (0x0)
iostat: disabling TTY statistics
            ada0            pass0             cpu
  KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
 28.37 1743 48.30   0.00   0  0.00   1  0  1  2 96

------------------------------------------------------------------------
ipcs -a

Message Queues:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME
Shared Memory:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME

Semaphores:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME


------------------------------------------------------------------------
ipcs -T

msginfo:
        msgmax:        16384    (max characters in a message)
        msgmni:           40    (# of message queues)
        msgmnb:         2048    (max characters in a message queue)
        msgtql:           40    (max # of messages in system)
        msgssz:            8    (size of a message segment)
        msgseg:         2048    (# of message segments in system)

shminfo:
        shmmax:    536870912    (max shared memory segment size)
        shmmin:            1    (min shared memory segment size)
        shmmni:          192    (max number of shared memory identifiers)
        shmseg:          128    (max shared memory segments per process)
        shmall:       131072    (max amount of shared memory in pages)

seminfo:
        semmni:           50    (# of semaphore identifiers)
        semmns:          340    (# of semaphores in system)
        semmnu:          150    (# of undo structures in system)
        semmsl:          340    (max # of semaphores per id)
        semopm:          100    (max # of operations per semop call)
        semume:           50    (max # of undo entries per process)
        semusz:          632    (size in bytes of undo structure)
        semvmx:        32767    (semaphore maximum value)
        semaem:        16384    (adjust on exit max value)


------------------------------------------------------------------------
nfsstat

Client Info:
Rpc Counts:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Rpc Info:
 TimedOut   Invalid X Replies   Retries  Requests
        0         0         0         0         0
Cache Info:
Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW Hits    Misses
        0         0         0         0         0         0         0         0
BioRLHits    Misses BioD Hits    Misses DirE Hits    Misses Accs Hits    Misses
        0         0         0         0         0         0         0         0

Server Info:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Server Ret-Failed
                0
Server Faults
            0
Server Cache Stats:
   Inprog      Idem  Non-idem    Misses
        0         0         0         0
Server Write Gathering:
 WriteOps  WriteRPC   Opsaved
        0         0         0

------------------------------------------------------------------------
netstat -s

tcp:
        1660510 packets sent
                1041406 data packets (315042341 bytes)
                986 data packets (653126 bytes) retransmitted
                68 data packets unnecessarily retransmitted
                0 resends initiated by MTU discovery
                539100 ack-only packets (40832 delayed)
                0 URG only packets
                0 window probe packets
                3325 window update packets
                75696 control packets
        1978927 packets received
                1069010 acks (for 314829340 bytes)
                23145 duplicate acks
                0 acks for unsent data
                1729711 packets (1130361494 bytes) received in-sequence
                23425 completely duplicate packets (1232259 bytes)
                6 old duplicate packets
                2 packets with some dup. data (128 bytes duped)
                95 out-of-order packets (20137 bytes)
                0 packets (0 bytes) of data after window
                0 window probes
                1357 window update packets
                845 packets received after close
                4 discarded for bad checksums
                0 discarded for bad header offset fields
               0 discarded because packet too short
                0 discarded due to memory problems
        29212 connection requests
        18076 connection accepts
        0 bad connection attempts
        0 listen queue overflows
        2701 ignored RSTs in the windows
        47255 connections established (including accepts)
        47134 connections closed (including 740 drops)
                34134 connections updated cached RTT on close
                34161 connections updated cached RTT variance on close
                3422 connections updated cached ssthresh on close
        31 embryonic connections dropped
        1048219 segments updated rtt (of 1001884 attempts)
        998 retransmit timeouts
                34 connections dropped by rexmit timeout
        0 persist timeouts
                0 connections dropped by persist timeout
        0 Connections (fin_wait_2) dropped because of timeout
        1 keepalive timeout
                1 keepalive probe sent
                0 connections dropped by keepalive
        48130 correct ACK header predictions
        795511 correct data packet header predictions
        18190 syncache entries added
                124 retransmitted
                93 dupsyn
                0 dropped
               18076 completed
                0 bucket overflow
                0 cache overflow
                93 reset
                21 stale
                0 aborted
                0 badack
                0 unreach
                0 zone failures
        18190 cookies sent
        0 cookies received
        377 hostcache entries added
                0 bucket overflow
        48 SACK recovery episodes
        198 segment rexmits in SACK recovery episodes
        273529 byte rexmits in SACK recovery episodes
        1294 SACK options (SACK blocks) received
        23 SACK options (SACK blocks) sent
        0 SACK scoreboard overflow
        0 packets with ECN CE bit set
        0 packets with ECN ECT(0) bit set
        0 packets with ECN ECT(1) bit set
        0 successful ECN handshakes
        0 times ECN reduced the congestion window
        0 packets with valid tcp-md5 signature received
        0 packets with invalid tcp-md5 signature received
        0 packets with tcp-md5 signature mismatch
        0 packets with unexpected tcp-md5 signature received
        0 packets without expected tcp-md5 signature received
udp:
        458605 datagrams received
        0 with incomplete header
        0 with bad data length field
        0 with bad checksum
        101 with no checksum
        9031 dropped due to no socket
        20123 broadcast/multicast datagrams undelivered
        0 dropped due to full socket buffers
        0 not for hashed pcb
        429451 delivered
        129273 datagrams output
        0 times multicast source filter matched
ip:
        480216969 total packets received
        11 bad header checksums
        0 with size smaller than minimum
        0 with data size < data length
        0 with ip length > max ip packet size
        0 with header length < data size
        0 with data length < header length
        0 with bad options
        0 with incorrect version number
        847 fragments received
        0 fragments dropped (dup or out of space)
        1 fragment dropped after timeout
        423 packets reassembled ok
       29851512 packets for this host
        6149 packets for unknown/unsupported protocol
        404815446 packets forwarded (0 packets fast forwarded)
        868794 packets not forwardable
        0 packets received for unknown multicast group
        5390419 redirects sent
        44168795 packets sent from this host
        6219 packets sent with fabricated ip header
        0 output packets dropped due to no bufs, etc.
        0 output packets discarded due to no route
        19941 output datagrams fragmented
        39882 fragments created
        6539 datagrams that can't be fragmented
        0 tunneling packets that can't find gif
        0 datagrams with bad address in header
icmp:
        72237 calls to icmp_error
        32 errors not generated in response to an icmp message
        Output histogram:
                echo reply: 7827
                destination unreachable: 36856
                routing redirect: 5368505
                time exceeded: 35345
        0 messages with bad code fields
        0 messages less than the minimum length
        4440 messages with bad checksum
        2 messages with bad length
        0 multicast echo requests ignored
        0 multicast timestamp requests ignored
        Input histogram:
                echo reply: 125925
                destination unreachable: 2359
                routing redirect: 1
                echo: 7827
                time exceeded: 13856
        7827 message responses generated
        0 invalid return addresses
        0 no return routes
arp:
        52798 ARP requests sent
        84477 ARP replies sent
        198080 ARP requests received
        10014 ARP replies received
        208094 ARP packets received
        59144 total packets dropped due to no ARP entry
        32636 ARP entrys timed out
        0 Duplicate IPs seen
ip6:
        1984 total packets received
        0 with size smaller than minimum
        0 with data size < data length
        0 with bad options
        0 with incorrect version number
        0 fragments received
        0 fragments dropped (dup or out of space)
        0 fragments dropped after timeout
      0 fragments that exceeded limit
        0 packets reassembled ok
        1984 packets for this host
        0 packets forwarded
        0 packets not forwardable
        0 redirects sent
        5260 packets sent from this host
        0 packets sent with fabricated ip header
        0 output packets dropped due to no bufs, etc.
        21 output packets discarded due to no route
        0 output datagrams fragmented
        0 fragments created
        0 datagrams that can't be fragmented
        0 packets that violated scope rules
        0 multicast packets which we don't join
        Input histogram:
                TCP: 1984
        Mbuf statistics:
                1246 one mbuf
                two or more mbuf:
                        lo0= 492
                855 one ext mbuf
                0 two or more ext mbuf
        0 packets whose headers are not contiguous
        0 tunneling packets that can't find gif
        0 packets discarded because of too many headers
        246 failures of source address selection
        source addresses on a non-outgoing I/F
                246 addresses scope=f
        Source addresses selection rule applied:
                246 same address
icmp6:
        0 calls to icmp6_error
        0 errors not generated in response to an icmp6 message
        0 errors not generated because of rate limitation
        Output histogram:
                neighbor solicitation: 725
        0 messages with bad code fields
        0 messages < minimum length
        0 bad checksums
        0 messages with bad length
        Histogram of error messages to be generated:
                0 no route
                0 administratively prohibited
                0 beyond scope
                0 address unreachable
                0 port unreachable
                0 packet too big
                0 time exceed transit
                0 time exceed reassembly
                0 erroneous header field
                0 unrecognized next header
                0 unrecognized option
                0 redirect
                0 unknown
        0 message responses generated
        0 messages with too many ND options
        0 messages with bad ND options
        0 bad neighbor solicitation messages
        0 bad neighbor advertisement messages
        0 bad router solicitation messages
        0 bad router advertisement messages
        0 bad redirect messages
        0 path MTU changes
rip6:
        0 messages received
        0 checksum calculations on inbound
        0 messages with bad checksum
        0 messages dropped due to no socket
        0 multicast messages dropped due to no socket
        0 messages dropped due to full socket buffers
        0 delivered
        0 datagrams output

------------------------------------------------------------------------
------------------------------------------------------------------------
netstat -m

16503/9321/25824 mbufs in use (current/cache/total)
16493/4283/20776/251622 mbuf clusters in use (current/cache/total/max)
16502/4253 mbuf+clusters out of packet secondary zone in use (current/cache)
0/175/175/125811 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/111831 9k jumbo clusters in use (current/cache/total/max)
0/0/0/83872 16k jumbo clusters in use (current/cache/total/max)
37142K/11596K/48738K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
--------
Comment 1 Alexander 2016-05-13 13:19:24 UTC
I found resolve as I think.
I had 2 pipes:
#1:
pipe tablearg log logamount 10000 ip from any to table(60) out
pipe tablearg log logamount 10001 ip from table(61) to any in

#2:
pipe with 128Kbit/s
and I have php script, which by cron every minute connect to mysql and put in pipe #2 IP which <0 balance
in billing.

I think that on any moment one IP may be in one and in the second pipe. And its crashed core by dummynet.
Comment 2 Andrey V. Elsukov freebsd_committer 2016-05-13 15:17:38 UTC
It looks like interface, where a packet was received and then delayed for some time in the dummynet pipe, has been disappeared. And panic occurs when ip_input() tries to get access to stale ifnet data.
Comment 3 Eugene Grosbein 2016-05-21 12:05:11 UTC
(In reply to Andrey V. Elsukov from comment #2)

See also my PR https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=162558
I have similar problem and similar crash backtrace.
Comment 4 Andrey V. Elsukov freebsd_committer 2016-05-21 19:57:26 UTC
(In reply to eugen from comment #3)
> (In reply to Andrey V. Elsukov from comment #2)
> 
> See also my PR https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=162558
> I have similar problem and similar crash backtrace.

I don't see the easy way to fix this problem.
The only way that can be applied to current code - do delayed destroy of ifnet structures in a garbage collector task thread. IMHO.
Comment 5 Andrey 2017-09-25 18:00:52 UTC
I confirm. I have the same problem, but in mpd5.
If necessary, then I preloaded the logs.
I apologize, earlier I was not engaged in the publication of reports and bugs.
Comment 6 Eugene Grosbein freebsd_committer 2017-09-28 11:23:12 UTC
Please try a patch proposed in PR https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222617. Also, you may find usable another fix from https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=220076 if you use netgraph interfaces ng_iface(4) named like ngXXX.

*** This bug has been marked as a duplicate of bug 222617 ***