Bug 169945

Summary: [zfs] [panic] Kernel panic while importing zpool (after having a kernel panic)
Product: Base System Reporter: Jerome Lebel <lebel.jerome>
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Open ---    
Severity: Affects Only Me Keywords: crash
Priority: Normal    
Version: 9.0-RELEASE   
Hardware: Any   
OS: Any   

Description Jerome Lebel 2012-07-17 18:30:10 UTC 
I tried to remove a snapshot. Then I had a kernel panic. Now each time I want to import my zpool again, I need to do zpool import -f storage. And this command always trigger a new kernel panic now.

I was not able to get the stack trace from the first kernel panic. So I have no clue if it was related or not. But anyway, now, I can't use my zpool anymore.

I will try to see if I can debug myself the kernel. Anybody is welcome to give me some help on that :-)

freebsd-test16# cat /var/crash/core.txt.0
freebsd-test16 dumped core - see /var/crash/vmcore.0

Tue Jul 17 18:06:48 CEST 2012

FreeBSD freebsd-test16 9.0-RELEASE-p3 FreeBSD 9.0-RELEASE-p3 #0: Tue Jun 12 02:52:29 UTC 2012     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64

panic: page fault

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
Solaris: WARNING: metaslab_free_dva(): bad DVA 33554432:3456550719488


Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 03
fault virtual address	= 0x468
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff81673525
stack pointer	        = 0x28:0xffffff811c7e1370
frame pointer	        = 0x28:0xffffff811c7e1390
code segment		= base rx0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 1469 (txg_thread_enter)
trap number		= 12
panic: page fault
cpuid = 3
KDB: stack backtrace:
#0 0xffffffff808680fe at kdb_backtrace+0x5e
#1 0xffffffff80832cb7 at panic+0x187
#2 0xffffffff80b185a0 at trap_fatal+0x290
#3 0xffffffff80b188e9 at trap_pfault+0x1f9
#4 0xffffffff80b18daf at trap+0x3df
#5 0xffffffff80b0324f at calltrap+0x8
#6 0xffffffff8167357e at bp_get_dsize_sync+0x3e
#7 0xffffffff8165e9b0 at dsl_scan_free_cb+0x120
#8 0xffffffff81630c74 at bpobj_iterate_impl+0xd4
#9 0xffffffff81630ebb at bpobj_iterate_impl+0x31b
#10 0xffffffff8165fbc3 at dsl_scan_sync+0x4c3
#11 0xffffffff816670e0 at spa_sync+0x390
#12 0xffffffff81677739 at txg_sync_thread+0x139
#13 0xffffffff8080682f at fork_exit+0x11f
#14 0xffffffff80b0377e at fork_trampoline+0xe
Uptime: 1m9s
Dumping 300 out of 4064 MB:..6%..11%..22%..32%..43%..54%..64%..75%..86%..91%

Reading symbols from /boot/kernel/ichwd.ko...Reading symbols from /boot/kernel/ichwd.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ichwd.ko
Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/opensolaris.ko
#0  doadump (textdump=Variable "textdump" is not available.
) at pcpu.h:224
224	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) #0  doadump (textdump=Variable "textdump" is not available.
) at pcpu.h:224
#1  0xffffffff808327f5 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:442
#2  0xffffffff80832ca1 in panic (fmt=Variable "fmt" is not available.
)
    at /usr/src/sys/kern/kern_shutdown.c:607
#3  0xffffffff80b185a0 in trap_fatal (frame=0xc, eva=Variable "eva" is not available.
)
    at /usr/src/sys/amd64/amd64/trap.c:819
#4  0xffffffff80b188e9 in trap_pfault (frame=0xffffff811c7e12c0, usermode=0)
    at /usr/src/sys/amd64/amd64/trap.c:735
#5  0xffffffff80b18daf in trap (frame=0xffffff811c7e12c0)
    at /usr/src/sys/amd64/amd64/trap.c:474
#6  0xffffffff80b0324f in calltrap ()
    at /usr/src/sys/amd64/amd64/exception.S:228
#7  0xffffffff81673525 in dva_get_dsize_sync (spa=Variable "spa" is not available.
)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c:1452
#8  0xffffffff8167357e in bp_get_dsize_sync (spa=0xfffffe012f431000, 
    bp=0xffffff8001c2e580)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c:1464
#9  0xffffffff8165e9b0 in dsl_scan_free_cb (arg=0xfffffe012f68a500, 
    bp=0xffffff8001c2e580, tx=0xfffffe012f68ab00)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c:1415
#10 0xffffffff81630c74 in bpobj_iterate_impl (bpo=0xffffff811c7e15c0, 
    func=0xffffffff8165e890 <dsl_scan_free_cb>, arg=0xfffffe012f68a500, 
    tx=0xfffffe012f68ab00, free=1)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/bpobj.c:184
#11 0xffffffff81630ebb in bpobj_iterate_impl (bpo=0xfffffe0002986500, 
    func=0xffffffff8165e890 <dsl_scan_free_cb>, arg=0xfffffe012f68a500, 
    tx=0xfffffe012f68ab00, free=1)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/bpobj.c:251
#12 0xffffffff8165fbc3 in dsl_scan_sync (dp=0xfffffe0002986400, 
    tx=0xfffffe012f68ab00)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c:1484
#13 0xffffffff816670e0 in spa_sync (spa=0xfffffe012f431000, txg=1710332)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:5612
#14 0xffffffff81677739 in txg_sync_thread (arg=Variable "arg" is not available.
)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:423
#15 0xffffffff8080682f in fork_exit (
    callout=0xffffffff81677600 <txg_sync_thread>, arg=0xfffffe0002986400, 
    frame=0xffffff811c7e1c50) at /usr/src/sys/kern/kern_fork.c:995
#16 0xffffffff80b0377e in fork_trampoline ()
    at /usr/src/sys/amd64/amd64/exception.S:602
#17 0x0000000000000000 in ?? ()
#18 0x0000000000000000 in ?? ()
#19 0x0000000000000001 in ?? ()
#20 0x0000000000000000 in ?? ()
#21 0x0000000000000000 in ?? ()
#22 0x0000000000000000 in ?? ()
#23 0x0000000000000000 in ?? ()
#24 0x0000000000000000 in ?? ()
#25 0x0000000000000000 in ?? ()
#26 0x0000000000000000 in ?? ()
#27 0x0000000000000000 in ?? ()
#28 0x0000000000000000 in ?? ()
#29 0x0000000000000000 in ?? ()
#30 0x0000000000000000 in ?? ()
#31 0x0000000000000000 in ?? ()
#32 0x0000000000000000 in ?? ()
#33 0x0000000000000000 in ?? ()
#34 0x0000000000000000 in ?? ()
#35 0x0000000000000000 in ?? ()
#36 0x0000000000000000 in ?? ()
#37 0x0000000000000000 in ?? ()
#38 0x0000000000000000 in ?? ()
#39 0x0000000000000000 in ?? ()
#40 0x0000000000000000 in ?? ()
#41 0xffffffff81119b80 in affinity ()
#42 0x0000000000000000 in ?? ()
#43 0x0000000000000000 in ?? ()
#44 0xfffffe012f7c1460 in ?? ()
#45 0xffffff811c7e08f0 in ?? ()
#46 0xffffff811c7e0898 in ?? ()
#47 0xfffffe0002919460 in ?? ()
#48 0xffffffff8085acc2 in sched_switch (td=0xffffffff81677600, 
    newtd=0xfffffe0002986400, flags=Variable "flags" is not available.
) at /usr/src/sys/kern/sched_ule.c:1848
Previous frame inner to this frame (corrupt stack?)
(kgdb) 

------------------------------------------------------------------------
ps -axl

  UID   PID  PPID CPU PRI NI    VSZ    RSS MWCHAN STAT  TT     TIME COMMAND
    0     0     0   0  -8  0      0      0 fffffe012f665480 DLs   ??  0:00.28 [kernel]
    0     1     0   0  52  0   6280      0 wait   DLs   ??  0:00.01 [init]
    0     2     0   0 -16  0      0      0 waitin DL    ??  0:00.00 [sctp_itera
    0     3     0   0 -16  0      0      0 ccb_sc DL    ??  0:00.00 [xpt_thrd]
    0     4     0   0 -16  0      0      0 psleep DL    ??  0:00.00 [pagedaemon
    0     5     0   0 -16  0      0      0 psleep DL    ??  0:00.00 [vmdaemon]
    0     6     0   0 155  0      0      0 pgzero DL    ??  0:00.00 [pagezero]
    0     7     0   0 -16  0      0      0 -      RL    ??  0:00.00 [bufdaemon]
    0     8     0   0  16  0      0      0 syncer DL    ??  0:00.00 [syncer]
    0     9     0   0 -16  0      0      0 vlruwt DL    ??  0:00.00 [vnlru]
    0    10     0   0 -16  0      0      0 audit_ DL    ??  0:00.00 [audit]
    0    11     0   0 155  0      0      0 -      RL    ??  1:29.94 [idle]
    0    12     0   0 -84  0      0      0 -      WL    ??  0:00.12 [intr]
    0    13     0   0  -8  0      0      0 -      DL    ??  0:00.05 [geom]
    0    14     0   0 -16  0      0      0 -      RL    ??  0:00.00 [yarrow]
    0    15     0   0 -68  0      0      0 -      DL    ??  0:00.09 [usb]
    0    16     0   0 -16  0      0      0 sdflus DL    ??  0:00.00 [softdepflu
    0   931     1   0  52  0  10052      0 select Ds    ??  0:00.00 [dhclient]
   65   983     1   0  48  0  10052      0 select Ds    ??  0:00.00 [dhclient]
    0   984     1   0  20  0  10372      0 select Ds    ??  0:00.00 [devd]
    0  1125     1   0   1  0  12184      0 -      Rs    ??  0:00.01 [syslogd]
    0  1263     1   0 -52  0  14368      0 -      Rs    ??  0:00.00 [watchdogd]
    0  1330     1   0  20  0  22332      0 -      Rs    ??  0:00.01 [ntpd]
    0  1367     1   0  23  0  46876      0 select Ds    ??  0:00.00 [sshd]
    0  1374     1   0  20  0  20384      0 select Ds    ??  0:00.00 [sendmail]
   25  1380     1   0  52  0  20384      0 pause  Ds    ??  0:00.00 [sendmail]
    0  1386     1   0  20  0  14260      0 nanslp Ds    ??  0:00.00 [cron]
    0  1447     1   0  52  0  14636      0 wait   D     ??  0:00.00 [sh]
    0  1448     1   0  52  0  10052      0 piperd D     ??  0:00.00 [logger]
    0  1449  1447   0  52  0   3916      0 -      R     ??  0:00.00 [sleep]
    0  1452     1   0  52  0  12184      0 ttyin  Ds+   ??  0:00.00 [getty]
    0  1453     1   0  52  0  12184      0 ttyin  Ds+   ??  0:00.00 [getty]
    0  1454     1   0  52  0  12184      0 ttyin  Ds+   ??  0:00.00 [getty]
    0  1455     1   0  52  0  12184      0 ttyin  Ds+   ??  0:00.00 [getty]
    0  1456     1   0  52  0  12184      0 ttyin  Ds+   ??  0:00.00 [getty]
    0  1457     1   0  52  0  12184      0 ttyin  Ds+   ??  0:00.00 [getty]
    0  1458     1   0  52  0  12184      0 ttyin  Ds+   ??  0:00.00 [getty]
    0  1459     1   0  52  0  12184      0 ttyin  Ds+   ??  0:00.00 [getty]
    0  1460  1367   0  20  0  51536      0 select Ds    ??  0:00.00 [sshd]
    0  1462  1460   0  20  0  17664      0 pause  Ds    ??  0:00.01 [csh]
    0  1466  1462   0  20  0  17572      0 wait   D     ??  0:00.00 [bash]
    0  1468  1466   0  23  0  35604      0 tx->tx D+    ??  0:00.00 [zpool]
    0  1469     0   0  -8  0      0      0 -      RL    ??  0:00.00 [zfskern]

------------------------------------------------------------------------
vmstat -s

  6117994 cpu context switches
    16250 device interrupts
    13828 software interrupts
   114445 traps
   162547 system calls
       17 kernel threads created
     1448  fork() calls
        4 vfork() calls
        0 rfork() calls
        0 swap pager pageins
        0 swap pager pages paged in
        0 swap pager pageouts
        0 swap pager pages paged out
      349 vnode pager pageins
     2768 vnode pager pages paged in
        0 vnode pager pageouts
        0 vnode pager pages paged out
        0 page daemon wakeups
        0 pages examined by the page daemon
      103 pages reactivated
    46147 copy-on-write faults
      411 copy-on-write optimized faults
    33553 zero fill pages zeroed
        0 zero fill pages prezeroed
        6 intransit blocking page faults
   127758 total VM faults taken
        0 pages affected by kernel thread creation
   737123 pages affected by  fork()
     1998 pages affected by vfork()
        0 pages affected by rfork()
        0 pages cached
   132012 pages freed
        0 pages freed by daemon
        0 pages freed by exiting processes
     3621 pages active
     2422 pages inactive
        5 pages in VM cache
    33374 pages wired down
   962470 pages free
     4096 bytes per page
    19393 total name lookups
          cache hits (85% pos + 7% neg) system 0% per-directory
          deletions 0%, falsehits 0%, toolong 0%

------------------------------------------------------------------------
vmstat -m

         Type InUse MemUse HighUse Requests  Size(s)
       feeder    18     2K       -       23  32,128
       isadev     9     2K       -        9  128
         cdev     7     2K       -        7  256
      acpidev    34     3K       -       34  64
        mixer     4    16K       -        4  4096
        sigio     1     1K       -        1  64
     filedesc    48    31K       -     1477  32,512,1024,4096
         kenv   114    13K       -      124  16,32,64,128
       kqueue     0     0K       -       40  256,2048
    proc-args    26     2K       -      457  16,32,64,128,256
        hhook     2     1K       -        2  128
      ithread    87    14K       -       87  32,128,256
       KTRACE   100    13K       -      100  128
       linker   190   792K       -      210  16,32,64,128,256,512,2048,4096
        lockf    20     3K       -       36  64,128
   loginclass     2     1K       -        2  64
       ip6ndp    12     1K       -       14  32,64,128
       ip6opt     0     0K       -        1  32
         temp    30    13K       -     7263  16,32,64,128,256,512,1024,2048,4096
       devbuf 19997 45575K       -    20179  16,32,64,128,256,512,1024,2048,4096
       module   451    57K       -      451  128
     mtx_pool     2    16K       -        2  
          osd     3     1K       -        4  16,64
      subproc   144   223K       -     1571  512,4096
         proc     2    16K       -        2  
      session    22     3K       -       22  128
         pgrp    24     3K       -       27  128
         cred    28     5K       -     3860  64,256
      uidinfo     4     3K       -        5  128,2048
       plimit    13     4K       -      147  256
    sysctltmp     0     0K       -      345  16,32,64,128,4096
    sysctloid  4161   206K       -     4248  16,32,64,128
       sysctl     0     0K       -      516  16,32,64
      tidhash     1    16K       -        1  
      callout     3  1536K       -        3  
         umtx   678    85K       -      678  128
     p1003.1b     1     1K       -        1  16
         SWAP     2   889K       -        2  64
       bus-sc    71   305K       -     2207  16,32,64,128,256,512,1024,2048,4096
          bus  1151    96K       -     4518  16,32,64,128,256,1024
      devstat     8    17K       -        8  32,4096
 eventhandler    79     7K       -       79  64,128
     pci_link    16     2K       -       16  64,128
         kobj   322  1288K       -      419  4096
      Per-cpu     1     1K       -        1  32
         rman   224    27K       -      614  16,32,128
         sbuf     0     0K       -     1007  16,32,64,128,256,512,1024,2048,4096
        stack     0     0K       -        2  256
    taskqueue    51     5K       -      111  16,32,64,128,1024
       Unitno    12     1K       -     3836  32,64
          iov     0     0K       -      315  16,64,128,256,512
       select    12     2K       -       12  128
     ioctlops     1     8K       -      965  16,32,64,128,256,512,1024
          msg     4    30K       -        4  2048,4096
          sem     4   106K       -        4  2048,4096
          shm     1    20K       -        1  
          tty    20    20K       -       22  1024,2048
          pts     1     1K       -        1  256
     mbuf_tag     0     0K       -       23  32,128
        shmfd     1     8K       -        1  
          pcb    18   157K       -       67  16,32,128,1024,2048,4096
       soname     3     1K       -      316  16,32,128
     vfscache     1  2048K       -        1  
     vfs_hash     1  1024K       -        1  
     acpiintr     1     1K       -        1  64
       vnodes     2     1K       -        2  256
       acpica  2820   283K       -    36511  16,32,64,128,256,512,1024
  vnodemarker     0     0K       -        8  512
        mount    16     1K       -      119  16,32,64,128,256
          BPF    12    10K       -       12  128,512,4096
  ether_multi    33     2K       -       44  16,32,64
       ifaddr    61    18K       -       62  32,64,128,256,512,4096
        ifnet     8    15K       -        8  128,2048
        clone     6    24K       -        6  4096
       arpcom     1     1K       -        1  16
      lltable    21     9K       -       21  256,512
       USBdev    24     7K       -       24  64,128,1024
          USB    44    39K       -       48  16,32,128,256,512,2048
     acpitask     1     2K       -        1  2048
CAM dev queue    10     2K       -       10  128
    CAM queue    41     2K       -      176  16,32,256
     routetbl    41     6K       -      151  32,64,128,256,512
         igmp     7     2K       -        7  256
      acpisem    56     7K       -       56  128
      CAM SIM    10     3K       -       10  256
     in_multi     2     1K       -        3  256
    sctp_iter     0     0K       -        5  256
     sctp_ifn     2     1K       -        2  128
     sctp_ifa     6     1K       -        6  128
     sctp_vrf     1     1K       -        1  64
    sctp_a_it     0     0K       -        5  16
    hostcache     1    28K       -        1  
     syncache     1    96K       -        1  
    in6_multi    24     3K       -       24  32,256
      entropy  1024    64K       -     1024  64
   CAM periph    14     4K       -       39  16,32,64,128,256
          mld     7     1K       -        7  128
          rpc     2     1K       -        2  256
audit_evclass   179     6K       -      218  32
      jblocks     2     1K       -        2  128,256
     savedino     0     0K       -        4  256
        sbdep     0     0K       -        3  64
      jsegdep    26     2K       -      114  64
         jseg     8     1K       -        8  128
    jfreefrag     0     0K       -        1  128
      jnewblk     0     0K       -       29  128
      jremref     0     0K       -       42  128
      jaddref     0     0K       -       42  128
     freework     1     1K       -       26  16,128
    newdirblk     2     1K       -        6  64
       dirrem     0     0K       -       30  128
        mkdir     2     1K       -       12  128
       diradd    16     2K       -       30  128
     freefile     6     1K       -       28  64
     freeblks     0     0K       -       25  128
     freefrag     0     0K       -        1  128
       newblk     5   129K       -       30  256
    bmsafemap     1     8K       -       10  256
     inodedep    26  1037K       -       64  512
      pagedep     5   129K       -       12  256
  ufs_dirhash    30     6K       -       30  16,32,64,128,512
    ufs_mount     3    13K       -        3  512,4096
    vm_pgdata     2   129K       -        2  128
      CAM XPT   137   133K       -      304  32,64,128,256,1024,2048
      scsi_da     0     0K       -       20  16
       kbdmux     6    18K       -        6  16,512,1024,2048
       DEVFS1   109    55K       -      112  512
       DEVFS3   129    33K       -      133  256
      memdesc     1     4K       -        1  4096
        DEVFS    21     1K       -       22  16,128
       DEVFSP     1     1K       -        1  64
     atkbddev     2     1K       -        2  64
       apmdev     1     1K       -        1  128
    pfs_nodes    21     6K       -       21  256
      io_apic     1     2K       -        1  2048
          LED    14     1K       -       14  16,128
         GEOM    90    20K       -      571  16,32,64,128,256,512,1024
         hdac    13    22K       -       14  64,128,256,512,1024,2048,4096
          msi     6     1K       -        6  128
     nexusdev     3     1K       -        3  16
      solaris 36133 16422K       -    74628  16,32,64,128,256,512,1024,2048,4096
   kstat_data     4     1K       -        4  64

------------------------------------------------------------------------
vmstat -z

ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               208,      0,     182,       5,     182,   0,   0
UMA Zones:              896,      0,     182,       2,     182,   0,   0
UMA Slabs:              568,      0,    2216,      38,    3504,   0,   0
UMA RCntSlabs:          568,      0,     669,       3,     669,   0,   0
UMA Hash:               256,      0,      81,       9,      81,   0,   0
16 Bucket:              152,      0,     102,      23,     102,   0,   0
32 Bucket:              280,      0,     113,      13,     113,   0,   0
64 Bucket:              536,      0,     105,       0,     105,  57,   0
128 Bucket:            1048,      0,     102,       0,     102,2030,   0
VM OBJECT:              216,      0,    1146,     132,   18726,   0,   0
MAP:                    232,      0,       7,      25,       7,   0,   0
KMAP ENTRY:             120, 149916,      40,     487,   14383,   0,   0
MAP ENTRY:              120,      0,     623,     152,   41730,   0,   0
fakepg:                 120,      0,       0,       0,       0,   0,   0
mt_zone:               4112,      0,     304,      35,     304,   0,   0
16:                      16,      0,    2066,     622,   23147,   0,   0
32:                      32,      0,    2101,     424,   13436,   0,   0
64:                      64,      0,   46758,     450,   76757,   0,   0
128:                    128,      0,    5163,     608,    8252,   0,   0
256:                    256,      0,     510,     285,    4020,   0,   0
512:                    512,      0,     900,     115,   17533,   0,   0
1024:                  1024,      0,      70,     150,    2735,   0,   0
2048:                  2048,      0,     276,      66,     926,   0,   0
4096:                  4096,      0,     703,     617,   12406,   0,   0
Files:                   80,      0,      76,     329,    5943,   0,   0
TURNSTILE:              136,      0,     340,      80,     340,   0,   0
umtx pi:                 96,      0,       0,       0,       0,   0,   0
MAC labels:              40,      0,       0,       0,       0,   0,   0
PROC:                  1160,      0,      42,      57,    1469,   0,   0
THREAD:                1112,      0,     295,      44,     579,   0,   0
SLEEPQUEUE:              80,      0,     340,      66,     340,   0,   0
VMSPACE:                392,      0,      26,      64,    1454,   0,   0
cpuset:                  72,      0,       2,      98,       2,   0,   0
audit_record:           960,      0,       0,       0,       0,   0,   0
mbuf_packet:            256,      0,    1023,     258,    1241,   0,   0
mbuf:                   256,      0,       1,     384,     484,   0,   0
mbuf_cluster:          2048,  25600,    1280,      48,    1280,   0,   0
mbuf_jumbo_page:       4096,  12800,       0,       5,       5,   0,   0
mbuf_jumbo_9k:         9216,  19200,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384,  12800,       0,       0,       0,   0,   0
mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   0
ttyinq:                 160,      0,     135,      81,     270,   0,   0
ttyoutq:                256,      0,      72,      48,     144,   0,   0
g_bio:                  232,      0,       0,     192,    8737,   0,   0
ata_request:            328,      0,       0,       0,       0,   0,   0
ata_composite:          336,      0,       0,       0,       0,   0,   0
VNODE:                  480,      0,     498,      70,     528,   0,   0
VNODEPOLL:              112,      0,       0,       0,       0,   0,   0
NAMEI:                 1024,      0,       0,      56,    9094,   0,   0
S VFS Cache:            108,      0,     480,     147,    1181,   0,   0
L VFS Cache:            328,      0,       0,       0,       0,   0,   0
DIRHASH:               1024,      0,      42,       6,      42,   0,   0
NCLNODE:                560,      0,       0,       0,       0,   0,   0
pipe:                   728,      0,       3,      62,     957,   0,   0
Mountpoints:            768,      0,       2,       8,       2,   0,   0
ksiginfo:               112,      0,      85,     971,      88,   0,   0
itimer:                 344,      0,       1,      21,       1,   0,   0
KNOTE:                  128,      0,       0,     116,      40,   0,   0
socket:                 680,  25602,      23,      25,     285,   0,   0
unpcb:                  240,  25600,       6,      58,      40,   0,   0
ipq:                     56,    819,       0,       0,       0,   0,   0
udp_inpcb:              392,  25600,      10,      30,     221,   0,   0
udpcb:                   16,  25704,      10,     494,     221,   0,   0
tcp_inpcb:              392,  25600,       4,      46,      10,   0,   0
tcpcb:                  976,  25600,       4,      24,      10,   0,   0
tcptw:                   72,   5150,       0,       0,       0,   0,   0
syncache:               152,  15375,       0,      50,       1,   0,   0
hostcache:              136,  15372,       0,       0,       0,   0,   0
tcpreass:                40,   1680,       0,       0,       0,   0,   0
sackhole:                32,      0,       0,       0,       0,   0,   0
sctp_ep:               1368,  25600,       0,       0,       0,   0,   0
sctp_asoc:             2280,  40000,       0,       0,       0,   0,   0
sctp_laddr:              48,  80064,       0,     288,       5,   0,   0
sctp_raddr:             704,  80000,       0,       0,       0,   0,   0
sctp_chunk:             136, 400008,       0,       0,       0,   0,   0
sctp_readq:             104, 400032,       0,       0,       0,   0,   0
sctp_stream_msg_out:    112, 400026,       0,       0,       0,   0,   0
sctp_asconf:             40, 400008,       0,       0,       0,   0,   0
sctp_asconf_ack:         48, 400032,       0,       0,       0,   0,   0
ripcb:                  392,  25600,       1,      19,       3,   0,   0
rtentry:                200,      0,      20,      56,      21,   0,   0
selfd:                   56,      0,      36,     279,    1735,   0,   0
SWAPMETA:               288, 116519,       0,       0,       0,   0,   0
FFS inode:              168,      0,     456,      94,     484,   0,   0
FFS1 dinode:            128,      0,       0,       0,       0,   0,   0
FFS2 dinode:            256,      0,     456,     114,     484,   0,   0
taskq_zone:              48,      0,       0,     216,       5,   0,   0
zio_cache:              880,      0,       2,     954,    7021,   0,   0
zio_link_cache:          48,      0,       0,    1800,    7316,   0,   0
zio_buf_512:            512,      0,       0,       0,       0,   0,   0
zio_data_buf_512:       512,      0,       0,       0,       0,   0,   0
zio_buf_1024:          1024,      0,       0,       0,       0,   0,   0
zio_data_buf_1024:     1024,      0,       0,       0,       0,   0,   0
zio_buf_1536:          1536,      0,       0,       0,       0,   0,   0
zio_data_buf_1536:     1536,      0,       0,       0,       0,   0,   0
zio_buf_2048:          2048,      0,       0,       0,       0,   0,   0
zio_data_buf_2048:     2048,      0,       0,       0,       0,   0,   0
zio_buf_2560:          2560,      0,       0,       0,       0,   0,   0
zio_data_buf_2560:     2560,      0,       0,       0,       0,   0,   0
zio_buf_3072:          3072,      0,       0,       0,       0,   0,   0
zio_data_buf_3072:     3072,      0,       0,       0,       0,   0,   0
zio_buf_3584:          3584,      0,       0,       0,       0,   0,   0
zio_data_buf_3584:     3584,      0,       0,       0,       0,   0,   0
zio_buf_4096:          4096,      0,       0,       0,       0,   0,   0
zio_data_buf_4096:     4096,      0,       0,       0,       0,   0,   0
zio_buf_5120:          5120,      0,       0,       0,       0,   0,   0
zio_data_buf_5120:     5120,      0,       0,       0,       0,   0,   0
zio_buf_6144:          6144,      0,       0,       0,       0,   0,   0
zio_data_buf_6144:     6144,      0,       0,       0,       0,   0,   0
zio_buf_7168:          7168,      0,       0,       0,       0,   0,   0
zio_data_buf_7168:     7168,      0,       0,       0,       0,   0,   0
zio_buf_8192:          8192,      0,       0,       0,       0,   0,   0
zio_data_buf_8192:     8192,      0,       0,       0,       0,   0,   0
zio_buf_10240:        10240,      0,       0,       0,       0,   0,   0
zio_data_buf_10240:   10240,      0,       0,       0,       0,   0,   0
zio_buf_12288:        12288,      0,       0,       0,       0,   0,   0
zio_data_buf_12288:   12288,      0,       0,       0,       0,   0,   0
zio_buf_14336:        14336,      0,       0,       0,       0,   0,   0
zio_data_buf_14336:   14336,      0,       0,       0,       0,   0,   0
zio_buf_16384:        16384,      0,       0,       0,       0,   0,   0
zio_data_buf_16384:   16384,      0,       0,       0,       0,   0,   0
zio_buf_20480:        20480,      0,       0,       0,       0,   0,   0
zio_data_buf_20480:   20480,      0,       0,       0,       0,   0,   0
zio_buf_24576:        24576,      0,       0,       0,       0,   0,   0
zio_data_buf_24576:   24576,      0,       0,       0,       0,   0,   0
zio_buf_28672:        28672,      0,       0,       0,       0,   0,   0
zio_data_buf_28672:   28672,      0,       0,       0,       0,   0,   0
zio_buf_32768:        32768,      0,       0,       0,       0,   0,   0
zio_data_buf_32768:   32768,      0,       0,       0,       0,   0,   0
zio_buf_36864:        36864,      0,       0,       0,       0,   0,   0
zio_data_buf_36864:   36864,      0,       0,       0,       0,   0,   0
zio_buf_40960:        40960,      0,       0,       0,       0,   0,   0
zio_data_buf_40960:   40960,      0,       0,       0,       0,   0,   0
zio_buf_45056:        45056,      0,       0,       0,       0,   0,   0
zio_data_buf_45056:   45056,      0,       0,       0,       0,   0,   0
zio_buf_49152:        49152,      0,       0,       0,       0,   0,   0
zio_data_buf_49152:   49152,      0,       0,       0,       0,   0,   0
zio_buf_53248:        53248,      0,       0,       0,       0,   0,   0
zio_data_buf_53248:   53248,      0,       0,       0,       0,   0,   0
zio_buf_57344:        57344,      0,       0,       0,       0,   0,   0
zio_data_buf_57344:   57344,      0,       0,       0,       0,   0,   0
zio_buf_61440:        61440,      0,       0,       0,       0,   0,   0
zio_data_buf_61440:   61440,      0,       0,       0,       0,   0,   0
zio_buf_65536:        65536,      0,       0,       0,       0,   0,   0
zio_data_buf_65536:   65536,      0,       0,       0,       0,   0,   0
zio_buf_69632:        69632,      0,       0,       0,       0,   0,   0
zio_data_buf_69632:   69632,      0,       0,       0,       0,   0,   0
zio_buf_73728:        73728,      0,       0,       0,       0,   0,   0
zio_data_buf_73728:   73728,      0,       0,       0,       0,   0,   0
zio_buf_77824:        77824,      0,       0,       0,       0,   0,   0
zio_data_buf_77824:   77824,      0,       0,       0,       0,   0,   0
zio_buf_81920:        81920,      0,       0,       0,       0,   0,   0
zio_data_buf_81920:   81920,      0,       0,       0,       0,   0,   0
zio_buf_86016:        86016,      0,       0,       0,       0,   0,   0
zio_data_buf_86016:   86016,      0,       0,       0,       0,   0,   0
zio_buf_90112:        90112,      0,       0,       0,       0,   0,   0
zio_data_buf_90112:   90112,      0,       0,       0,       0,   0,   0
zio_buf_94208:        94208,      0,       0,       0,       0,   0,   0
zio_data_buf_94208:   94208,      0,       0,       0,       0,   0,   0
zio_buf_98304:        98304,      0,       0,       0,       0,   0,   0
zio_data_buf_98304:   98304,      0,       0,       0,       0,   0,   0
zio_buf_102400:      102400,      0,       0,       0,       0,   0,   0
zio_data_buf_102400: 102400,      0,       0,       0,       0,   0,   0
zio_buf_106496:      106496,      0,       0,       0,       0,   0,   0
zio_data_buf_106496: 106496,      0,       0,       0,       0,   0,   0
zio_buf_110592:      110592,      0,       0,       0,       0,   0,   0
zio_data_buf_110592: 110592,      0,       0,       0,       0,   0,   0
zio_buf_114688:      114688,      0,       0,       0,       0,   0,   0
zio_data_buf_114688: 114688,      0,       0,       0,       0,   0,   0
zio_buf_118784:      118784,      0,       0,       0,       0,   0,   0
zio_data_buf_118784: 118784,      0,       0,       0,       0,   0,   0
zio_buf_122880:      122880,      0,       0,       0,       0,   0,   0
zio_data_buf_122880: 122880,      0,       0,       0,       0,   0,   0
zio_buf_126976:      126976,      0,       0,       0,       0,   0,   0
zio_data_buf_126976: 126976,      0,       0,       0,       0,   0,   0
zio_buf_131072:      131072,      0,       0,       0,       0,   0,   0
zio_data_buf_131072: 131072,      0,       0,       0,       0,   0,   0
sa_cache:                80,      0,       0,       0,       0,   0,   0
dnode_t:                856,      0,     517,      63,    1108,   0,   0
dmu_buf_impl_t:         224,      0,     800,      33,    1137,   0,   0
arc_buf_hdr_t:          216,      0,     466,      56,     553,   0,   0
arc_buf_t:              104,      0,     463,     113,     574,   0,   0
zil_lwb_cache:          192,      0,       0,       0,       0,   0,   0
zfs_znode_cache:        400,      0,       0,       0,       0,   0,   0


------------------------------------------------------------------------
vmstat -i

interrupt                          total       rate
irq18: siis0 uhci2                   228          1
irq23: uhci0 ehci0                 14657         76
cpu0:timer                         17638         92
irq256: hdac0                         39          0
irq257: em0:rx 0                     239          1
irq258: em0:tx 0                     118          0
irq259: em0:link                       2          0
irq260: siis1                        462          2
irq261: ahci0                        505          2
cpu1:timer                          6988         36
cpu3:timer                          4440         23
cpu2:timer                          9051         47
Total                              54367        284

------------------------------------------------------------------------
pstat -T

 76/12328 files
0M/6594M swap space

------------------------------------------------------------------------
pstat -s

Device          512-blocks     Used    Avail Capacity
/dev/da0p3        13505336        0 13505336     0%

------------------------------------------------------------------------
iostat

iostat: kvm_read(_tk_nin): invalid address (0x0)
iostat: disabling TTY statistics
            ada0             ada1             ada2             cpu
  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
 17.21   1  0.02  18.74   1  0.02  20.68   1  0.02   0  0  5  0 94

------------------------------------------------------------------------
ipcs -a

Message Queues:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME   

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME   

Semaphores:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME   


------------------------------------------------------------------------
ipcs -T

msginfo:
	msgmax:        16384	(max characters in a message)
	msgmni:           40	(# of message queues)
	msgmnb:         2048	(max characters in a message queue)
	msgtql:           40	(max # of messages in system)
	msgssz:            8	(size of a message segment)
	msgseg:         2048	(# of message segments in system)

shminfo:
	shmmax:    536870912	(max shared memory segment size)
	shmmin:            1	(min shared memory segment size)
	shmmni:          192	(max number of shared memory identifiers)
	shmseg:          128	(max shared memory segments per process)
	shmall:       131072	(max amount of shared memory in pages)

seminfo:
	semmni:           50	(# of semaphore identifiers)
	semmns:          340	(# of semaphores in system)
	semmnu:          150	(# of undo structures in system)
	semmsl:          340	(max # of semaphores per id)
	semopm:          100	(max # of operations per semop call)
	semume:           50	(max # of undo entries per process)
	semusz:          632	(size in bytes of undo structure)
	semvmx:        32767	(semaphore maximum value)
	semaem:        16384	(adjust on exit max value)


------------------------------------------------------------------------
nfsstat

Client Info:
Rpc Counts:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Rpc Info:
 TimedOut   Invalid X Replies   Retries  Requests
        0         0         0         0         0
Cache Info:
Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW Hits    Misses
        0         0         0         0         0         0         0         0
BioRLHits    Misses BioD Hits    Misses DirE Hits    Misses Accs Hits    Misses
        0         0         0         0         0         0         0         0

Server Info:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Server Ret-Failed
                0
Server Faults
            0
Server Cache Stats:
   Inprog      Idem  Non-idem    Misses
        0         0         0         0
Server Write Gathering:
 WriteOps  WriteRPC   Opsaved
        0         0         0

------------------------------------------------------------------------
netstat -s

tcp:
	75 packets sent
		68 data packets (6513 bytes)
		0 data packets (0 bytes) retransmitted
		0 data packets unnecessarily retransmitted
		0 resends initiated by MTU discovery
		7 ack-only packets (6 delayed)
		0 URG only packets
		0 window probe packets
		0 window update packets
		0 control packets
	129 packets received
		67 acks (for 6514 bytes)
		0 duplicate acks
		0 acks for unsent data
		59 packets (5109 bytes) received in-sequence
		0 completely duplicate packets (0 bytes)
		0 old duplicate packets
		0 packets with some dup. data (0 bytes duped)
		0 out-of-order packets (0 bytes)
		0 packets (0 bytes) of data after window
		0 window probes
		0 window update packets
		0 packets received after close
		0 discarded for bad checksums
		0 discarded for bad header offset fields
		0 discarded because packet too short
		0 discarded due to memory problems
	0 connection requests
	1 connection accept
	0 bad connection attempts
	0 listen queue overflows
	0 ignored RSTs in the windows
	1 connection established (including accepts)
	6 connections closed (including 0 drops)
		0 connections updated cached RTT on close
		0 connections updated cached RTT variance on close
		0 connections updated cached ssthresh on close
	0 embryonic connections dropped
	67 segments updated rtt (of 59 attempts)
	0 retransmit timeouts
		0 connections dropped by rexmit timeout
	0 persist timeouts
		0 connections dropped by persist timeout
	0 Connections (fin_wait_2) dropped because of timeout
	0 keepalive timeouts
		0 keepalive probes sent
		0 connections dropped by keepalive
	2 correct ACK header predictions
	3 correct data packet header predictions
	1 syncache entry added
		0 retransmitted
		0 dupsyn
		0 dropped
		1 completed
		0 bucket overflow
		0 cache overflow
		0 reset
		0 stale
		0 aborted
		0 badack
		0 unreach
		0 zone failures
	1 cookie sent
	0 cookies received
	0 hostcache entries added
		0 bucket overflow
	0 SACK recovery episodes
	0 segment rexmits in SACK recovery episodes
	0 byte rexmits in SACK recovery episodes
	0 SACK options (SACK blocks) received
	0 SACK options (SACK blocks) sent
	0 SACK scoreboard overflow
	0 packets with ECN CE bit set
	0 packets with ECN ECT(0) bit set
	0 packets with ECN ECT(1) bit set
	0 successful ECN handshakes
	0 times ECN reduced the congestion window
udp:
	35 datagrams received
	0 with incomplete header
	0 with bad data length field
	0 with bad checksum
	0 with no checksum
	0 dropped due to no socket
	0 broadcast/multicast datagrams undelivered
	0 dropped due to full socket buffers
	0 not for hashed pcb
	35 delivered
	35 datagrams output
	0 times multicast source filter matched
ip:
	165 total packets received
	0 bad header checksums
	0 with size smaller than minimum
	0 with data size < data length
	0 with ip length > max ip packet size
	0 with header length < data size
	0 with data length < header length
	0 with bad options
	0 with incorrect version number
	0 fragments received
	0 fragments dropped (dup or out of space)
	0 fragments dropped after timeout
	0 packets reassembled ok
	164 packets for this host
	0 packets for unknown/unsupported protocol
	0 packets forwarded (0 packets fast forwarded)
	1 packet not forwardable
	0 packets received for unknown multicast group
	0 redirects sent
	112 packets sent from this host
	0 packets sent with fabricated ip header
	0 output packets dropped due to no bufs, etc.
	1 output packet discarded due to no route
	0 output datagrams fragmented
	0 fragments created
	0 datagrams that can't be fragmented
	0 tunneling packets that can't find gif
	0 datagrams with bad address in header
icmp:
	0 calls to icmp_error
	0 errors not generated in response to an icmp message
	0 messages with bad code fields
	0 messages less than the minimum length
	0 messages with bad checksum
	0 messages with bad length
	0 multicast echo requests ignored
	0 multicast timestamp requests ignored
	0 message responses generated
	0 invalid return addresses
	0 no return routes
igmp:
	0 messages received
	0 messages received with too few bytes
	0 messages received with wrong TTL
	0 messages received with bad checksum
	0 V1/V2 membership queries received
	0 V3 membership queries received
	0 membership queries received with invalid field(s)
	0 general queries received
	0 group queries received
	0 group-source queries received
	0 group-source queries dropped
	0 membership reports received
	0 membership reports received with invalid field(s)
	0 membership reports received for groups to which we belong
	0 V3 reports received without Router Alert
	0 membership reports sent
arp:
	2 ARP requests sent
	2 ARP replies sent
	2 ARP requests received
	1 ARP reply received
	6 ARP packets received
	0 total packets dropped due to no ARP entry
	0 ARP entrys timed out
	0 Duplicate IPs seen
ip6:
	2 total packets received
	0 with size smaller than minimum
	0 with data size < data length
	0 with bad options
	0 with incorrect version number
	0 fragments received
	0 fragments dropped (dup or out of space)
	0 fragments dropped after timeout
	0 fragments that exceeded limit
	0 packets reassembled ok
	2 packets for this host
	0 packets forwarded
	0 packets not forwardable
	0 redirects sent
	7 packets sent from this host
	0 packets sent with fabricated ip header
	0 output packets dropped due to no bufs, etc.
	0 output packets discarded due to no route
	0 output datagrams fragmented
	0 fragments created
	0 datagrams that can't be fragmented
	0 packets that violated scope rules
	1 multicast packet which we don't join
	Input histogram:
		ICMP6: 2
	Mbuf statistics:
		1 one mbuf
		1 one ext mbuf
		0 two or more ext mbuf
	0 packets whose headers are not contiguous
	0 tunneling packets that can't find gif
	0 packets discarded because of too many headers
	0 failures of source address selection
	Source addresses selection rule applied:
		4 first candidate
		3 appropriate scope
icmp6:
	0 calls to icmp6_error
	0 errors not generated in response to an icmp6 message
	0 errors not generated because of rate limitation
	Output histogram:
		router solicitation: 1
		neighbor solicitation: 2
		MLDv2 listener report: 4
	0 messages with bad code fields
	0 messages < minimum length
	0 bad checksums
	0 messages with bad length
	Input histogram:
		router advertisement: 1
	Histogram of error messages to be generated:
		0 no route
		0 administratively prohibited
		0 beyond scope
		0 address unreachable
		0 port unreachable
		0 packet too big
		0 time exceed transit
		0 time exceed reassembly
		0 erroneous header field
		0 unrecognized next header
		0 unrecognized option
		0 redirect
		0 unknown
	0 message responses generated
	0 messages with too many ND options
	0 messages with bad ND options
	0 bad neighbor solicitation messages
	0 bad neighbor advertisement messages
	0 bad router solicitation messages
	0 bad router advertisement messages
	0 bad redirect messages
	0 path MTU changes
rip6:
	0 messages received
	0 checksum calculations on inbound
	0 messages with bad checksum
	0 messages dropped due to no socket
	0 multicast messages dropped due to no socket
	0 messages dropped due to full socket buffers
	0 delivered
	0 datagrams output

------------------------------------------------------------------------
netstat -m

1024/642/1666 mbufs in use (current/cache/total)
1022/306/1328/25600 mbuf clusters in use (current/cache/total/max)
1023/258 mbuf+clusters out of packet secondary zone in use (current/cache)
0/5/5/12800 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/19200 9k jumbo clusters in use (current/cache/total/max)
0/0/0/12800 16k jumbo clusters in use (current/cache/total/max)
2300K/792K/3092K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
0 calls to protocol drain routines

------------------------------------------------------------------------
netstat -id

Name    Mtu Network       Address              Ipkts Ierrs Idrop    Opkts Oerrs  Coll Drop
em0    1500 <Link#1>      00:22:4d:7a:2e:b1      188     0     0      123     0     0    0 
em0    1500 fe80::222:4df fe80::222:4dff:fe        0     -     -        4     -     -    - 
em0    1500 2a01:e35:2ea4 2a01:e35:2ea4:29e        0     -     -        0     -     -    - 
em0    1500 192.168.174.0 192.168.174.202        164     -     -      111     -     -    - 
usbus     0 <Link#2>                               0     0     0        0     0     0    0 
usbus     0 <Link#3>                               0     0     0        0     0     0    0 
usbus     0 <Link#4>                               0     0     0        0     0     0    0 
usbus     0 <Link#5>                               0     0     0        0     0     0    0 
usbus     0 <Link#6>                               0     0     0        0     0     0    0 
lo0   16384 <Link#7>                               0     0     0        0     0     0    0 
lo0   16384 localhost     ::1                      0     -     -        0     -     -    - 
lo0   16384 fe80::1%lo0   fe80::1                  0     -     -        0     -     -    - 
lo0   16384 your-net      localhost                0     -     -        0     -     -    - 

------------------------------------------------------------------------
netstat -anr

Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.174.254    UGS         0       91    em0
127.0.0.1          link#7             UH          0        0    lo0
192.168.174.0/24   link#1             U           0       20    em0
192.168.174.202    link#1             UHS         0        0    lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::/96                             ::1                           UGRS        lo0 =>
default                           fe80::224:d4ff:feac:5a6%em0   UG          em0
::1                               ::1                           UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
2a01:e35:2ea4:29e0::/64           link#1                        U           em0
2a01:e35:2ea4:29e0:222:4dff:fe7a:2eb1 link#1                        UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%em0/64                     link#1                        U           em0
fe80::222:4dff:fe7a:2eb1%em0      link#1                        UHS         lo0
fe80::%lo0/64                     link#7                        U           lo0
fe80::1%lo0                       link#7                        UHS         lo0
ff01::%em0/32                     fe80::222:4dff:fe7a:2eb1%em0  U           em0
ff01::%lo0/32                     ::1                           U           lo0
ff02::/16                         ::1                           UGRS        lo0
ff02::%em0/32                     fe80::222:4dff:fe7a:2eb1%em0  U           em0
ff02::%lo0/32                     ::1                           U           lo0

------------------------------------------------------------------------
netstat -anA

Active Internet connections (including servers)
Tcpcb            Proto Recv-Q Send-Q Local Address      Foreign Address    (state)
fffffe0005e7ab70 tcp4       0      0 192.168.174.202.22 91.90.101.134.3785 ESTABLISHED
fffffe0005ec03d0 tcp4       0      0 127.0.0.1.25       *.*                LISTEN
fffffe0005edc000 tcp4       0      0 *.22               *.*                LISTEN
fffffe0005edc3d0 tcp6       0      0 *.22               *.*                LISTEN
fffffe0005a64620 udp4       0      0 127.0.0.1.123      *.*                
fffffe0005a64498 udp6       0      0 fe80:7::1.123      *.*                
fffffe0005a64188 udp6       0      0 ::1.123            *.*                
fffffe00059707a8 udp4       0      0 192.168.174.202.12 *.*                
fffffe0005970498 udp6       0      0 2a01:e35:2ea4:29.1 *.*                
fffffe0005970620 udp6       0      0 fe80:1::222:4dff.1 *.*                
fffffe0005970930 udp6       0      0 *.123              *.*                
fffffe0005970ab8 udp4       0      0 *.123              *.*                
fffffe0005970c40 udp4       0      0 *.514              *.*                
fffffe0005970dc8 udp6       0      0 *.514              *.*                
Active UNIX domain sockets
Address  Type   Recv-Q Send-Q    Inode     Conn     Refs  Nextref Addr
fffffe0005d960f0 stream      0      0 fffffe0005a16b40        0        0        0 /var/run/devd.pipe
fffffe0005a593c0 dgram       0      0        0 fffffe0005a59690        0        0
fffffe0005dea3c0 dgram       0      0        0 fffffe0005dea4b0        0 fffffe0005e424b0
fffffe0005e424b0 dgram       0      0        0 fffffe0005dea4b0        0        0
fffffe0005dea4b0 dgram       0      0 fffffe0005e18960        0 fffffe0005dea3c0        0 /var/run/logpriv
fffffe0005a59690 dgram       0      0 fffffe0005dc61e0        0 fffffe0005a593c0        0 /var/run/log

------------------------------------------------------------------------
netstat -aL

Current listen queue sizes (qlen/incqlen/maxqlen)
Proto Listen         Local Address         
tcp4  0/0/10         localhost.smtp         
tcp4  0/0/128        *.ssh                  
tcp6  0/0/128        *.ssh                  
unix  0/0/4          /var/run/devd.pipe

------------------------------------------------------------------------
fstat

USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
root     zfskern     1469 root /             2 drwxr-xr-x    1024  r
root     zfskern     1469   wd /             2 drwxr-xr-x    1024  r
root     zpool       1468 root /             2 drwxr-xr-x    1024  r
root     zpool       1468   wd /        142081 drwxr-xr-x     512  r
root     zpool       1468 text /         94723 -r-xr-xr-x   89176  r
root     zpool       1468 ctty /dev        116 crw--w----   pts/0 rw
root     zpool       1468    0 /dev        116 crw--w----   pts/0 rw
root     zpool       1468    1 /dev        116 crw--w----   pts/0 rw
root     zpool       1468    2 /dev        116 crw--w----   pts/0 rw
root     zpool       1468    3 /dev        119 crw-rw-rw-     zfs rw
root     zpool       1468    4 /dev         32 crw-rw-rw-    zero  r
root     bash        1466 root /             2 drwxr-xr-x    1024  r
root     bash        1466   wd /        142081 drwxr-xr-x     512  r
root     bash        1466 text /        403841 -rwxr-xr-x  765608  r
root     bash        1466 ctty /dev        116 crw--w----   pts/0 rw
root     bash        1466    0 /dev        116 crw--w----   pts/0 rw
root     bash        1466    1 /dev        116 crw--w----   pts/0 rw
root     bash        1466    2 /dev        116 crw--w----   pts/0 rw
root     bash        1466  255 /dev        116 crw--w----   pts/0 rw
root     csh         1462 root /             2 drwxr-xr-x    1024  r
root     csh         1462   wd /        142081 drwxr-xr-x     512  r
root     csh         1462 text /         47367 -r-xr-xr-x  368896  r
root     csh         1462 ctty /dev        116 crw--w----   pts/0 rw
root     csh         1462   15 /dev        116 crw--w----   pts/0 rw
root     csh         1462   16 /dev        116 crw--w----   pts/0 rw
root     csh         1462   17 /dev        116 crw--w----   pts/0 rw
root     csh         1462   18 /dev        116 crw--w----   pts/0 rw
root     csh         1462   19 /dev        116 crw--w----   pts/0 rw
root     sshd        1460 root /             2 drwxr-xr-x    1024  r
root     sshd        1460   wd /             2 drwxr-xr-x    1024  r
root     sshd        1460 text /        338811 -r-xr-xr-x  269544  r
root     sshd        1460    0 /dev         31 crw-rw-rw-    null rw
root     sshd        1460    1 /dev         31 crw-rw-rw-    null rw
root     sshd        1460    2 /dev         31 crw-rw-rw-    null rw
root     sshd        1460    3* internet stream tcp fffffe0005e7ab70
root     sshd        1460    4* pipe fffffe0005a8f2d8 <-> fffffe0005a8f430      0 rw
root     sshd        1460    5* pipe fffffe0005a8f430 <-> fffffe0005a8f2d8      0 rw
root     sshd        1460    6* pseudo-terminal master      pts/0 rw
root     sshd        1460    8* pseudo-terminal master      pts/0 rw
root     sshd        1460    9* pseudo-terminal master      pts/0 rw
root     getty       1459 root /             2 drwxr-xr-x    1024  r
root     getty       1459   wd /             2 drwxr-xr-x    1024  r
root     getty       1459 text /        339084 -r-xr-xr-x   27512  r
root     getty       1459 ctty /dev         50 crw-------   ttyv7 rw
root     getty       1459    0 /dev         50 crw-------   ttyv7 rw
root     getty       1459    1 /dev         50 crw-------   ttyv7 rw
root     getty       1459    2 /dev         50 crw-------   ttyv7 rw
root     getty       1458 root /             2 drwxr-xr-x    1024  r
root     getty       1458   wd /             2 drwxr-xr-x    1024  r
root     getty       1458 text /        339084 -r-xr-xr-x   27512  r
root     getty       1458 ctty /dev         49 crw-------   ttyv6 rw
root     getty       1458    0 /dev         49 crw-------   ttyv6 rw
root     getty       1458    1 /dev         49 crw-------   ttyv6 rw
root     getty       1458    2 /dev         49 crw-------   ttyv6 rw
root     getty       1457 root /             2 drwxr-xr-x    1024  r
root     getty       1457   wd /             2 drwxr-xr-x    1024  r
root     getty       1457 text /        339084 -r-xr-xr-x   27512  r
root     getty       1457 ctty /dev         48 crw-------   ttyv5 rw
root     getty       1457    0 /dev         48 crw-------   ttyv5 rw
root     getty       1457    1 /dev         48 crw-------   ttyv5 rw
root     getty       1457    2 /dev         48 crw-------   ttyv5 rw
root     getty       1456 root /             2 drwxr-xr-x    1024  r
root     getty       1456   wd /             2 drwxr-xr-x    1024  r
root     getty       1456 text /        339084 -r-xr-xr-x   27512  r
root     getty       1456 ctty /dev         47 crw-------   ttyv4 rw
root     getty       1456    0 /dev         47 crw-------   ttyv4 rw
root     getty       1456    1 /dev         47 crw-------   ttyv4 rw
root     getty       1456    2 /dev         47 crw-------   ttyv4 rw
root     getty       1455 root /             2 drwxr-xr-x    1024  r
root     getty       1455   wd /             2 drwxr-xr-x    1024  r
root     getty       1455 text /        339084 -r-xr-xr-x   27512  r
root     getty       1455 ctty /dev         46 crw-------   ttyv3 rw
root     getty       1455    0 /dev         46 crw-------   ttyv3 rw
root     getty       1455    1 /dev         46 crw-------   ttyv3 rw
root     getty       1455    2 /dev         46 crw-------   ttyv3 rw
root     getty       1454 root /             2 drwxr-xr-x    1024  r
root     getty       1454   wd /             2 drwxr-xr-x    1024  r
root     getty       1454 text /        339084 -r-xr-xr-x   27512  r
root     getty       1454 ctty /dev         45 crw-------   ttyv2 rw
root     getty       1454    0 /dev         45 crw-------   ttyv2 rw
root     getty       1454    1 /dev         45 crw-------   ttyv2 rw
root     getty       1454    2 /dev         45 crw-------   ttyv2 rw
root     getty       1453 root /             2 drwxr-xr-x    1024  r
root     getty       1453   wd /             2 drwxr-xr-x    1024  r
root     getty       1453 text /        339084 -r-xr-xr-x   27512  r
root     getty       1453 ctty /dev         44 crw-------   ttyv1 rw
root     getty       1453    0 /dev         44 crw-------   ttyv1 rw
root     getty       1453    1 /dev         44 crw-------   ttyv1 rw
root     getty       1453    2 /dev         44 crw-------   ttyv1 rw
root     getty       1452 root /             2 drwxr-xr-x    1024  r
root     getty       1452   wd /             2 drwxr-xr-x    1024  r
root     getty       1452 text /        339084 -r-xr-xr-x   27512  r
root     getty       1452 ctty /dev         43 crw-------   ttyv0 rw
root     getty       1452    0 /dev         43 crw-------   ttyv0 rw
root     getty       1452    1 /dev         43 crw-------   ttyv0 rw
root     getty       1452    2 /dev         43 crw-------   ttyv0 rw
root     sleep       1449 root /             2 drwxr-xr-x    1024  r
root     sleep       1449   wd /             2 drwxr-xr-x    1024  r
root     sleep       1449 text /         47395 -r-xr-xr-x    5496  r
root     sleep       1449    0 /dev         31 crw-rw-rw-    null  r
root     sleep       1449    1* pipe fffffe00059a59e0 <-> fffffe00059a5888      0 rw
root     sleep       1449    2* pipe fffffe00059a59e0 <-> fffffe00059a5888      0 rw
root     logger      1448 root /             2 drwxr-xr-x    1024  r
root     logger      1448   wd /             2 drwxr-xr-x    1024  r
root     logger      1448 text /        342162 -r-xr-xr-x   12016  r
root     logger      1448    0* pipe fffffe00059a5888 <-> fffffe00059a59e0      0 rw
root     logger      1448    2 -         -         bad    -
root     sh          1447 root /             2 drwxr-xr-x    1024  r
root     sh          1447   wd /             2 drwxr-xr-x    1024  r
root     sh          1447 text /         47394 -r-xr-xr-x  142880  r
root     sh          1447    0 /dev         31 crw-rw-rw-    null  r
root     sh          1447    1* pipe fffffe00059a59e0 <-> fffffe00059a5888      0 rw
root     sh          1447    2* pipe fffffe00059a59e0 <-> fffffe00059a5888      0 rw
root     cron        1386 root /             2 drwxr-xr-x    1024  r
root     cron        1386   wd /        142089 drwxr-x---     512  r
root     cron        1386 text /        338869 -r-xr-xr-x   41344  r
root     cron        1386    0 /dev         31 crw-rw-rw-    null rw
root     cron        1386    1 /dev         31 crw-rw-rw-    null rw
root     cron        1386    2 /dev         31 crw-rw-rw-    null rw
root     cron        1386    3 /        234165 -rw-------       4  w
smmsp    sendmail    1380 root /             2 drwxr-xr-x    1024  r
smmsp    sendmail    1380   wd /        142112 drwxrwx---     512  r
smmsp    sendmail    1380 text /        339132 -r-xr-sr-x  707160  r
smmsp    sendmail    1380    0 /dev         31 crw-rw-rw-    null  r
smmsp    sendmail    1380    1 /dev         31 crw-rw-rw-    null  w
smmsp    sendmail    1380    2 /dev         31 crw-rw-rw-    null  w
smmsp    sendmail    1380    3* local dgram fffffe0005a593c0 <-> fffffe0005a59690
smmsp    sendmail    1380    4 /        180428 -rw-------      50  w
root     sendmail    1374 root /             2 drwxr-xr-x    1024  r
root     sendmail    1374   wd /        142109 drwxr-xr-x     512  r
root     sendmail    1374 text /        339132 -r-xr-sr-x  707160  r
root     sendmail    1374    0 /dev         31 crw-rw-rw-    null  r
root     sendmail    1374    1 /dev         31 crw-rw-rw-    null  w
root     sendmail    1374    2 /dev         31 crw-rw-rw-    null  w
root     sendmail    1374    3* internet stream tcp fffffe0005ec03d0
root     sendmail    1374    4* local dgram fffffe0005dea3c0 <-> fffffe0005dea4b0
root     sendmail    1374    5 /        234164 -rw-------      79  w
root     sshd        1367 root /             2 drwxr-xr-x    1024  r
root     sshd        1367   wd /             2 drwxr-xr-x    1024  r
root     sshd        1367 text /        338811 -r-xr-xr-x  269544  r
root     sshd        1367    0 /dev         31 crw-rw-rw-    null rw
root     sshd        1367    1 /dev         31 crw-rw-rw-    null rw
root     sshd        1367    2 /dev         31 crw-rw-rw-    null rw
root     sshd        1367    3* internet6 stream tcp fffffe0005edc3d0
root     sshd        1367    4* internet stream tcp fffffe0005edc000
root     ntpd        1330 root /             2 drwxr-xr-x    1024  r
root     ntpd        1330   wd /             2 drwxr-xr-x    1024  r
root     ntpd        1330 text /        338972 -r-xr-xr-x  391448  r
root     ntpd        1330    0 /dev         31 crw-rw-rw-    null rw
root     ntpd        1330    1 /dev         31 crw-rw-rw-    null rw
root     ntpd        1330    2 /dev         31 crw-rw-rw-    null rw
root     ntpd        1330    3* local dgram fffffe0005e424b0 <-> fffffe0005dea4b0
root     ntpd        1330   20* internet dgram udp fffffe0005970ab8
root     ntpd        1330   21* internet6 dgram udp fffffe0005970930
root     ntpd        1330   22* internet6 dgram udp fffffe0005970620
root     ntpd        1330   23* internet6 dgram udp fffffe0005970498
root     ntpd        1330   24* internet dgram udp fffffe00059707a8
root     ntpd        1330   25* internet6 dgram udp fffffe0005a64188
root     ntpd        1330   26* internet6 dgram udp fffffe0005a64498
root     ntpd        1330   27* internet dgram udp fffffe0005a64620
root     ntpd        1330   28* route raw 0 fffffe0005a617f8
root     watchdogd   1263 root /             2 drwxr-xr-x    1024  r
root     watchdogd   1263   wd /             2 drwxr-xr-x    1024  r
root     watchdogd   1263 text /        339056 -r-xr-xr-x    9984  r
root     watchdogd   1263    0 /dev         31 crw-rw-rw-    null rw
root     watchdogd   1263    1 /dev         31 crw-rw-rw-    null rw
root     watchdogd   1263    2 /dev         31 crw-rw-rw-    null rw
root     watchdogd   1263    3 /dev         17 crw-------    fido rw
root     watchdogd   1263    4 /        234162 -rw-------       4  w
root     syslogd     1125 root /             2 drwxr-xr-x    1024  r
root     syslogd     1125   wd /             2 drwxr-xr-x    1024  r
root     syslogd     1125 text /        339035 -r-xr-xr-x   39808  r
root     syslogd     1125    0 /dev         31 crw-rw-rw-    null rw
root     syslogd     1125    1 /dev         31 crw-rw-rw-    null rw
root     syslogd     1125    2 /dev         31 crw-rw-rw-    null rw
root     syslogd     1125    3 /        187116 -rw-------       4  w
root     syslogd     1125    4* local dgram fffffe0005a59690
root     syslogd     1125    5* local dgram fffffe0005dea4b0
root     syslogd     1125    6* internet6 dgram udp fffffe0005970dc8
root     syslogd     1125    7* internet dgram udp fffffe0005970c40
root     syslogd     1125    8 /dev          9 crw-------    klog  r
root     syslogd     1125   10 -         -         bad    -
root     syslogd     1125   11 /        180433 -rw-r--r--   52193  w
root     syslogd     1125   12 /        180398 -rw-------      70  w
root     syslogd     1125   13 /        180391 -rw-------    7257  w
root     syslogd     1125   14 /        142145 -rw-r-----    3561  w
root     syslogd     1125   15 /        180394 -rw-r--r--      70  w
root     syslogd     1125   16 /        180399 -rw-------      70  w
root     syslogd     1125   17 /        180392 -rw-------   25301  w
root     syslogd     1125   18 /        180393 -rw-------      70  w
root     syslogd     1125   19 /        180397 -rw-r-----      70  w
root     devd         984 root /             2 drwxr-xr-x    1024  r
root     devd         984   wd /             2 drwxr-xr-x    1024  r
root     devd         984 text /         94735 -r-xr-xr-x  423552  r
root     devd         984    0 /dev         31
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2012-07-23 02:07:23 UTC 
Responsible Changed
From-To: freebsd-amd64->freebsd-fs

reclassify.
Comment 2 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:59:20 UTC 
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped