Attachment 158219 Details for Bug 201245 – core.txt.0 from panic

core.txt.0 from panic

core.txt.0.txt (text/plain), 57.58 KB, created by Kurt Lidl on 2015-07-01 12:13:11 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Kurt Lidl

Created: 2015-07-01 12:13:11 UTC

Size: 57.58 KB

patch

obsolete

>spork.pix.net dumped core - see /var/crash/vmcore.0
>
>Tue Jun 30 21:56:37 EDT 2015
>
>FreeBSD spork.pix.net 10.2-PRERELEASE FreeBSD 10.2-PRERELEASE #0 r284813: Thu Jun 25 21:49:53 UTC 2015     root@releng1.nyi.freebsd.org:/usr/obj/sparc64.sparc64/usr/src/sys/GENERIC  sparc64
>
>panic: spin lock held too long
>
>GNU gdb 6.1.1 [FreeBSD]
>Copyright 2004 Free Software Foundation, Inc.
>GDB is free software, covered by the GNU General Public License, and you are
>welcome to change it and/or distribute copies of it under certain conditions.
>Type "show copying" to see the conditions.
>There is absolutely no warranty for GDB.  Type "show warranty" for details.
>This GDB was configured as "sparc64-marcel-freebsd"...
>
>Unread portion of the kernel message buffer:
>Copyright (c) 1992-2015 The FreeBSD Project.
>Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>	The Regents of the University of California. All rights reserved.
>FreeBSD is a registered trademark of The FreeBSD Foundation.
>FreeBSD 10.2-PRERELEASE #0 r284813: Thu Jun 25 21:49:53 UTC 2015
>    root@releng1.nyi.freebsd.org:/usr/obj/sparc64.sparc64/usr/src/sys/GENERIC sparc64
>gcc version 4.2.1 20070831 patched [FreeBSD]
>real memory  = 8589934592 (8192 MB)
>avail memory = 8379719680 (7991 MB)
>cpu0: Sun Microsystems UltraSparc-IIIi Processor (1503.00 MHz CPU)
>cpu1: Sun Microsystems UltraSparc-IIIi Processor (1503.00 MHz CPU)
>FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
>random: <Software, Yarrow> initialized
>kbd0 at kbdmux0
>nexus0: <Open Firmware Nexus device>
>pcib0: <Sun Host-PCI bridge> mem 0x4000ff00000-0x4000ff0afff,0x4000fc10000-0x4000fc1701f,0x7f600000000-0x7f6000000ff,0x4000ff80000-0x4000ff8ffff irq 2035,2032,2033,2036,2019 on nexus0
>pcib0: Tomatillo, version 4, IGN 0x1f, bus B, PCI mode, 66MHz
>pcib0: DVMA map: 0xc0000000 to 0xdfffffff 65536 entries
>pci0: <OFW PCI bus> on pcib0
>pcib1: <Sun Host-PCI bridge> mem 0x4000f600000-0x4000f60afff,0x4000f410000-0x4000f41701f,0x7fe00000000-0x7fe000000ff,0x4000f780000-0x4000f78ffff irq 1970,1968,1969,1972,1953 on nexus0
>pcib1: Tomatillo, version 4, IGN 0x1e, bus A, PCI mode, 33MHz
>pcib1: DVMA map: 0xc0000000 to 0xdfffffff 65536 entries
>pci1: <OFW PCI bus> on pcib1
>pcib2: <Sun Host-PCI bridge> mem 0x4000e600000-0x4000e60afff,0x4000e410000-0x4000e41701f,0x7ce00000000-0x7ce000000ff,0x4000e780000-0x4000e78ffff irq 1842,1840,1841,1844,1827 on nexus0
>pcib2: Tomatillo, version 4, IGN 0x1c, bus A, PCI mode, 66MHz
>pcib2: DVMA map: 0xc0000000 to 0xdfffffff 65536 entries
>pci2: <OFW PCI bus> on pcib2
>pcib3: <Sun Host-PCI bridge> mem 0x4000ef00000-0x4000ef0afff,0x4000ec10000-0x4000ec1701f,0x7c600000000-0x7c6000000ff,0x4000ef80000-0x4000ef8ffff irq 1907,1904,1905,1908,1893 on nexus0
>pcib3: Tomatillo, version 4, IGN 0x1d, bus B, PCI mode, 66MHz
>pcib3: DVMA map: 0xc0000000 to 0xdfffffff 65536 entries
>pci3: <OFW PCI bus> on pcib3
>nexus0: <memory-controller> mem 0x40000000000-0x40000000007 type memory-controller (no driver attached)
>nexus0: <memory-controller> mem 0x40000800000-0x40000800007 type memory-controller (no driver attached)
>bge0: <Broadcom BCM5704 A3, ASIC rev. 0x002003> mem 0x200000-0x20ffff,0x110000-0x11ffff at device 2.0 on pci0
>bge0: CHIP ID 0x00002003; ASIC REV 0x02; CHIP REV 0x20; PCI on PCI-X 66 MHz; 64bit
>miibus0: <MII bus> on bge0
>brgphy0: <BCM5704 1000BASE-T media interface> PHY 1 on miibus0
>brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
>bge0: Ethernet address: 00:03:ba:e0:ce:07
>bge1: <Broadcom BCM5704 A3, ASIC rev. 0x002003> mem 0x400000-0x40ffff,0x120000-0x12ffff at device 2.1 on pci0
>bge1: CHIP ID 0x00002003; ASIC REV 0x02; CHIP REV 0x20; PCI on PCI-X 66 MHz; 64bit
>miibus1: <MII bus> on bge1
>brgphy1: <BCM5704 1000BASE-T media interface> PHY 1 on miibus1
>brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
>bge1: Ethernet address: 00:03:ba:e0:ce:08
>isab0: <PCI-ISA bridge> at device 7.0 on pci1
>isa0: <ISA bus> on isab0
>pci1: <old, non-VGA display device> at device 6.0 (no driver attached)
>ohci0: <AcerLabs M5237 (Aladdin-V) USB controller> mem 0x1000000-0x1000fff at device 10.0 on pci1
>usbus0 on ohci0
>atapci0: <AcerLabs M5229 UDMA100 controller> port 0x900-0x907,0x918-0x91b,0x910-0x917,0x908-0x90b,0x920-0x92f at device 13.0 on pci1
>atapci0: using PIO transfers above 137GB as workaround for 48bit DMA access bug, expect reduced performance
>ata2: <ATA channel> at channel 0 on atapci0
>ata3: <ATA channel> at channel 1 on atapci0
>sym0: <1010-66> port 0x900-0x9ff mem 0x100000-0x1003ff,0x102000-0x103fff at device 2.0 on pci2
>sym0: No NVRAM, ID 7, Fast-80, LVD, parity checking
>sym1: <1010-66> port 0xa00-0xaff mem 0x104000-0x1043ff,0x106000-0x107fff at device 2.1 on pci2
>sym1: No NVRAM, ID 7, Fast-80, LVD, parity checking
>bge2: <Broadcom BCM5704 A3, ASIC rev. 0x002003> mem 0x200000-0x20ffff,0x110000-0x11ffff at device 2.0 on pci3
>bge2: CHIP ID 0x00002003; ASIC REV 0x02; CHIP REV 0x20; PCI on PCI-X 66 MHz; 64bit
>miibus2: <MII bus> on bge2
>brgphy2: <BCM5704 1000BASE-T media interface> PHY 1 on miibus2
>brgphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
>bge2: Ethernet address: 00:03:ba:e0:ce:09
>bge3: <Broadcom BCM5704 A3, ASIC rev. 0x002003> mem 0x400000-0x40ffff,0x120000-0x12ffff at device 2.1 on pci3
>bge3: CHIP ID 0x00002003; ASIC REV 0x02; CHIP REV 0x20; PCI on PCI-X 66 MHz; 64bit
>miibus3: <MII bus> on bge3
>brgphy3: <BCM5704 1000BASE-T media interface> PHY 1 on miibus3
>brgphy3:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
>bge3: Ethernet address: 00:03:ba:e0:ce:0a
>nexus0: <syscons> type unknown (no driver attached)
>rtc0: <Real-Time Clock> at port 0x70-0x71 on isa0
>uart0: <16550 or compatible> at port 0x3f8-0x3ff irq 44 on isa0
>uart0: console (9600,n,8,1)
>uart1: <16550 or compatible> at port 0x2e8-0x2ef irq 44 on isa0
>usbus0: 12Mbps Full Speed USB v1.0
>ZFS filesystem version: 5
>ZFS storage pool version: features support (5000)
>Timecounter "tick" frequency 1503000000 Hz quality 10
>Timecounter "stick" frequency 12000000 Hz quality 10
>Event timer "tick" frequency 1503000000 Hz quality 1000
>Timecounters tick every 1.000 msec
>ugen0.1: <AcerLabs> at usbus0
>uhub0: <AcerLabs OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
>uhub0: 2 ports with 2 removable, self powered
>random: unblocking device.
>da0 at sym0 bus 0 scbus2 target 0 lun 0
>da0: <HITACHI HUS10733ASUN72G PA05> Fixed Direct Access SCSI-3 device
>SMP: AP CPU #1 Launched!
>da0: Serial Number 10RW6DG9          50V3W6DG9A
>da0: 160.000MB/s transfers (80.000MHz DT, offset 62, 16bit)
>da0: Command Queueing enabled
>da0: 70007MB (143374738 512 byte sectors: 255H 63S/T 8924C)
>da3 at sym0 bus 0 scbus2 target 3 lun 0
>da3: <HITACHI HUS10733ASUN72G PA05> Fixed Direct Access SCSI-3 device
>da3: Serial Number 10RW6NXT          50V3W6NXTA
>da3: 160.000MB/s transfers (80.000MHz DT, offset 62, 16bit)
>da3: Command Queueing enabled
>da3: 70007MB (143374738 512 byte sectors: 255H 63S/T 8924C)
>da2 at sym0 bus 0 scbus2 target 2 lun 0
>da2: <SEAGATE ST373405LC 2203> Fixed Direct Access SCSI-3 device
>da2: Serial Number 3EK1AXHL00007237WL46
>da2: 160.000MB/s transfers (80.000MHz DT, offset 62, 16bit)
>da2: Command Queueing enabled
>da2: 70007MB (143374650 512 byte sectors: 255H 63S/T 8924C)
>da1 at sym0 bus 0 scbus2 target 1 lun 0
>da1: <IBM-ESXS ST373307LC    FN B25H> Fixed Direct Access SCSI-3 device
>da1: Serial Number 3HZ9AC0B00007512MDRA
>da1: 160.000MB/s transfers (80.000MHz DT, offset 62, 16bit)
>da1: Command Queueing enabled
>da1: 70006MB (143374000 512 byte sectors: 255H 63S/T 8924C)
>cd0 at ata2 bus 0 scbus0 target 0 lun 0
>cd0: <TEAC DV-28E-C 1.4B> Removable CD-ROM SCSI device
>cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
>cd0: cd present [256000 x 2048 byte records]
>GEOM_MIRROR: Device mirror/gswap launched (2/2).
>GEOM_MIRROR: Device gswap already configured.
>GEOM_MIRROR: Device gswap already configured.
>GEOM_MIRROR: Device gswap already configured.
>GEOM_MIRROR: Device gswap already configured.
>Trying to mount root from zfs:sys/ROOT/default []...
><118>Setting hostuuid: 633f56fe-1f8f-11e5-8d92-0003bae0ce07.
><118>Setting hostid: 0xbab5179d.
><118>Entropy harvesting: interrupts ethernet point_to_point swi.
><118>Starting file system checks:
><118>Mounting local file systems:.
><118>Writing entropy file:.
><118>Setting hostname: spork.pix.net.
>spin lock 0xc0cb9e38 (smp rendezvous) held by 0xfffff80003e93240 (tid 100340) too long
>timeout stopping cpus
>panic: spin lock held too long
>cpuid = 1
>KDB: stack backtrace:
>#0 0xc0575380 at panic+0x20
>#1 0xc0558e10 at _mtx_lock_spin_failed+0x50
>#2 0xc0558ed8 at _mtx_lock_spin_cookie+0xb8
>#3 0xc08d7b9c at tick_get_timecount_mp+0xdc
>#4 0xc0583c88 at binuptime+0x48
>#5 0xc08a3b8c at timercb+0x6c
>#6 0xc08d7f00 at tick_intr+0x220
>Uptime: 29s
>Dumping 8192 MB (4 chunks)
>  chunk at 0: 2147483648 bytes |/-\|
>
>Reading symbols from /boot/kernel/zfs.ko.symbols...done.
>Loaded symbols for /boot/kernel/zfs.ko.symbols
>Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
>Loaded symbols for /boot/kernel/opensolaris.ko.symbols
>Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done.
>Loaded symbols for /boot/kernel/geom_mirror.ko.symbols
>Reading symbols from /boot/kernel/tmpfs.ko.symbols...done.
>Loaded symbols for /boot/kernel/tmpfs.ko.symbols
>#0  0x00000000c05745bc in doadump (textdump=<value optimized out>)
>    at /usr/src/sys/kern/kern_shutdown.c:262
>262		savectx(&dumppcb);
>(kgdb) #0  0x00000000c05745bc in doadump (textdump=<value optimized out>)
>    at /usr/src/sys/kern/kern_shutdown.c:262
>#1  0x00000000c0574fb0 in kern_reboot (howto=260)
>    at /usr/src/sys/kern/kern_shutdown.c:451
>#2  0x00000000c0575358 in vpanic (fmt=0xc0b22fe0 "spin lock held too long", 
>    ap=0x1fa2da638) at /usr/src/sys/kern/kern_shutdown.c:758
>#3  0x00000000c0575388 in panic (fmt=0xc0b22fe0 "spin lock held too long")
>    at /usr/src/sys/kern/kern_shutdown.c:687
>#4  0x00000000c0558e18 in _mtx_lock_spin_failed (m=0xc0cb9e38)
>    at /usr/src/sys/kern/kern_mutex.c:561
>#5  0x00000000c0558ee0 in _mtx_lock_spin_cookie (c=0xfffff80003e93240, 
>    tid=18446735277669594832, opts=0, file=0x0, line=0)
>    at /usr/src/sys/kern/kern_mutex.c:608
>#6  0x00000000c08d7ba4 in tick_get_timecount_mp (tc=0xc0d13378) at smp.h:206
>#7  0x00000000c0583c90 in binuptime (bt=0x1fa2da980)
>    at /usr/src/sys/kern/kern_tc.c:188
>#8  0x00000000c08a3b94 in timercb (et=0xc0d13308, arg=<value optimized out>)
>    at time.h:418
>#9  0x00000000c08d7f08 in tick_intr (tf=0x1fa2dab20)
>    at /usr/src/sys/sparc64/sparc64/tick.c:252
>#10 0x00000000c00a11bc in tl1_intr ()
>#11 0x00000000c08c934c in spinlock_exit ()
>    at /usr/src/sys/sparc64/sparc64/machdep.c:244
>#12 0x00000000c08c9330 in spinlock_exit ()
>    at /usr/src/sys/sparc64/sparc64/machdep.c:240
>#13 0x00000000c051a194 in cnputs (p=0x1fa2db11a "")
>    at /usr/src/sys/kern/kern_cons.c:530
>#14 0x00000000c05c06e0 in putchar (c=10, arg=0x1fa2db0c8)
>    at /usr/src/sys/kern/subr_prf.c:437
>#15 0x00000000c05bee90 in kvprintf (fmt=0xc0b2fb95 "", 
>    func=0xc05c02e0 <putchar>, arg=0x1fa2db0c8, radix=10, ap=0x1fa2db300)
>    at /usr/src/sys/kern/subr_prf.c:655
>#16 0x00000000c05bfe80 in _vprintf (level=5, flags=1, 
>    fmt=0xc0b2fb78 "%s: link state changed to %s\n", ap=0x1fa2db2f0)
>    at /usr/src/sys/kern/subr_prf.c:281
>#17 0x00000000c05c0270 in log (level=5, 
>    fmt=0xc0b2fb78 "%s: link state changed to %s\n")
>    at /usr/src/sys/kern/subr_prf.c:308
>#18 0x00000000c064ec28 in do_link_state_change (arg=0xfffff80003396800, 
>    pending=1) at /usr/src/sys/net/if.c:2131
>#19 0x00000000c05cab38 in taskqueue_run_locked (queue=0xfffff80003288000)
>    at /usr/src/sys/kern/subr_taskqueue.c:342
>#20 0x00000000c05cacec in taskqueue_run (queue=0xfffff80003288000)
>    at /usr/src/sys/kern/subr_taskqueue.c:358
>#21 0x00000000c05cae20 in taskqueue_swi_run (dummy=0x0)
>    at /usr/src/sys/kern/subr_taskqueue.c:471
>#22 0x00000000c0539cc4 in intr_event_execute_handlers (p=0xfffff80003295860, 
>    ie=0xfffff80003287e00) at /usr/src/sys/kern/kern_intr.c:1264
>#23 0x00000000c053b86c in ithread_loop (arg=0xfffff8000324c080)
>    at /usr/src/sys/kern/kern_intr.c:1277
>#24 0x00000000c0536428 in fork_exit (callout=0xc053b780 <ithread_loop>, 
>    arg=0xfffff8000324c080, frame=0x1fa2db880)
>    at /usr/src/sys/kern/kern_fork.c:1018
>#25 0x00000000c00a1270 in fork_trampoline ()
>#26 0x00000000c00a1270 in fork_trampoline ()
>Previous frame identical to this frame (corrupt stack?)
>(kgdb) 
>
>------------------------------------------------------------------------
>ps -axlww
>
>UID PID PPID CPU PRI NI   VSZ               RSS MWCHAN   STAT TT     TIME COMMAND
>  0   0    0   0  -8  0     0                 0 -        DLs   -  0:00.08 [kernel]
>  0   1    0   0  21  0  9416 18014398509481976 wait     DLs   -  0:00.00 [init]
>  0   2    0   0 -16  0     0                 0 -        DL    -  0:00.00 [cam]
>  0   3    0   0 -16  0     0                 0 4        DL    -  0:00.00 [zfskern]
>  0   4    0   0 -16  0     0                 0 waiting_ DL    -  0:00.00 [sctp_iterator]
>  0   5    0   0 -16  0     0                 0 umarcl   DL    -  0:00.00 [pagedaemon]
>  0   6    0   0 -16  0     0                 0 psleep   DL    -  0:00.00 [vmdaemon]
>  0   7    0   0 155  0     0                 0 pgzero   DL    -  0:00.00 [pagezero]
>  0   8    0   0 -16  0     0                 0 psleep   DL    -  0:00.00 [bufdaemon]
>  0   9    0   0 -16  0     0                 0 vlruwt   DL    -  0:00.00 [vnlru]
>  0  10    0   0 -16  0     0                 0 audit_wo DL    -  0:00.00 [audit]
>  0  11    0   0 155  0     0                 0 -        RL    -  0:00.00 [idle]
>  0  12    0   0 -76  0     0                 0 -        WL    -  0:00.00 [intr]
>  0  13    0   0  -8  0     0                 0 -        DL    -  0:00.00 [geom]
>  0  14    0   0 -16  0     0                 0 -        DL    -  0:00.00 [rand_harvestq]
>  0  15    0   0 -68  0     0                 0 -        DL    -  0:00.00 [usb]
>  0  16    0   0  16  0     0                 0 syncer   DL    -  0:00.00 [syncer]
>  0  17    0   0  -8  0     0                 0 m:w1     DL    -  0:00.00 [g_mirror gswap]
>  0  18    1   0  52  0 14168 18014398509481976 wait     Ds+   -  0:00.04 [sh]
>  0 144   18   0  52  0 14168 18014398509481976 wait     D+    -  0:00.00 [sh]
>  0 232  144   0  79  0 15928 18014398509481976 -        R+    -  0:00.00 [ifconfig]
>
>------------------------------------------------------------------------
>vmstat -s
>
>    22696 cpu context switches
>    54644 device interrupts
>      267 software interrupts
>    12089 traps
>   143999 system calls
>       17 kernel threads created
>      172  fork() calls
>       43 vfork() calls
>        0 rfork() calls
>        0 swap pager pageins
>        0 swap pager pages paged in
>        0 swap pager pageouts
>        0 swap pager pages paged out
>      138 vnode pager pageins
>      610 vnode pager pages paged in
>        0 vnode pager pageouts
>        0 vnode pager pages paged out
>        0 page daemon wakeups
>        0 pages examined by the page daemon
>        0 pages reactivated
>     5940 copy-on-write faults
>       28 copy-on-write optimized faults
>     3693 zero fill pages zeroed
>      395 zero fill pages prezeroed
>        0 intransit blocking page faults
>    11322 total VM faults taken
>       89 page faults requiring I/O
>        0 pages affected by kernel thread creation
>     3124 pages affected by  fork()
>      774 pages affected by vfork()
>        0 pages affected by rfork()
>        0 pages cached
>    14615 pages freed
>        0 pages freed by daemon
>        0 pages freed by exiting processes
>      294 pages active
>      594 pages inactive
>        0 pages in VM cache
>    10610 pages wired down
>  1013231 pages free
>     8192 bytes per page
>     4930 total name lookups
>          cache hits (84% pos + 7% neg) system 0% per-directory
>          deletions 0%, falsehits 0%, toolong 0%
>
>------------------------------------------------------------------------
>vmstat -m
>
>         Type InUse MemUse HighUse Requests  Size(s)
>      entropy  1026    65K       -     1027  32,64,8192
>CAM dev queue     5     1K       -        5  64
>    CAM queue    15     8K       -      312  16,32,64,128,256,512,1024
>     CAM path    15     1K       -      255  32
>       isadev     7     1K       -        7  128
>      CAM CCB     0     0K       -     2224  2048
>       USBdev     5     1K       -        5  32,64,128,256
>          USB     6     5K       -        6  16,128,512,4096
>         cdev     5     2K       -        5  256
>      scsi_cd     0     0K       -       10  16
>     filedesc    21    42K       -      233  2048
>         kenv    14     9K       -       15  16,32,64,128,8192
>    proc-args     3     1K       -       84  16,32,64,128
>        hhook     2     1K       -        2  256
>      ithread   104    19K       -      104  32,128,256
>      scsi_da     0     0K       -       90  16
>       kbdmux     5    22K       -        5  512,1024,2048,16384
>       KTRACE   100    13K       -      100  128
>       linker   128  1102K       -      162  16,32,64,512,2048,4096,8192
>        lockf     2     1K       -        2  64,128
>   loginclass     2     1K       -        2  64
>       ip6ndp     6     1K       -        6  64,128
>         temp    32   353K       -      306  16,32,64,128,256,1024,16384
>       devbuf  6487  2564K       -     6525  16,32,64,256,512,1024,2048,4096,8192,65536
>       module   293    37K       -      293  128
>     mtx_pool     2    16K       -        2  8192
>          osd     5     1K       -       11  16,32,64
>     pmchooks     1     1K       -        1  128
>       feeder     7     1K       -        7  32
>      subproc    61   104K       -      356  512,4096
>         proc     2    32K       -        2  16384
>      session     2     1K       -        2  128
>         pgrp     2     1K       -        2  128
>         cred    24     4K       -      380  64,256
>      uidinfo     2     5K       -        2  128,4096
>       plimit     2     1K       -        2  256
>      CAM DEV    10    20K       -      106  2048
>      CAM XPT    46     5K       -      221  32,64,128,256,512,1024,2048
>      CAM SIM     5     2K       -        5  256
>    sysctltmp     0     0K       -       11  16,32,64
>    sysctloid  2824   142K       -     2862  16,32,64,128
>       sysctl     0     0K       -       81  16,32,64
>      tidhash     1    32K       -        1  32768
>      callout     3  1672K       -        3  
>         umtx   434    55K       -      434  128
>     p1003.1b     1     1K       -        1  16
>         SWAP     2   617K       -        2  64
>       DEVFS1    71    36K       -       79  512
>       bus-sc    32   159K       -     1339  16,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
>          bus   874   109K       -     2309  16,32,64,128,256,1024
>      devstat     6    25K       -        6  32,8192
> eventhandler    65     5K       -       65  64,128
>       DEVFS3    87    22K       -       97  256
>         kobj   176   704K       -      228  4096
>      Per-cpu     1     1K       -        1  32
>        DEVFS     9     1K       -       10  16,32,128
>         rman  7500   937K       -     7508  16,32,128
> NFSD session     1     1K       -        1  1024
>         sbuf     0     0K       -      755  16,32,64,128,256
>    taskqueue    75    12K       -      125  16,64,128,256
>       Unitno    22     2K       -       70  32,64
>         vmem     2     1K       -        2  512
>          iov     0     0K       -       23  64,256,512
>          msg     4    30K       -        4  2048,4096,8192,16384
>          sem     4   106K       -        4  2048,4096
>          shm     1    32K       -        1  32768
>          tty     3     3K       -        3  1024
>        shmfd     1     8K       -        1  8192
>          pcb    14   663K       -       14  16,128,1024,2048,8192
>     vfscache     1  2048K       -        1  
>     vfs_hash     1  1024K       -        1  
>       vnodes     1     1K       -        1  256
>        mount    97     4K       -      240  16,32,64,128,256
>  vnodemarker     0     0K       -        5  512
>          BPF     5     1K       -        5  128
>  ether_multi    12     1K       -       12  16,32,64
>       ifaddr    85    18K       -       85  32,64,128,256,512,2048,4096
>        ifnet     6    11K       -        6  128,2048
>        clone     7     1K       -        7  128
>       arpcom     4     1K       -        4  16
>      lltable    10     5K       -       10  512
>     routetbl    16     5K       -       43  32,64,128,256,512
>         igmp     5     2K       -        5  256
>     in_multi     1     1K       -        1  256
>    sctp_iter     0     0K       -        2  256
>     sctp_ifn     1     1K       -        1  128
>     sctp_ifa     3     1K       -        3  128
>     sctp_vrf     1     1K       -        1  64
>    sctp_a_it     0     0K       -        2  16
>    hostcache     1    32K       -        1  32768
>     syncache     1    64K       -        1  65536
> NFSD lckfile     1     1K       -        1  256
>    in6_multi    15     2K       -       15  32,256
>          mld     5     1K       -        5  128
>          rpc     2    16K       -        2  8192
>audit_evclass   187     6K       -      229  32
>    ufs_quota     1  1024K       -        1  
>    vm_pgdata     2   513K       -        2  128
>NFSD V4client     1     1K       -        1  256
>   CAM periph    12     3K       -       84  16,32,64,128,256
>    pfs_nodes    21     6K       -       21  256
>     openfirm    68     4K       -      113  16,32,64,128,256,512
>         GEOM   141    27K       -      922  16,32,64,128,256,512,1024,2048,8192
>         UART     6     5K       -        6  16,1024
>      dvmamem  7455  1301K       -     7455  32,128,512,2048,8192,16384,32768
>   kstat_data     6     1K       -        6  64
>      solaris  7403 37023K       -    65718  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
>  mirror_data     3     1K       -        9  64,128,512
>  tmpfs mount     1     1K       -        1  128
>
>------------------------------------------------------------------------
>vmstat -z
>
>ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
>
>UMA Kegs:               384,      0,     112,      14,     112,   0,   0
>UMA Zones:              448,      0,     112,      14,     112,   0,   0
>UMA Slabs:               80,      0,    1004,       6,    1088,   0,   0
>UMA RCntSlabs:           96,      0,     190,      62,     190,   0,   0
>UMA Hash:               256,      0,       7,      24,       7,   0,   0
>4 Bucket:                32,      0,     196,     816,    3031,   0,   0
>6 Bucket:                48,      0,       0,       0,       0,   0,   0
>8 Bucket:                64,      0,      18,     612,     458,   0,   0
>12 Bucket:               96,      0,       6,     246,      46,   0,   0
>16 Bucket:              128,      0,      49,     329,     519,   0,   0
>32 Bucket:              256,      0,      17,     107,      65,   0,   0
>64 Bucket:              512,      0,      36,      54,     143,   0,   0
>128 Bucket:            1024,      0,      31,      25,     134,   0,   0
>256 Bucket:            2048,      0,      43,       5,     168,   0,   0
>vmem btag:               56,      0,     619,     389,     642,   7,   0
>VM OBJECT:              256,      0,     535,      85,    3038,   0,   0
>RADIX NODE:             144,      0,     855,     153,    5443,   0,   0
>MAP:                    240,      0,       3,      63,       3,   0,   0
>KMAP ENTRY:             128,      0,       9,     180,       9,   0,   0
>MAP ENTRY:              128,      0,      68,     310,    5829,   0,   0
>VMSPACE:                640,      0,       4,      44,     217,   0,   0
>fakepg:                 128,      0,       0,       0,       0,   0,   0
>mt_zone:               4112,      0,     292,       0,     292,   0,   0
>16:                      16,      0,    1806,     218,    9508,   0,   0
>32:                      32,      0,   15610,     329,   22528,   0,   0
>64:                      64,      0,    5156,     766,   20818,   0,   0
>128:                    128,      0,    9886,     257,   17490,   0,   0
>256:                    256,      0,     739,     315,    2547,   0,   0
>512:                    512,      0,     583,      62,   15972,   0,   0
>1024:                  1024,      0,    1619,      37,    8302,   0,   0
>2048:                  2048,      0,     120,      60,    2829,   0,   0
>4096:                  4096,      0,     316,     142,    2450,   0,   0
>8192:                  8192,      0,     124,       9,     567,   0,   0
>16384:                16384,      0,     179,      26,     523,   0,   0
>32768:                32768,      0,      39,       2,     147,   0,   0
>65536:                65536,      0,      11,       5,     120,   0,   0
>SLEEPQUEUE:              80,      0,     218,      34,     218,   0,   0
>64 pcpu:                  8,      0,    1654,     394,    1654,   0,   0
>Files:                   80,      0,       4,     299,    1216,   0,   0
>rl_entry:                40,      0,      10,     596,      10,   0,   0
>TURNSTILE:              136,      0,     218,      32,     218,   0,   0
>umtx pi:                 96,      0,       0,       0,       0,   0,   0
>MAC labels:              40,      0,       0,       0,       0,   0,   0
>PROC:                  1248,      0,      20,      16,     232,   0,   0
>THREAD:                1168,      0,     210,       7,     346,   0,   0
>cpuset:                  72,      0,     193,     143,     329,   0,   0
>audit_record:          1248,      0,       0,       0,       0,   0,   0
>mbuf_packet:            256, 1639590,     512,     258,     512,   0,   0
>mbuf:                   256, 1639590,       1,     263,       1,   0,   0
>mbuf_cluster:          2048, 256184,     759,       1,     759,   0,   0
>mbuf_jumbo_page:       8192,  64045,       0,       0,       0,   0,   0
>mbuf_jumbo_9k:         9216,  75904,       0,       0,       0,   0,   0
>mbuf_jumbo_16k:       16384,  42696,       0,       0,       0,   0,   0
>mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   0
>g_bio:                  248,      0,       0,     192,    5670,   0,   0
>ttyinq:                 160,      0,      15,      85,      15,   0,   0
>ttyoutq:                256,      0,       8,      54,       8,   0,   0
>ata_request:            336,      0,       0,      96,      98,   0,   0
>taskq_zone:              48,      0,       0,     338,       8,   0,   0
>VNODE:                  472,      0,     306,      51,     309,   0,   0
>VNODEPOLL:              112,      0,       0,       0,       0,   0,   0
>BUF TRIE:               144,      0,       0,   25648,       0,   0,   0
>S VFS Cache:            108,      0,     276,     156,     379,   0,   0
>STS VFS Cache:          148,      0,       0,       0,       0,   0,   0
>L VFS Cache:            328,      0,       0,       0,       0,   0,   0
>LTS VFS Cache:          368,      0,       0,       0,       0,   0,   0
>NAMEI:                 1024,      0,       0,      40,    2341,   0,   0
>DIRHASH:               1024,      0,       0,       0,       0,   0,   0
>NCLNODE:                528,      0,       0,       0,       0,   0,   0
>range_seg_cache:         64,      0,     258,     624,     503,   0,   0
>zio_cache:              920,      0,      63,     281,   11391,   0,   0
>zio_link_cache:          48,      0,      72,     773,    7232,   0,   0
>lz4_ctx:              16384,      0,       0,       2,     183,   0,   0
>sa_cache:                80,      0,     277,     228,     279,   0,   0
>dnode_t:                752,      0,     431,      19,     815,   0,   0
>dmu_buf_impl_t:         224,      0,     763,      65,    1823,   0,   0
>arc_buf_hdr_t:          216,      0,     457,      98,    1507,   0,   0
>arc_buf_t:               72,      0,     457,     215,    1511,   0,   0
>zil_lwb_cache:          192,      0,       2,      82,       3,   0,   0
>zfs_znode_cache:        368,      0,     277,      31,     279,   0,   0
>pipe:                   616,      0,       0,      26,      93,   0,   0
>procdesc:               128,      0,       0,       0,       0,   0,   0
>Mountpoints:            816,      0,       9,      18,       9,   0,   0
>ksiginfo:               112,      0,      26,    1054,      26,   0,   0
>itimer:                 352,      0,       0,       0,       0,   0,   0
>KNOTE:                  128,      0,       0,       0,       0,   0,   0
>socket:                 696, 130669,       1,      32,      14,   0,   0
>unpcb:                  240, 130680,       0,       0,       0,   0,   0
>ipq:                     56,   8064,       0,       0,       0,   0,   0
>udp_inpcb:              392, 130680,       1,      59,      14,   0,   0
>udpcb:                   24, 130801,       1,     505,      14,   0,   0
>tcp_inpcb:              392, 130680,       0,       0,       0,   0,   0
>tcpcb:                 1032, 130669,       0,       0,       0,   0,   0
>tcptw:                   88,  26220,       0,       0,       0,   0,   0
>syncache:               160,  15400,       0,       0,       0,   0,   0
>hostcache:              136,  15399,       0,       0,       0,   0,   0
>tcpreass:                40,  16160,       0,       0,       0,   0,   0
>sackhole:                32,      0,       0,       0,       0,   0,   0
>sctp_ep:               1424, 130670,       0,       0,       0,   0,   0
>sctp_asoc:             2416,  40002,       0,       0,       0,   0,   0
>sctp_laddr:              48,  80106,       0,     507,       2,   0,   0
>sctp_raddr:             728,  80003,       0,       0,       0,   0,   0
>sctp_chunk:             136, 400020,       0,       0,       0,   0,   0
>sctp_readq:             104, 400062,       0,       0,       0,   0,   0
>sctp_stream_msg_out:    104, 400062,       0,       0,       0,   0,   0
>sctp_asconf:             40, 400162,       0,       0,       0,   0,   0
>sctp_asconf_ack:         48, 400023,       0,       0,       0,   0,   0
>udplite_inpcb:          392, 130680,       0,       0,       0,   0,   0
>ripcb:                  392, 130680,       0,       0,       0,   0,   0
>rtentry:                200,      0,       7,     153,       7,   0,   0
>selfd:                   56,      0,       0,       0,       0,   0,   0
>SWAPMETA:               288, 512372,       0,       0,       0,   0,   0
>TMPFS dirent:            64,      0,       0,       0,       0,   0,   0
>TMPFS node:             232,      0,       1,      67,       1,   0,   0
>
>
>------------------------------------------------------------------------
>vmstat -i
>
>interrupt                          total       rate
>pil4: rndzvs                         166          0
>pil2: preempt                        984          3
>pil3: ithrd                         3249         11
>vec1992: bge0                          1          0
>vec1944: atapci0                    1207          4
>vec1833: sym0                       2012          7
>vec1832: sym1                         30          0
>pil11: filter                         42          0
>vec1964: uart0+                       42          0
>pil14: tick                        49876        180
>Total                              57609        207
>
>------------------------------------------------------------------------
>pstat -T
>
>  4/130664 files
>0M/8487M swap space
>
>------------------------------------------------------------------------
>pstat -s
>
>Device          512-blocks     Used    Avail Capacity
>/dev/mirror/gswap   17381808        0 17381808     0%
>
>------------------------------------------------------------------------
>iostat
>
>iostat: kvm_read(_tk_nin): invalid address (0)
>iostat: disabling TTY statistics
>             da0              da1              da2             cpu
>  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
> 26.81   0  0.01  30.83   0  0.01  16.38   3  0.05   1  0 17  1 80
>
>------------------------------------------------------------------------
>ipcs -a
>
>Message Queues:
>T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME   
>
>Shared Memory:
>T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME   
>
>Semaphores:
>T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME   
>
>
>------------------------------------------------------------------------
>ipcs -T
>
>msginfo:
>	msgmax:        16384	(max characters in a message)
>	msgmni:           40	(# of message queues)
>	msgmnb:         2048	(max characters in a message queue)
>	msgtql:           40	(max # of messages in system)
>	msgssz:            8	(size of a message segment)
>	msgseg:         2048	(# of message segments in system)
>
>shminfo:
>	shmmax:   1073741824	(max shared memory segment size)
>	shmmin:            1	(min shared memory segment size)
>	shmmni:          192	(max number of shared memory identifiers)
>	shmseg:          128	(max shared memory segments per process)
>	shmall:       131072	(max amount of shared memory in pages)
>
>seminfo:
>	semmni:           50	(# of semaphore identifiers)
>	semmns:          340	(# of semaphores in system)
>	semmnu:          150	(# of undo structures in system)
>	semmsl:          340	(max # of semaphores per id)
>	semopm:          100	(max # of operations per semop call)
>	semume:           50	(max # of undo entries per process)
>	semusz:          632	(size in bytes of undo structure)
>	semvmx:        32767	(semaphore maximum value)
>	semaem:        16384	(adjust on exit max value)
>
>
>------------------------------------------------------------------------
>nfsstat
>
>Client Info:
>Rpc Counts:
>  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
>        0         0         0         0         0         0         0         0
>   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
>        0         0         0         0         0         0         0         0
>    Mknod    Fsstat    Fsinfo  PathConf    Commit
>        0         0         0         0         0
>Rpc Info:
> TimedOut   Invalid X Replies   Retries  Requests
>        0         0         0         0         0
>Cache Info:
>Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW Hits    Misses
>        0         0         0         0         0         0         0         0
>BioRLHits    Misses BioD Hits    Misses DirE Hits    Misses Accs Hits    Misses
>        0         0         0         0         0         0         0         0
>
>Server Info:
>  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
>        0         0         0         0         0         0         0         0
>   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
>        0         0         0         0         0         0         0         0
>    Mknod    Fsstat    Fsinfo  PathConf    Commit
>        0         0         0         0         0
>Server Ret-Failed
>                0
>Server Faults
>            0
>Server Cache Stats:
>   Inprog      Idem  Non-idem    Misses
>        0         0         0         0
>Server Write Gathering:
> WriteOps  WriteRPC   Opsaved
>        0         0         0
>
>------------------------------------------------------------------------
>netstat -s
>
>tcp:
>	0 packets sent
>		0 data packets (0 bytes)
>		0 data packets (0 bytes) retransmitted
>		0 data packets unnecessarily retransmitted
>		0 resends initiated by MTU discovery
>		0 ack-only packets (0 delayed)
>		0 URG only packets
>		0 window probe packets
>		0 window update packets
>		0 control packets
>	0 packets received
>		0 acks (for 0 bytes)
>		0 duplicate acks
>		0 acks for unsent data
>		0 packets (0 bytes) received in-sequence
>		0 completely duplicate packets (0 bytes)
>		0 old duplicate packets
>		0 packets with some dup. data (0 bytes duped)
>		0 out-of-order packets (0 bytes)
>		0 packets (0 bytes) of data after window
>		0 window probes
>		0 window update packets
>		0 packets received after close
>		0 discarded for bad checksums
>		0 discarded for bad header offset fields
>		0 discarded because packet too short
>		0 discarded due to memory problems
>	0 connection requests
>	0 connection accepts
>	0 bad connection attempts
>	0 listen queue overflows
>	0 ignored RSTs in the windows
>	0 connections established (including accepts)
>	0 connections closed (including 0 drops)
>		0 connections updated cached RTT on close
>		0 connections updated cached RTT variance on close
>		0 connections updated cached ssthresh on close
>	0 embryonic connections dropped
>	0 segments updated rtt (of 0 attempts)
>	0 retransmit timeouts
>		0 connections dropped by rexmit timeout
>	0 persist timeouts
>		0 connections dropped by persist timeout
>	0 Connections (fin_wait_2) dropped because of timeout
>	0 keepalive timeouts
>		0 keepalive probes sent
>		0 connections dropped by keepalive
>	0 correct ACK header predictions
>	0 correct data packet header predictions
>	0 syncache entries added
>		0 retransmitted
>		0 dupsyn
>		0 dropped
>		0 completed
>		0 bucket overflow
>		0 cache overflow
>		0 reset
>		0 stale
>		0 aborted
>		0 badack
>		0 unreach
>		0 zone failures
>	0 cookies sent
>	0 cookies received
>	0 hostcache entries added
>		0 bucket overflow
>	0 SACK recovery episodes
>	0 segment rexmits in SACK recovery episodes
>	0 byte rexmits in SACK recovery episodes
>	0 SACK options (SACK blocks) received
>	0 SACK options (SACK blocks) sent
>	0 SACK scoreboard overflow
>	0 packets with ECN CE bit set
>	0 packets with ECN ECT(0) bit set
>	0 packets with ECN ECT(1) bit set
>	0 successful ECN handshakes
>	0 times ECN reduced the congestion window
>	0 packets with valid tcp-md5 signature received
>	0 packets with invalid tcp-md5 signature received
>	0 packets with tcp-md5 signature mismatch
>	0 packets with unexpected tcp-md5 signature received
>	0 packets without expected tcp-md5 signature received
>udp:
>	0 datagrams received
>	0 with incomplete header
>	0 with bad data length field
>	0 with bad checksum
>	0 with no checksum
>	0 dropped due to no socket
>	0 broadcast/multicast datagrams undelivered
>	0 dropped due to full socket buffers
>	0 not for hashed pcb
>	0 delivered
>	0 datagrams output
>	0 times multicast source filter matched
>ip:
>	0 total packets received
>	0 bad header checksums
>	0 with size smaller than minimum
>	0 with data size < data length
>	0 with ip length > max ip packet size
>	0 with header length < data size
>	0 with data length < header length
>	0 with bad options
>	0 with incorrect version number
>	0 fragments received
>	0 fragments dropped (dup or out of space)
>	0 fragments dropped after timeout
>	0 packets reassembled ok
>	0 packets for this host
>	0 packets for unknown/unsupported protocol
>	0 packets forwarded (0 packets fast forwarded)
>	0 packets not forwardable
>	0 packets received for unknown multicast group
>	0 redirects sent
>	0 packets sent from this host
>	0 packets sent with fabricated ip header
>	0 output packets dropped due to no bufs, etc.
>	0 output packets discarded due to no route
>	0 output datagrams fragmented
>	0 fragments created
>	0 datagrams that can't be fragmented
>	0 tunneling packets that can't find gif
>	0 datagrams with bad address in header
>icmp:
>	0 calls to icmp_error
>	0 errors not generated in response to an icmp message
>	0 messages with bad code fields
>	0 messages less than the minimum length
>	0 messages with bad checksum
>	0 messages with bad length
>	0 multicast echo requests ignored
>	0 multicast timestamp requests ignored
>	0 message responses generated
>	0 invalid return addresses
>	0 no return routes
>igmp:
>	0 messages received
>	0 messages received with too few bytes
>	0 messages received with wrong TTL
>	0 messages received with bad checksum
>	0 V1/V2 membership queries received
>	0 V3 membership queries received
>	0 membership queries received with invalid field(s)
>	0 general queries received
>	0 group queries received
>	0 group-source queries received
>	0 group-source queries dropped
>	0 membership reports received
>	0 membership reports received with invalid field(s)
>	0 membership reports received for groups to which we belong
>	0 V3 reports received without Router Alert
>	0 membership reports sent
>arp:
>	0 ARP requests sent
>	0 ARP replies sent
>	0 ARP requests received
>	0 ARP replies received
>	0 ARP packets received
>	0 total packets dropped due to no ARP entry
>	0 ARP entrys timed out
>	0 Duplicate IPs seen
>ip6:
>	0 total packets received
>	0 with size smaller than minimum
>	0 with data size < data length
>	0 with bad options
>	0 with incorrect version number
>	0 fragments received
>	0 fragments dropped (dup or out of space)
>	0 fragments dropped after timeout
>	0 fragments that exceeded limit
>	0 packets reassembled ok
>	0 packets for this host
>	0 packets forwarded
>	0 packets not forwardable
>	0 redirects sent
>	0 packets sent from this host
>	0 packets sent with fabricated ip header
>	0 output packets dropped due to no bufs, etc.
>	0 output packets discarded due to no route
>	0 output datagrams fragmented
>	0 fragments created
>	0 datagrams that can't be fragmented
>	0 packets that violated scope rules
>	0 multicast packets which we don't join
>	Mbuf statistics:
>		0 one mbuf
>		0 one ext mbuf
>		0 two or more ext mbuf
>	0 packets whose headers are not contiguous
>	0 tunneling packets that can't find gif
>	0 packets discarded because of too many headers
>	0 failures of source address selection
>	Source addresses selection rule applied:
>icmp6:
>	0 calls to icmp6_error
>	0 errors not generated in response to an icmp6 message
>	0 errors not generated because of rate limitation
>	0 messages with bad code fields
>	0 messages < minimum length
>	0 bad checksums
>	0 messages with bad length
>	Histogram of error messages to be generated:
>		0 no route
>		0 administratively prohibited
>		0 beyond scope
>		0 address unreachable
>		0 port unreachable
>		0 packet too big
>		0 time exceed transit
>		0 time exceed reassembly
>		0 erroneous header field
>		0 unrecognized next header
>		0 unrecognized option
>		0 redirect
>		0 unknown
>	0 message responses generated
>	0 messages with too many ND options
>	0 messages with bad ND options
>	0 bad neighbor solicitation messages
>	0 bad neighbor advertisement messages
>	0 bad router solicitation messages
>	0 bad router advertisement messages
>	0 bad redirect messages
>	0 path MTU changes
>rip6:
>	0 messages received
>	0 checksum calculations on inbound
>	0 messages with bad checksum
>	0 messages dropped due to no socket
>	0 multicast messages dropped due to no socket
>	0 messages dropped due to full socket buffers
>	0 delivered
>	0 datagrams output
>
>------------------------------------------------------------------------
>netstat -m
>
>netstat: invalid address (0)
>513/521/1034 mbufs in use (current/cache/total)
>501/259/760/256184 mbuf clusters in use (current/cache/total/max)
>512/258 mbuf+clusters out of packet secondary zone in use (current/cache)
>0/0/0/64045 8k (page size) jumbo clusters in use (current/cache/total/max)
>0/0/0/75904 9k jumbo clusters in use (current/cache/total/max)
>0/0/0/42696 16k jumbo clusters in use (current/cache/total/max)
>1130K/648K/1778K bytes allocated to network (current/cache/total)
>0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
>0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
>0/0/0 requests for jumbo clusters delayed (8k/9k/16k)
>0/0/0 requests for jumbo clusters denied (8k/9k/16k)
>
>------------------------------------------------------------------------
>netstat -anr
>
>Routing tables
>
>Internet:
>Destination        Gateway            Flags      Netif Expire
>127.0.0.1          link#5             UH          lo0
>
>Internet6:
>Destination                       Gateway                       Flags      Netif Expire
>::1                               link#5                        UH          lo0
>fe80::%lo0/64                     link#5                        U           lo0
>fe80::1%lo0                       link#5                        UHS         lo0
>ff01::%lo0/32                     ::1                           U           lo0
>ff02::%lo0/32                     ::1                           U           lo0
>
>------------------------------------------------------------------------
>netstat -anA
>
>Active Internet connections (including servers)
>Tcpcb            Proto Recv-Q Send-Q Local Address      Foreign Address    (state)
>fffff8003301ac40 udp4       0      0 *.*                *.*                
>
>------------------------------------------------------------------------
>netstat -aL
>
>Current listen queue sizes (qlen/incqlen/maxqlen)
>Proto Listen         Local Address         
>
>------------------------------------------------------------------------
>fstat
>
>fstat: can't read file 1 at 0x200007fffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read file 1 at 0x200007fffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0xffff000000000078
>fstat: can't read file 5 at 0x2
>fstat: can't read file 7 at 0x200007fffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 10 at 0xffff000000000078
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read file 1 at 0x200007fffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0xffff000000000078
>fstat: can't read file 5 at 0x2
>fstat: can't read file 7 at 0x200007fffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 10 at 0xffff000000000078
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>fstat: can't read znode_phys at 0x1
>USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
>root     ifconfig     232 root -         -       error    -
>root     ifconfig     232   wd -         -       error    -
>root     ifconfig     232 text -         -       error    -
>root     ifconfig     232 ctty /dev          3 crw-------  console rw
>root     ifconfig     232    0 /dev          3 crw-------  console rw
>root     sh           144 root -         -       error    -
>root     sh           144   wd -         -       error    -
>root     sh           144 text -         -       error    -
>root     sh           144 ctty /dev          3 crw-------  console rw
>root     sh           144    0 /dev          3 crw-------  console rw
>root     sh           144    6 /dev          3 crw-------  console rw
>root     sh            18 root -         -       error    -
>root     sh            18   wd -         -       error    -
>root     sh            18 text -         -       error    -
>root     sh            18 ctty /dev          3 crw-------  console rw
>root     sh            18    0 /dev          3 crw-------  console rw
>root     sh            18    6 /dev          3 crw-------  console rw
>root     zfskern        3 root -         -       error    -
>root     zfskern        3   wd -         -       error    -
>root     init           1 root -         -       error    -
>root     init           1   wd -         -       error    -
>root     init           1 text -         -       error    -
>root     kernel         0 root -         -       error    -
>root     kernel         0   wd -         -       error    -
>
>------------------------------------------------------------------------
>dmesg
>
>Copyright (c) 1992-2015 The FreeBSD Project.
>Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>	The Regents of the University of California. All rights reserved.
>FreeBSD is a registered trademark of The FreeBSD Foundation.
>FreeBSD 10.2-PRERELEASE #0 r284813: Thu Jun 25 21:49:53 UTC 2015
>    root@releng1.nyi.freebsd.org:/usr/obj/sparc64.sparc64/usr/src/sys/GENERIC sparc64
>gcc version 4.2.1 20070831 patched [FreeBSD]
>real memory  = 8589934592 (8192 MB)
>avail memory = 8379719680 (7991 MB)
>cpu0: Sun Microsystems UltraSparc-IIIi Processor (1503.00 MHz CPU)
>cpu1: Sun Microsystems UltraSparc-IIIi Processor (1503.00 MHz CPU)
>FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
>random: <Software, Yarrow> initialized
>kbd0 at kbdmux0
>nexus0: <Open Firmware Nexus device>
>pcib0: <Sun Host-PCI bridge> mem 0x4000ff00000-0x4000ff0afff,0x4000fc10000-0x4000fc1701f,0x7f600000000-0x7f6000000ff,0x4000ff80000-0x4000ff8ffff irq 2035,2032,2033,2036,2019 on nexus0
>pcib0: Tomatillo, version 4, IGN 0x1f, bus B, PCI mode, 66MHz
>pcib0: DVMA map: 0xc0000000 to 0xdfffffff 65536 entries
>pci0: <OFW PCI bus> on pcib0
>pcib1: <Sun Host-PCI bridge> mem 0x4000f600000-0x4000f60afff,0x4000f410000-0x4000f41701f,0x7fe00000000-0x7fe000000ff,0x4000f780000-0x4000f78ffff irq 1970,1968,1969,1972,1953 on nexus0
>pcib1: Tomatillo, version 4, IGN 0x1e, bus A, PCI mode, 33MHz
>pcib1: DVMA map: 0xc0000000 to 0xdfffffff 65536 entries
>pci1: <OFW PCI bus> on pcib1
>pcib2: <Sun Host-PCI bridge> mem 0x4000e600000-0x4000e60afff,0x4000e410000-0x4000e41701f,0x7ce00000000-0x7ce000000ff,0x4000e780000-0x4000e78ffff irq 1842,1840,1841,1844,1827 on nexus0
>pcib2: Tomatillo, version 4, IGN 0x1c, bus A, PCI mode, 66MHz
>pcib2: DVMA map: 0xc0000000 to 0xdfffffff 65536 entries
>pci2: <OFW PCI bus> on pcib2
>pcib3: <Sun Host-PCI bridge> mem 0x4000ef00000-0x4000ef0afff,0x4000ec10000-0x4000ec1701f,0x7c600000000-0x7c6000000ff,0x4000ef80000-0x4000ef8ffff irq 1907,1904,1905,1908,1893 on nexus0
>pcib3: Tomatillo, version 4, IGN 0x1d, bus B, PCI mode, 66MHz
>pcib3: DVMA map: 0xc0000000 to 0xdfffffff 65536 entries
>pci3: <OFW PCI bus> on pcib3
>nexus0: <memory-controller> mem 0x40000000000-0x40000000007 type memory-controller (no driver attached)
>nexus0: <memory-controller> mem 0x40000800000-0x40000800007 type memory-controller (no driver attached)
>bge0: <Broadcom BCM5704 A3, ASIC rev. 0x002003> mem 0x200000-0x20ffff,0x110000-0x11ffff at device 2.0 on pci0
>bge0: CHIP ID 0x00002003; ASIC REV 0x02; CHIP REV 0x20; PCI on PCI-X 66 MHz; 64bit
>miibus0: <MII bus> on bge0
>brgphy0: <BCM5704 1000BASE-T media interface> PHY 1 on miibus0
>brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
>bge0: Ethernet address: 00:03:ba:e0:ce:07
>bge1: <Broadcom BCM5704 A3, ASIC rev. 0x002003> mem 0x400000-0x40ffff,0x120000-0x12ffff at device 2.1 on pci0
>bge1: CHIP ID 0x00002003; ASIC REV 0x02; CHIP REV 0x20; PCI on PCI-X 66 MHz; 64bit
>miibus1: <MII bus> on bge1
>brgphy1: <BCM5704 1000BASE-T media interface> PHY 1 on miibus1
>brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
>bge1: Ethernet address: 00:03:ba:e0:ce:08
>isab0: <PCI-ISA bridge> at device 7.0 on pci1
>isa0: <ISA bus> on isab0
>pci1: <old, non-VGA display device> at device 6.0 (no driver attached)
>ohci0: <AcerLabs M5237 (Aladdin-V) USB controller> mem 0x1000000-0x1000fff at device 10.0 on pci1
>usbus0 on ohci0
>atapci0: <AcerLabs M5229 UDMA100 controller> port 0x900-0x907,0x918-0x91b,0x910-0x917,0x908-0x90b,0x920-0x92f at device 13.0 on pci1
>atapci0: using PIO transfers above 137GB as workaround for 48bit DMA access bug, expect reduced performance
>ata2: <ATA channel> at channel 0 on atapci0
>ata3: <ATA channel> at channel 1 on atapci0
>sym0: <1010-66> port 0x900-0x9ff mem 0x100000-0x1003ff,0x102000-0x103fff at device 2.0 on pci2
>sym0: No NVRAM, ID 7, Fast-80, LVD, parity checking
>sym1: <1010-66> port 0xa00-0xaff mem 0x104000-0x1043ff,0x106000-0x107fff at device 2.1 on pci2
>sym1: No NVRAM, ID 7, Fast-80, LVD, parity checking
>bge2: <Broadcom BCM5704 A3, ASIC rev. 0x002003> mem 0x200000-0x20ffff,0x110000-0x11ffff at device 2.0 on pci3
>bge2: CHIP ID 0x00002003; ASIC REV 0x02; CHIP REV 0x20; PCI on PCI-X 66 MHz; 64bit
>miibus2: <MII bus> on bge2
>brgphy2: <BCM5704 1000BASE-T media interface> PHY 1 on miibus2
>brgphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
>bge2: Ethernet address: 00:03:ba:e0:ce:09
>bge3: <Broadcom BCM5704 A3, ASIC rev. 0x002003> mem 0x400000-0x40ffff,0x120000-0x12ffff at device 2.1 on pci3
>bge3: CHIP ID 0x00002003; ASIC REV 0x02; CHIP REV 0x20; PCI on PCI-X 66 MHz; 64bit
>miibus3: <MII bus> on bge3
>brgphy3: <BCM5704 1000BASE-T media interface> PHY 1 on miibus3
>brgphy3:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
>bge3: Ethernet address: 00:03:ba:e0:ce:0a
>nexus0: <syscons> type unknown (no driver attached)
>rtc0: <Real-Time Clock> at port 0x70-0x71 on isa0
>uart0: <16550 or compatible> at port 0x3f8-0x3ff irq 44 on isa0
>uart0: console (9600,n,8,1)
>uart1: <16550 or compatible> at port 0x2e8-0x2ef irq 44 on isa0
>usbus0: 12Mbps Full Speed USB v1.0
>ZFS filesystem version: 5
>ZFS storage pool version: features support (5000)
>Timecounter "tick" frequency 1503000000 Hz quality 10
>Timecounter "stick" frequency 12000000 Hz quality 10
>Event timer "tick" frequency 1503000000 Hz quality 1000
>Timecounters tick every 1.000 msec
>ugen0.1: <AcerLabs> at usbus0
>uhub0: <AcerLabs OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
>uhub0: 2 ports with 2 removable, self powered
>random: unblocking device.
>da0 at sym0 bus 0 scbus2 target 0 lun 0
>da0: <HITACHI HUS10733ASUN72G PA05> Fixed Direct Access SCSI-3 device
>SMP: AP CPU #1 Launched!
>da0: Serial Number 10RW6DG9          50V3W6DG9A
>da0: 160.000MB/s transfers (80.000MHz DT, offset 62, 16bit)
>da0: Command Queueing enabled
>da0: 70007MB (143374738 512 byte sectors: 255H 63S/T 8924C)
>da3 at sym0 bus 0 scbus2 target 3 lun 0
>da3: <HITACHI HUS10733ASUN72G PA05> Fixed Direct Access SCSI-3 device
>da3: Serial Number 10RW6NXT          50V3W6NXTA
>da3: 160.000MB/s transfers (80.000MHz DT, offset 62, 16bit)
>da3: Command Queueing enabled
>da3: 70007MB (143374738 512 byte sectors: 255H 63S/T 8924C)
>da2 at sym0 bus 0 scbus2 target 2 lun 0
>da2: <SEAGATE ST373405LC 2203> Fixed Direct Access SCSI-3 device
>da2: Serial Number 3EK1AXHL00007237WL46
>da2: 160.000MB/s transfers (80.000MHz DT, offset 62, 16bit)
>da2: Command Queueing enabled
>da2: 70007MB (143374650 512 byte sectors: 255H 63S/T 8924C)
>da1 at sym0 bus 0 scbus2 target 1 lun 0
>da1: <IBM-ESXS ST373307LC    FN B25H> Fixed Direct Access SCSI-3 device
>da1: Serial Number 3HZ9AC0B00007512MDRA
>da1: 160.000MB/s transfers (80.000MHz DT, offset 62, 16bit)
>da1: Command Queueing enabled
>da1: 70006MB (143374000 512 byte sectors: 255H 63S/T 8924C)
>cd0 at ata2 bus 0 scbus0 target 0 lun 0
>cd0: <TEAC DV-28E-C 1.4B> Removable CD-ROM SCSI device
>cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
>cd0: cd present [256000 x 2048 byte records]
>GEOM_MIRROR: Device mirror/gswap launched (2/2).
>GEOM_MIRROR: Device gswap already configured.
>GEOM_MIRROR: Device gswap already configured.
>GEOM_MIRROR: Device gswap already configured.
>GEOM_MIRROR: Device gswap already configured.
>Trying to mount root from zfs:sys/ROOT/default []...
>Setting hostuuid: 633f56fe-1f8f-11e5-8d92-0003bae0ce07.
>Setting hostid: 0xbab5179d.
>Entropy harvesting: interrupts ethernet point_to_point swi.
>Starting file system checks:
>Mounting local file systems:.
>Writing entropy file:.
>Setting hostname: spork.pix.net.
>spin lock 0xc0cb9e38 (smp rendezvous) held by 0xfffff80003e93240 (tid 100340) too long
>timeout stopping cpus
>panic: spin lock held too long
>cpuid = 1
>KDB: stack backtrace:
>#0 0xc0575380 at panic+0x20
>#1 0xc0558e10 at _mtx_lock_spin_failed+0x50
>#2 0xc0558ed8 at _mtx_lock_spin_cookie+0xb8
>#3 0xc08d7b9c at tick_get_timecount_mp+0xdc
>#4 0xc0583c88 at binuptime+0x48
>#5 0xc08a3b8c at timercb+0x6c
>#6 0xc08d7f00 at tick_intr+0x220
>Uptime: 29s
>Dumping 8192 MB (4 chunks)
>  chunk at 0: 2147483648 bytes |\^H/\^H-\^H\134\^H|\^H
>
>------------------------------------------------------------------------
>kernel config
>
>Assertion failed: (i == size - 1 && ("\\0 found in the middle of a file")), function kernconfdump, file /usr/src/usr.sbin/config/main.c, line 726.
>options	CONFIG_AUTOGENERATED
>ident	GENERIC
>machine	sparc64
>cpu	SUN4U
>makeoptions	DEBUG=-g
>options	USB_DEBUG
>options	AH_SUPPORT_AR5416
>options	IEEE80211_SUPPORT_MESH
>options	IEEE80211_AMPDU_AGE
>options	IEEE80211_DEBUG
>options	KBD_INSTALL_CDEV
>options	AHC_REG_PRETTY_PRINT
>options	AHC_ALLOW_MEMIO
>options	SMP
>options	KDB_TRACE
>options	KDB
>options	RCTL
>options	RACCT_DEFAULT_TO_DISABLED
>options	RACCT
>options	INCLUDE_CONFIG_FILE
>options	MAC
>options	PROCDESC
>options	CAPABILITIES
>options	CAPABILITY_MODE
>options	AUDIT
>options	HWPMC_HOOKS
>options	PRINTF_BUFR_SIZE=128
>options	_KPOSIX_PRIORITY_SCHEDULING
>options	SYSVSEM
>options	SYSVMSG
>options	SYSVSHM
>options	STACK
>options	KTRACE
>options	SCSI_DELAY=5000
>options	COMPAT_FREEBSD7
>options	COMPAT_FREEBSD6
>options	COMPAT_FREEBSD5
>options	GEOM_LABEL
>options	GEOM_PART_GPT
>options	PSEUDOFS
>options	PROCFS
>options	CD9660
>options	NFS_ROOT
>options	NFSLOCKD
>options	NFSD
>options	NFSCL
>options	MD_ROOT
>options	QUOTA
>options	UFS_GJOURNAL
>options	UFS_DIRHASH
>options	UFS_ACL
>options	SOFTUPDATES
>options	FFS
>options	SCTP
>options	INET6
>options	INET
>options	PREEMPTION
>options	SCHED_ULE
>options	NEW_PCIB
>options	SUNKBD_EMULATE_ATKBD
>options	GEOM_PART_VTOC8
>options	GEOM_PART_BSD
>device	mem
>device	uart_ns8250
>device	uart_sab82532
>device	uart_z8530
>device	ebus
>device	isa
>device	pci
>device	sbus
>device	central
>device	fhc
>device	ahci
>device	ata
>device	mvs
>device	siis
>device	ahc
>device	esp
>device	isp
>device	ispfw
>device	mpt
>device	mps
>device	mpr
>device	sym
>device	scbus
>device	ch
>device	da
>device	sa
>device	cd
>device	pass
>device	ses
>device	atkbdc
>device	atkbd
>device	psm
>device	kbdmux
>device	sc
>device	creator
>device	machfb
>device	splash
>device	auxio
>device	eeprom
>device	mk48txx
>device	rtc
>device	mc146818
>device	epic
>device	sbbc
>device	puc
>device	scc
>device	uart
>device	em
>device	le
>device	ti
>device	txp
>device	miibus
>device	bge
>device	cas
>device	dc
>device	fxp
>device	gem
>device	hme
>device	nge
>device	re
>device	rl
>device	sf
>device	sis
>device	sk
>device	ste
>device	stge
>device	vr
>device	xl
>device	wlan
>device	wlan_wep
>device	wlan_ccmp
>device	wlan_tkip
>device	wlan_amrr
>device	ath
>device	ath_pci
>device	ath_hal
>device	ath_rate_sample
>device	loop
>device	random
>device	ether
>device	vlan
>device	tun
>device	md
>device	gif
>device	faith
>device	firmware
>device	bpf
>device	uhci
>device	ohci
>device	ehci
>device	usb
>device	ukbd
>device	umass
>device	sound
>device	snd_audiocs
>device	snd_es137x
>device	snd_t4dwave
>Abort trap (core dumped)
>
>------------------------------------------------------------------------
>ddb capture buffer
>
>ddb: ddb_capture: kvm_nlist

Actions: View

Attachments on bug 201245: 158219