munnar dumped core - see /var/crash/vmcore.9 Tue Feb 20 09:54:14 IST 2018 FreeBSD munnar 12.0-CURRENT FreeBSD 12.0-CURRENT #8: Fri Feb 16 14:58:28 IST 2018 root@munnar:/usr/obj/usr/src/amd64.amd64/sys/MYKERNEL amd64 panic: Memory modified after free 0xfffff80003ba8b80(32) val=0 @ 0xfffff80003ba8b80 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: Memory modified after free 0xfffff80003ba8b80(32) val=0 @ 0xfffff80003ba8b80 cpuid = 5 time = 1519077207 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2c/frame 0xfffffe006210ee80 kdb_backtrace() at kdb_backtrace+0x53/frame 0xfffffe006210ef50 vpanic() at vpanic+0x268/frame 0xfffffe006210f020 kproc_shutdown() at kproc_shutdown/frame 0xfffffe006210f080 trash_ctor() at trash_ctor+0x6b/frame 0xfffffe006210f0c0 uma_zalloc_arg() at uma_zalloc_arg+0x284/frame 0xfffffe006210f200 bucket_alloc() at bucket_alloc+0x102/frame 0xfffffe006210f240 uma_zfree_arg() at uma_zfree_arg+0x647/frame 0xfffffe006210f330 uma_zfree() at uma_zfree+0x29/frame 0xfffffe006210f360 ctl_free_beio() at ctl_free_beio+0x80/frame 0xfffffe006210f390 ctl_complete_beio() at ctl_complete_beio+0x48/frame 0xfffffe006210f3b0 ctl_be_block_move_done() at ctl_be_block_move_done+0x266/frame 0xfffffe006210f430 cfiscsi_datamove_in() at cfiscsi_datamove_in+0x92d/frame 0xfffffe006210f510 cfiscsi_datamove() at cfiscsi_datamove+0x28/frame 0xfffffe006210f530 ctl_datamove() at ctl_datamove+0x498/frame 0xfffffe006210f7d0 ctl_be_block_dispatch_zvol() at ctl_be_block_dispatch_zvol+0x6c2/frame 0xfffffe006210f8e0 ctl_be_block_dispatch() at ctl_be_block_dispatch+0x626/frame 0xfffffe006210f9b0 ctl_be_block_worker() at ctl_be_block_worker+0x68d/frame 0xfffffe006210fa40 taskqueue_run_locked() at taskqueue_run_locked+0x246/frame 0xfffffe006210fab0 taskqueue_thread_loop() at taskqueue_thread_loop+0xc2/frame 0xfffffe006210fb30 fork_exit() at fork_exit+0x145/frame 0xfffffe006210fbb0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe006210fbb0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done. done. Loaded symbols for /boot/kernel/fdescfs.ko Reading symbols from /boot/kernel/uhid.ko...Reading symbols from /usr/lib/debug//boot/kernel/uhid.ko.debug...done. done. Loaded symbols for /boot/kernel/uhid.ko Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/ctl.ko...Reading symbols from /usr/lib/debug//boot/kernel/ctl.ko.debug...done. done. Loaded symbols for /boot/kernel/ctl.ko Reading symbols from /boot/kernel/cfiscsi.ko...Reading symbols from /usr/lib/debug//boot/kernel/cfiscsi.ko.debug...done. done. Loaded symbols for /boot/kernel/cfiscsi.ko Reading symbols from /boot/kernel/iscsi.ko...Reading symbols from /usr/lib/debug//boot/kernel/iscsi.ko.debug...done. done. Loaded symbols for /boot/kernel/iscsi.ko Reading symbols from /boot/kernel/t4_tom.ko...Reading symbols from /usr/lib/debug//boot/kernel/t4_tom.ko.debug...done. done. Loaded symbols for /boot/kernel/t4_tom.ko Reading symbols from /boot/kernel/toecore.ko...Reading symbols from /usr/lib/debug//boot/kernel/toecore.ko.debug...done. done. Loaded symbols for /boot/kernel/toecore.ko Reading symbols from /boot/kernel/if_cxgbe.ko...Reading symbols from /usr/lib/debug//boot/kernel/if_cxgbe.ko.debug...done. done. Loaded symbols for /boot/kernel/if_cxgbe.ko Reading symbols from /boot/kernel/dtraceall.ko...Reading symbols from /usr/lib/debug//boot/kernel/dtraceall.ko.debug...done. done. Loaded symbols for /boot/kernel/dtraceall.ko Reading symbols from /boot/kernel/dtrace.ko...Reading symbols from /usr/lib/debug//boot/kernel/dtrace.ko.debug...done. done. Loaded symbols for /boot/kernel/dtrace.ko Reading symbols from /boot/kernel/dtmalloc.ko...Reading symbols from /usr/lib/debug//boot/kernel/dtmalloc.ko.debug...done. done. Loaded symbols for /boot/kernel/dtmalloc.ko Reading symbols from /boot/kernel/dtnfscl.ko...Reading symbols from /usr/lib/debug//boot/kernel/dtnfscl.ko.debug...done. done. Loaded symbols for /boot/kernel/dtnfscl.ko Reading symbols from /boot/kernel/fbt.ko...Reading symbols from /usr/lib/debug//boot/kernel/fbt.ko.debug...done. done. Loaded symbols for /boot/kernel/fbt.ko Reading symbols from /boot/kernel/fasttrap.ko...Reading symbols from /usr/lib/debug//boot/kernel/fasttrap.ko.debug...done. done. Loaded symbols for /boot/kernel/fasttrap.ko Reading symbols from /boot/kernel/sdt.ko...Reading symbols from /usr/lib/debug//boot/kernel/sdt.ko.debug...done. done. Loaded symbols for /boot/kernel/sdt.ko Reading symbols from /boot/kernel/systrace.ko...Reading symbols from /usr/lib/debug//boot/kernel/systrace.ko.debug...done. done. Loaded symbols for /boot/kernel/systrace.ko Reading symbols from /boot/kernel/systrace_freebsd32.ko...Reading symbols from /usr/lib/debug//boot/kernel/systrace_freebsd32.ko.debug...done. done. Loaded symbols for /boot/kernel/systrace_freebsd32.ko Reading symbols from /boot/kernel/profile.ko...Reading symbols from /usr/lib/debug//boot/kernel/profile.ko.debug...done. done. Loaded symbols for /boot/kernel/profile.ko #0 doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:346 346 dumptid = curthread->td_tid; (kgdb) #0 doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:346 #1 0xffffffff8048cd72 in db_dump (dummy=-2129994987, dummy2=false, dummy3=-1, dummy4=0xfffffe006210e930 "") at /usr/src/sys/ddb/db_command.c:574 #2 0xffffffff8048cb5f in db_command (last_cmdp=0xffffffff82144080, cmd_table=0x0, dopager=1) at /usr/src/sys/ddb/db_command.c:481 #3 0xffffffff8048c669 in db_command_loop () at /usr/src/sys/ddb/db_command.c:534 #4 0xffffffff804915aa in db_trap (type=3, code=-61456) at /usr/src/sys/ddb/db_main.c:250 #5 0xffffffff810ae3e1 in kdb_trap (type=3, code=-61456, tf=0xfffffe006210ee60) at /usr/src/sys/kern/subr_kdb.c:697 #6 0xffffffff81728494 in trap (frame=0xfffffe006210ee60) at /usr/src/sys/amd64/amd64/trap.c:547 #7 0xffffffff817296ca in trap_check (frame=0xfffffe006210ee60) at /usr/src/sys/amd64/amd64/trap.c:608 #8 0xffffffff816f9c7c in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232 #9 0xffffffff810adb15 in breakpoint () at cpufunc.h:65 #10 0xffffffff810ad71a in kdb_enter (why=0xffffffff81a9fc95 "panic", msg=0xffffffff81a9fc95 "panic") at /usr/src/sys/kern/subr_kdb.c:478 #11 0xffffffff81041aba in vpanic ( fmt=0xffffffff81acda54 "Memory modified after free %p(%d) val=%x @ %p\n", ap=0xfffffe006210f060) at /usr/src/sys/kern/kern_shutdown.c:800 #12 0xffffffff81041ba0 in panic ( fmt=0xffffffff81acda54 "Memory modified after free %p(%d) val=%x @ %p\n") at /usr/src/sys/kern/kern_shutdown.c:738 #13 0xffffffff8156515b in trash_ctor (mem=0xfffff80003ba8b80, size=32, arg=0x108, flags=1) at /usr/src/sys/vm/uma_dbg.c:82 #14 0xffffffff8155c914 in uma_zalloc_arg (zone=0xfffff8022ffe4740, udata=0x108, flags=1) at /usr/src/sys/vm/uma_core.c:2240 #15 0xffffffff8155e192 in bucket_alloc (zone=0xfffff800c2139ae0, udata=0x108, flags=1) at /usr/src/sys/vm/uma_core.c:409 #16 0xffffffff8155dd87 in uma_zfree_arg (zone=0xfffff800c2139ae0, item=0xfffffe00ab636000, udata=0x0) at /usr/src/sys/vm/uma_core.c:2966 #17 0xffffffff82f8e549 in uma_zfree (zone=0xfffff800c2139ae0, item=0xfffffe00ab636000) at uma.h:395 #18 0xffffffff82f8e420 in ctl_free_beio (beio=0xfffff8018ae37cf0) at /usr/src/sys/cam/ctl/ctl_backend_block.c:322 #19 0xffffffff82f8e238 in ctl_complete_beio (beio=0xfffff8018ae37cf0) at /usr/src/sys/cam/ctl/ctl_backend_block.c:349 #20 0xffffffff82f8d616 in ctl_be_block_move_done (io=0xfffff80187c512a0) at /usr/src/sys/cam/ctl/ctl_backend_block.c:452 #21 0xffffffff82fc846d in cfiscsi_datamove_in (io=0xfffff80187c512a0) at /usr/src/sys/cam/ctl/ctl_frontend_iscsi.c:2614 #22 0xffffffff82fc7818 in cfiscsi_datamove (io=0xfffff80187c512a0) at /usr/src/sys/cam/ctl/ctl_frontend_iscsi.c:2775 #23 0xffffffff82f6bad8 in ctl_datamove (io=0xfffff80187c512a0) at /usr/src/sys/cam/ctl/ctl.c:12503 #24 0xffffffff82f90e12 in ctl_be_block_dispatch_zvol ( be_lun=0xfffff80005c9c400, beio=0xfffff8018ae37cf0) at /usr/src/sys/cam/ctl/ctl_backend_block.c:954 #25 0xffffffff82f95f56 in ctl_be_block_dispatch (be_lun=0xfffff80005c9c400, io=0xfffff80187c512a0) at /usr/src/sys/cam/ctl/ctl_backend_block.c:1645 #26 0xffffffff82f9562d in ctl_be_block_worker (context=0xfffff80005c9c400, pending=1) at /usr/src/sys/cam/ctl/ctl_backend_block.c:1725 #27 0xffffffff810ccd36 in taskqueue_run_locked (queue=0xfffff80018148a00) at /usr/src/sys/kern/subr_taskqueue.c:465 #28 0xffffffff810ce3e2 in taskqueue_thread_loop (arg=0xfffff80005c9c5d0) at /usr/src/sys/kern/subr_taskqueue.c:757 #29 0xffffffff80fd94c5 in fork_exit ( callout=0xffffffff810ce320 , arg=0xfffff80005c9c5d0, frame=0xfffffe006210fbc0) at /usr/src/sys/kern/kern_fork.c:1039 #30 0xffffffff816fa99e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:843 #31 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) ------------------------------------------------------------------------ ps -axlww UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 0 0 0 -16 0 0 0 swapin DLs - 0:11.06 [kernel] 0 1 0 0 20 0 10032 700 wait DLs - 0:00.14 [init] 0 2 0 0 -16 0 0 0 crypto_w DL - 0:00.00 [crypto] 0 3 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 0] 0 4 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 1] 0 5 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 2] 0 6 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 3] 0 7 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 4] 0 8 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 5] 0 9 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 6] 0 10 0 0 -16 0 0 0 audit_wo DL - 0:00.00 [audit] 0 11 0 0 155 0 0 0 - RL - 496:10.34 [idle] 0 12 0 0 -52 0 0 0 - WL - 0:03.95 [intr] 0 13 0 0 -8 0 0 0 - DL - 0:00.01 [geom] 0 14 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 7] 0 15 0 0 -16 0 0 0 - DL - 0:00.14 [cam] 0 16 0 0 -68 0 0 0 - DL - 0:01.11 [usb] 0 17 0 0 -16 0 0 0 - DL - 0:00.00 [fdc0] 0 18 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod1] 0 19 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod2] 0 20 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod3] 0 21 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod4] 0 22 0 0 -16 0 0 0 waiting_ DL - 0:00.00 [sctp_iterator] 0 23 0 0 -16 0 0 0 - DL - 0:00.24 [rand_harvestq] 0 24 0 0 -16 0 0 0 psleep DL - 0:00.10 [pagedaemon] 0 25 0 0 -16 0 0 0 psleep DL - 0:00.00 [vmdaemon] 0 26 0 0 -16 0 0 0 - DL - 0:00.02 [bufspacedaemon] 0 27 0 0 -16 0 0 0 psleep DL - 0:00.09 [bufdaemon] 0 28 0 0 -16 0 0 0 vlruwt DL - 0:00.02 [vnlru] 0 29 0 0 16 0 0 0 syncer DL - 0:00.18 [syncer] 0 125 1 0 52 0 10880 0 pause DWs - 0:00.00 [adjkerntz] 0 425 1 0 20 0 13920 812 select Ds - 0:00.06 [devd] 0 571 1 0 20 0 11476 2320 select Ds - 0:00.05 [syslogd] 0 696 1 0 20 0 18200 5804 select Ds - 0:00.01 [sshd] 0 699 1 0 20 0 15912 5048 select Ds - 0:00.05 [sendmail] 25 702 1 0 20 0 15760 0 pause DWs - 0:00.00 [sendmail] 0 706 1 0 20 0 11468 844 nanslp Ds - 0:00.03 [cron] 0 753 1 0 52 0 10928 1936 ttyin Ds+ - 0:00.01 [getty] 0 754 1 0 52 0 10928 1936 ttyin Ds+ - 0:00.01 [getty] 0 755 1 0 52 0 10928 1936 ttyin Ds+ - 0:00.01 [getty] 0 756 1 0 52 0 10928 1936 ttyin Ds+ - 0:00.01 [getty] 0 757 1 0 52 0 10928 1936 ttyin Ds+ - 0:00.01 [getty] 0 758 1 0 52 0 10928 1936 ttyin Ds+ - 0:00.01 [getty] 0 759 1 0 52 0 10928 1936 ttyin Ds+ - 0:00.01 [getty] 0 760 1 0 52 0 10928 1936 ttyin Ds+ - 0:00.01 [getty] 0 761 1 0 52 0 10916 1936 ttyin Ds+ - 0:00.01 [getty] 0 859 0 0 -8 0 0 0 arc_recl DL - 0:00.04 [zfskern] 0 872 0 0 -16 0 0 0 - RL - 0:00.00 [ctl] 0 1137 0 0 -16 0 0 0 ftcl DL - 0:00.00 [ftcleanup] 0 1253 1 0 52 0 13200 2532 select Ds - 0:00.00 [ctld] 0 1979 706 0 20 0 11468 1680 piperd D - 0:00.00 [cron] 0 1981 1979 0 22 0 11756 0 wait DWs - 0:00.00 [sh] 0 1983 1981 0 22 0 10732 0 wait DW - 0:00.00 [lockf] 0 1984 1983 0 52 0 11756 0 wait DW - 0:00.00 [sh] 0 1993 1984 0 52 0 11804 0 wait DW - 0:00.00 [sh] 0 1994 1984 0 22 0 10936 2344 piperd D - 0:00.00 [mail] 0 2080 1993 0 52 0 11764 0 wait DW - 0:00.00 [sh] 0 2081 2080 0 52 0 11756 0 wait DW - 0:00.00 [sh] 0 2083 2081 0 52 0 10732 0 wait DW - 0:00.00 [lockf] 0 2084 2083 0 52 0 11756 0 wait DW - 0:00.00 [sh] 0 2092 2084 0 52 0 11804 0 wait DW - 0:00.00 [sh] 0 2093 2084 0 52 0 10936 2344 piperd D - 0:00.00 [mail] 0 2094 2092 0 52 0 11824 0 wait DW - 0:00.00 [sh] 0 2098 2094 0 22 0 14908 6336 - R - 0:00.00 [find] 0 2099 2094 0 52 0 11824 0 wait DW - 0:00.00 [sh] 0 2101 2099 0 52 0 11768 3196 piperd D - 0:00.00 [cat] ------------------------------------------------------------------------ vmstat -s 432795458 cpu context switches 53755621 device interrupts 1098616 software interrupts 650512 traps 45880077 system calls 33 kernel threads created 1224 fork() calls 876 vfork() calls 0 rfork() calls 482 swap pager pageins 1496 swap pager pages paged in 1652 swap pager pageouts 6894 swap pager pages paged out 1160 vnode pager pageins 8415 vnode pager pages paged in 0 vnode pager pageouts 0 vnode pager pages paged out 0 page daemon wakeups 0 pages examined by the page daemon 0 clean page reclamation shortfalls 0 pages reactivated by the page daemon 82890 copy-on-write faults 81 copy-on-write optimized faults 197614 zero fill pages zeroed 24 zero fill pages prezeroed 577 intransit blocking page faults 299311 total VM faults taken 1486 page faults requiring I/O 0 pages affected by kernel thread creation 61525 pages affected by fork() 31373 pages affected by vfork() 0 pages affected by rfork() 29782891 pages freed 0 pages freed by daemon 0 pages freed by exiting processes 0 pages active 0 pages inactive 0 pages in the laundry queue 0 pages wired down 0 pages free 0 bytes per page 0 total name lookups cache hits (0% pos + 0% neg) system 0% per-directory deletions 0%, falsehits 0%, toolong 0% ------------------------------------------------------------------------ vmstat -m Type InUse MemUse HighUse Requests Size(s) feeder 7 1K - 7 32 CAM I/O Scheduler 1 1K - 1 128 CAM queue 11 4K - 51 16,32,512 UART 6 5K - 6 16,1024 USB 41 153K - 42 16,128,256,512,1024,4096,32768 USBdev 35 5K - 35 32,64,128,256,512 CAM dev queue 5 1K - 5 64 ata_pci 2 1K - 2 64 vtbuf 24 1968K - 46 4096 vt 11 6K - 11 512 DEVFS3 242 61K - 265 256 DEVFS1 213 107K - 223 512 DEVFS_RULE 55 26K - 55 64,512 DEVFS 17 1K - 18 16,128 DEVFSP 0 0K - 13 64 NFSD V4client 1 1K - 1 256 NFSD lckfile 1 1K - 1 256 NFSD session 1 1K - 1 1024 pfs_nodes 20 5K - 20 256 GEOM 94 337K - 530 16,32,64,128,256,512,1024,2048,8192 raid_data 0 0K - 78 32,128,256 isadev 4 1K - 4 128 nvlist 112 11K - 112 16,64,128 ddb_capture 1 64K - 1 65536 cdev 3 1K - 3 256 filedesc 1 4K - 121 32,4096,16384 sigio 0 0K - 1 64 filecaps 0 0K - 367 16,32 kdtrace 2266 555K - 6734 64,256 kenv 120 12K - 124 16,32,64,128,8192 kqueue 66 5K - 2140 64,256,512,2048 proc-args 95 5K - 7600 16,32,64,128,256 hhook 13 4K - 13 256 ithread 264 42K - 264 32,128,256 KTR 1 14228K - 1 KTRACE 100 13K - 100 128 linker 517 3021K - 718 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 lockf 26 3K - 462 64,128 loginclass 3 1K - 3 64 cache 1 1K - 1 32 devbuf 5755 8347K - 6113 16,32,64,128,256,512,1024,2048,4096,8192,65536 temp 42 2K - 4114 16,32,64,128,256,512,1024,2048,4096,65536 module 542 68K - 546 128 mtx_pool 2 16K - 2 8192 osd 384 7K - 411 16,32,64,128,256 pmchooks 1 1K - 1 128 pgrp 19 3K - 307 128 session 19 3K - 187 128 proc 2 32K - 2 16384 subproc 202 332K - 2270 512,4096 cred 39 10K - 1844 256 acpiintr 1 1K - 1 64 acpica 1648 162K - 66398 16,32,64,128,256,512,1024 plimit 19 5K - 2786 256 uidinfo 5 5K - 101 128,4096 dumper 1 1K - 1 512 acpitask 1 64K - 1 65536 sysctl 0 0K - 1 64 sysctloid 11816 605K - 11958 16,32,64,128 sysctltmp 0 0K - 235 16,64,256,1024 kbdmux 7 22K - 7 16,512,1024,2048,16384 tidhash 1 32K - 1 32768 callout 9 3208K - 9 umtx 4592 574K - 4592 128 p1003.1b 1 1K - 1 16 SWAP 1 276K - 1 bus 1454 124K - 18685 16,32,64,128,256,1024 bus-sc 90 436K - 15636 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 devstat 18 37K - 18 32,4096 eventhandler 140 12K - 140 64,128 gtaskqueue 54 30K - 54 16,32,256,8192 kobj 356 1424K - 501 4096 Per-cpu 1 1K - 1 32 rman 409 47K - 778 16,32,128 sbuf 0 0K - 816 16,32,64,128,256,512,1024,2048,4096,8192,32768 sglist 8 8K - 8 1024 toponodes 30 4K - 30 128 taskqueue 453 59K - 573 16,32,64,128,256 terminal 11 3K - 11 256 Unitno 27 2K - 1734 32,64 vmem 3 1184K - 21 1024,2048,4096,8192,16384,32768,65536 Witness 1539 3282K - 1539 2048,16384 ioctlops 0 0K - 583 256,512,1024 select 29 4K - 29 128 iov 0 0K - 44914533 64,128,256,512 msg 4 30K - 4 2048,4096,8192,16384 sem 4 106K - 4 2048,4096 shm 1 32K - 1 32768 tty 14 14K - 17 1024 pts 0 0K - 3 256 mbuf_tag 0 0K - 7 32 shmfd 1 8K - 1 8192 soname 4 1K - 12820 16,32,128 pcb 18 1175K - 132 16,32,128,1024,2048,8192 acl 0 0K - 3 4096 vfscache 4 2145K - 4 256,32768,65536 cl_savebuf 0 0K - 63 64 vfs_hash 1 1024K - 1 vnodes 1 1K - 1 256 mount 31 2K - 101 16,32,64,128,256 statfs 0 0K - 7209 4096 vnodemarker 0 0K - 2733 512 chacha20random 1 2K - 1 2048 BPF 7 1K - 7 128 ifnet 8 15K - 8 128,2048 ifaddr 83 19K - 98 16,32,64,128,256,512,2048,4096 ether_multi 45 3K - 45 16,32,64 clone 8 1K - 8 128 ipsec 1 1K - 1 256 lltable 33 10K - 63 256,512 iflib 88 548K - 96 32,64,128,1024,4096,8192,16384,32768 routetbl 31 4K - 35 32,64,128,512 igmp 7 1K - 7 128 in_multi 3 1K - 3 256 encap_export_host 2 2K - 2 1024 sctp_a_it 0 0K - 4 16 sctp_vrf 1 1K - 1 64 sctp_ifa 6 1K - 6 128 sctp_ifn 3 1K - 3 128 sctp_iter 0 0K - 4 256 hostcache 1 32K - 1 32768 LRO 32 337K - 32 16,2048,8192,32768 tcpfunc 1 1K - 1 64 syncache 1 68K - 1 in6_multi 25 3K - 25 32,256 mld 7 1K - 7 128 ip6ndp 9 1K - 10 64,256 inpcbpolicy 106 4K - 369 32 secasvar 1 1K - 1 1024 sahead 1 1K - 1 1024 ipsecpolicy 2 2K - 2 256,1024 ipsec-saq 2 2K - 2 1024 crypto 2 2K - 2 512,1024 rpc 2 8K - 2 4096 audit_evclass 230 8K - 286 32 pagedep 1 256K - 93 256 inodedep 11 1029K - 762 512 bmsafemap 1 8K - 330 256,8192 newblk 1 2048K - 20707 256 indirdep 0 0K - 68 128,32768 freefrag 0 0K - 188 128 freeblks 0 0K - 207 256 freefile 1 1K - 113 64 diradd 0 0K - 482 128 mkdir 0 0K - 14 128 dirrem 0 0K - 455 128 newdirblk 0 0K - 7 64 freework 1 1K - 224 32,128 freedep 0 0K - 1 64 jaddref 0 0K - 496 128 jremref 0 0K - 467 128 jnewblk 0 0K - 20706 128 jfreefrag 0 0K - 188 128 jseg 1 1K - 4594 128 jsegdep 8 1K - 21857 64 sbdep 0 0K - 302 64 savedino 0 0K - 111 256 jblocks 2 1K - 2 128,256 softdep 1 1K - 1 512 ufs_dirhash 726 180K - 1042 16,32,64,128,256,512,1024 ufs_quota 1 1024K - 1 ufs_mount 3 17K - 4 512,4096,8192 vm_pgdata 1 1K - 1 128 UMAHash 122 19188K - 461 512,1024,2048,4096,8192,16384,32768,65536 memdesc 1 4K - 1 4096 pci_link 16 2K - 16 32,64,128 atkbddev 2 1K - 2 64 acpisem 41 6K - 41 128 acpidev 36 3K - 36 64 CAM SIM 5 2K - 5 256 sr_iov 4 2K - 4 512 apmdev 1 1K - 1 128 madt_table 0 0K - 2 64,4096 CAM XPT 25 2K - 107 16,32,64,128,256,512,1024,2048,65536 entropy 0 0K - 52 4096 CAM DEV 6 12K - 19 2048 CAM CCB 0 0K - 4324034 2048 CAM path 7 1K - 70 32 io_apic 2 4K - 2 2048 local_apic 1 8K - 1 8192 CAM periph 4 1K - 26 16,32,64,128,256 MCA 8 1K - 8 128 cpus 2 1K - 2 32 msi 54 7K - 54 128 nexusdev 5 1K - 5 16 fdesc_mount 1 1K - 1 16 solaris 2045562 241633K - 267431431 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 kstat_data 8 1K - 8 64 ctlmem 3609 8302K - 13486 16,32,64,128,256,512,4096,8192,65536 ctlblk 200 104K - 208 32,1024 iSCSI 1 1K - 1 128 icl 3 1K - 5 16,64 icl_soft 100 25K - 100 256 cfiscsi 200 200K - 208 1024 cxgbe 66 1755K - 17760 16,64,128,256,512,1024,2048,4096,8192,16384 fbt 119538 15199K - 119693 128 SDT 42 2K - 42 16,64 ------------------------------------------------------------------------ vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 280, 0, 443, 12, 448, 0, 0 UMA Zones: 904, 0, 460, 0, 465, 0, 0 UMA Slabs: 112, 0, 1207250, 82850,14201824, 0, 0 UMA Hash: 256, 0, 85, 20, 211, 0, 0 4 Bucket: 32, 0, 2041, 4655, 2313833, 0, 0 6 Bucket: 48, 0, 352, 3383, 322237, 0, 0 8 Bucket: 64, 0, 252, 2910, 190961, 11, 0 12 Bucket: 96, 0, 60, 1498, 76652, 0, 0 16 Bucket: 128, 0, 212, 1493, 25922, 0, 0 32 Bucket: 256, 0, 508, 482, 234762, 381, 0 64 Bucket: 512, 0, 421, 657, 253762,28255, 0 128 Bucket: 1024, 0, 331, 257, 193938, 0, 0 256 Bucket: 2048, 0, 2859, 355, 983628,98374, 0 vmem: 1792, 0, 2, 10, 3, 0, 0 vmem btag: 56, 0, 159302, 22742, 1176802,1314, 0 VM OBJECT: 256, 0, 10078, 182, 39099, 0, 0 RADIX NODE: 144, 0, 48411, 11610, 849059, 0, 0 MAP: 232, 0, 3, 65, 3, 0, 0 KMAP ENTRY: 120, 0, 25, 734, 28, 0, 0 MAP ENTRY: 120, 0, 940, 1898, 95760, 0, 0 VMSPACE: 2512, 0, 34, 32, 2102, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 mt_zone: 16400, 0, 428, 0, 428, 0, 0 16: 16, 0, 33, 1710, 27949, 0, 0 16: 16, 0, 7, 1238, 15, 0, 0 16: 16, 0, 1033513, 137534,50410986, 0, 0 16: 16, 0, 316, 1925, 12805, 0, 0 16: 16, 0, 5362, 1112, 7681, 0, 0 16: 16, 0, 269, 1972, 490, 0, 0 16: 16, 0, 132, 2109, 1525, 0, 0 16: 16, 0, 39, 2202, 112, 0, 0 32: 32, 0, 57, 1803, 7681, 0, 0 32: 32, 0, 437, 1671, 3008, 0, 0 32: 32, 0, 213045, 56779,48833930, 0, 0 32: 32, 0, 160, 1948, 5128, 0, 0 32: 32, 0, 6208, 1232, 8674, 0, 0 32: 32, 0, 381, 1727, 494, 0, 0 32: 32, 0, 170, 1938, 573, 0, 0 32: 32, 0, 0, 868, 35, 0, 0 64: 64, 0, 665, 1629, 8104, 0, 0 64: 64, 0, 125, 1921, 8969, 0, 0 64: 64, 0, 84826, 5074,43902104, 0, 0 64: 64, 0, 705, 1837, 1147, 0, 0 64: 64, 0, 1086, 3254, 26677, 0, 0 64: 64, 0, 158, 1888, 2342, 0, 0 64: 64, 0, 2038, 2054,44914727, 0, 0 64: 64, 0, 25, 2021, 169, 0, 0 128: 128, 0, 956, 811, 36988, 0, 0 128: 128, 0, 104, 919, 477, 0, 0 128: 128, 0, 124913, 4853,30046394, 0, 0 128: 128, 0, 369, 902, 1210, 0, 0 128: 128, 0, 123378, 622, 125887, 0, 0 128: 128, 0, 4823, 571, 4863, 0, 0 128: 128, 0, 1467, 951, 14202, 0, 0 128: 128, 0, 63, 960, 449, 0, 0 256: 256, 0, 19, 476, 1782, 0, 0 256: 256, 0, 2338, 287, 7686, 0, 0 256: 256, 0, 503847, 177408,63382647, 0, 0 256: 256, 0, 197, 478, 2896, 0, 0 256: 256, 0, 12, 423, 1155, 0, 0 256: 256, 0, 298, 437, 672, 0, 0 256: 256, 0, 430, 305, 2574, 0, 0 256: 256, 0, 40, 1715, 20858, 0, 0 512: 512, 0, 15, 132, 22, 0, 0 512: 512, 0, 30, 264, 1085, 0, 0 512: 512, 0, 84083, 10130,17829301, 0, 0 512: 512, 0, 4, 115, 4109, 0, 0 512: 512, 0, 2, 201, 335, 0, 0 512: 512, 0, 329, 112, 344, 0, 0 512: 512, 0, 327, 170, 3076, 0, 0 512: 512, 0, 5, 86, 154, 0, 0 1024: 1024, 0, 7, 85, 2026, 0, 0 1024: 1024, 0, 19, 125, 358, 0, 0 1024: 1024, 0, 1829, 139,12893546, 0, 0 1024: 1024, 0, 219, 57, 1740, 0, 0 1024: 1024, 0, 109, 87, 1942, 0, 0 1024: 1024, 0, 10, 42, 12, 0, 0 1024: 1024, 0, 58, 62, 63, 0, 0 1024: 1024, 0, 4, 48, 23, 0, 0 2048: 2048, 0, 9, 11, 13, 0, 0 2048: 2048, 0, 48, 22, 115, 0, 0 2048: 2048, 0, 205, 51, 1905, 0, 0 2048: 2048, 0, 3, 67, 4324561, 0, 0 2048: 2048, 0, 8, 22, 43, 0, 0 2048: 2048, 0, 1538, 12, 1539, 0, 0 2048: 2048, 0, 359, 21, 377, 0, 0 2048: 2048, 0, 9, 21, 22, 0, 0 4096: 4096, 0, 0, 1, 2, 0, 0 4096: 4096, 0, 59, 12, 7277, 0, 0 4096: 4096, 0, 200, 9, 3999, 0, 0 4096: 4096, 0, 360, 4, 1619, 0, 0 4096: 4096, 0, 221, 0, 415, 0, 0 4096: 4096, 0, 3, 0, 3, 0, 0 4096: 4096, 0, 28, 1, 30, 0, 0 4096: 4096, 0, 1, 10, 54, 0, 0 8192: 8192, 0, 5, 0, 5, 0, 0 8192: 8192, 0, 2, 2, 9, 0, 0 8192: 8192, 0, 5, 17, 157463, 0, 0 8192: 8192, 0, 1, 3, 1359, 0, 0 8192: 8192, 0, 100, 0, 102, 0, 0 8192: 8192, 0, 6, 0, 6, 0, 0 8192: 8192, 0, 39, 4, 43, 0, 0 8192: 8192, 0, 52, 1, 63, 0, 0 16384: 16384, 0, 30, 0, 30, 0, 0 16384: 16384, 0, 0, 2, 7, 0, 0 16384: 16384, 0, 2, 4, 9, 0, 0 16384: 16384, 0, 1, 3, 2085, 0, 0 16384: 16384, 0, 1, 0, 1, 0, 0 16384: 16384, 0, 1, 0, 1, 0, 0 16384: 16384, 0, 10, 13, 71, 0, 0 16384: 16384, 0, 1, 0, 1, 0, 0 32768: 32768, 0, 0, 0, 0, 0, 0 32768: 32768, 0, 5, 6, 66, 0, 0 32768: 32768, 0, 14, 8, 50, 0, 0 32768: 32768, 0, 0, 5, 1563, 0, 0 32768: 32768, 0, 0, 0, 0, 0, 0 32768: 32768, 0, 1, 0, 1, 0, 0 32768: 32768, 0, 17, 3, 20, 0, 0 32768: 32768, 0, 8, 0, 8, 0, 0 65536: 65536, 0, 0, 1, 8, 0, 0 65536: 65536, 0, 2, 1, 5, 0, 0 65536: 65536, 0, 0, 1, 2, 0, 0 65536: 65536, 0, 6, 3, 1068, 0, 0 65536: 65536, 0, 104, 7, 224, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 65536: 65536, 0, 11, 1, 14, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 SLEEPQUEUE: 88, 0, 2297, 586, 2297, 0, 0 64 pcpu: 8, 0, 3257, 1095, 3263, 0, 0 Files: 80, 0, 85, 944, 35224, 0, 0 filedesc0: 1104, 0, 66, 96, 2134, 0, 0 rl_entry: 40, 0, 88, 1991, 88, 0, 0 TURNSTILE: 136, 0, 2297, 343, 2297, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 umtx_shm: 88, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0 PROC: 1336, 0, 65, 70, 2133, 0, 0 THREAD: 1400, 0, 2257, 39, 2589, 0, 0 cpuset: 104, 0, 2167, 607, 2516, 0, 0 domainset: 32, 0, 0, 0, 0, 0, 0 audit_record: 1280, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 3246930, 1, 4313, 540, 0, 0 mbuf: 256, 3246930, 4590, 5011,176880457, 0, 0 mbuf_cluster: 2048, 507330, 6118, 74, 70330, 0, 0 mbuf_jumbo_page: 4096, 253665, 8608, 2499,73852940, 0, 0 mbuf_jumbo_9k: 9216, 225480, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 169108, 2032, 152, 4829, 0, 0 g_bio: 384, 0, 38, 762,17886150, 0, 0 ttyinq: 160, 0, 135, 537, 465, 0, 0 ttyoutq: 256, 0, 72, 483, 245, 0, 0 DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0 nvme_request: 128, 0, 0, 0, 0, 0, 0 cryptop: 128, 0, 0, 0, 0, 0, 0 cryptodesc: 72, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0 FPU_save_area: 576, 0, 0, 0, 0, 0, 0 VNODE: 472, 0, 148395, 85, 148515, 0, 0 VNODEPOLL: 120, 0, 1, 131, 1, 0, 0 BUF TRIE: 144, 0, 2109, 50730, 17971, 0, 0 NAMEI: 1024, 0, 1, 103, 211299, 0, 0 rentr: 24, 0, 0, 0, 0, 0, 0 S VFS Cache: 108, 0, 144126, 389, 148469, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 4586, 94, 4586, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 1192, 56, 1910, 0, 0 NCLNODE: 592, 0, 0, 0, 0, 0, 0 AIO: 208, 0, 0, 0, 0, 0, 0 AIOP: 32, 0, 0, 0, 0, 0, 0 AIOCB: 752, 0, 0, 0, 0, 0, 0 AIOLIO: 280, 0, 0, 0, 0, 0, 0 pipe: 760, 0, 6, 99, 892, 0, 0 Mountpoints: 2744, 0, 3, 3, 3, 0, 0 procdesc: 136, 0, 0, 0, 0, 0, 0 ksiginfo: 112, 0, 102, 1403, 1150, 0, 0 itimer: 352, 0, 0, 0, 0, 0, 0 KNOTE: 160, 0, 0, 432, 44, 0, 0 socket: 856, 260796, 114, 82, 2523, 0, 0 IPsec SA lft_c: 16, 0, 0, 0, 0, 0, 0 unpcb: 232, 260797, 8, 485, 2147, 0, 0 ipq: 56, 15904, 0, 0, 0, 0, 0 udp_inpcb: 424, 260802, 2, 232, 247, 0, 0 udpcb: 32, 260896, 2, 2106, 247, 0, 0 tcp_inpcb: 424, 260802, 104, 130, 122, 0, 0 tcpcb: 872, 260796, 104, 68, 122, 0, 0 tcptw: 88, 27810, 0, 405, 4, 0, 0 syncache: 168, 15364, 0, 506, 107, 0, 0 hostcache: 96, 15375, 0, 533, 5, 0, 0 sackhole: 32, 0, 0, 620, 3, 0, 0 tcpreass: 40, 31779, 0, 297, 2, 0, 0 sctp_ep: 1216, 260796, 0, 0, 0, 0, 0 sctp_asoc: 2408, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80012, 0, 830, 5, 0, 0 sctp_raddr: 736, 80000, 0, 0, 0, 0, 0 sctp_chunk: 152, 400010, 0, 0, 0, 0, 0 sctp_readq: 152, 400010, 0, 0, 0, 0, 0 sctp_stream_msg_out: 112, 400015, 0, 0, 0, 0, 0 sctp_asconf: 40, 400059, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0 udplite_inpcb: 424, 260802, 0, 0, 0, 0, 0 ripcb: 424, 260802, 0, 0, 0, 0, 0 rtentry: 208, 0, 15, 422, 17, 0, 0 selfd: 64, 0, 42, 1756, 36817, 0, 0 swpctrie: 144, 1014687, 179, 334, 247, 0, 0 swblk: 136, 1014681, 895, 352, 1147, 0, 0 FFS inode: 160, 0, 148252, 212, 148365, 0, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 148251, 204, 148364, 0, 0 taskq_zone: 48, 0, 0, 2656, 296782, 0, 0 reference_cache: 40, 0, 206, 2665,22861594, 0, 0 reference_history_cache: 8, 0, 6, 3480,22861394, 0, 0 range_seg_cache: 64, 0, 23743, 113959, 6807087, 0, 0 metaslab_alloc_trace_cache: 64, 0, 100, 2814,12786214, 0, 0 zio_cache: 1040, 0, 78782, 4981,88683224, 0, 0 zio_link_cache: 48, 0, 78901, 5261,70740133, 0, 0 zio_buf_512: 512, 0, 523, 149, 146958, 0, 0 zio_data_buf_512: 512, 0, 0, 0, 0, 0, 0 zio_buf_1024: 1024, 0, 0, 52, 7, 0, 0 zio_data_buf_1024: 1024, 0, 0, 0, 0, 0, 0 zio_buf_1536: 1536, 0, 0, 0, 0, 0, 0 zio_data_buf_1536: 1536, 0, 0, 0, 0, 0, 0 zio_buf_2048: 2048, 0, 101, 47, 33252, 0, 0 zio_data_buf_2048: 2048, 0, 0, 74, 112, 0, 0 zio_buf_2560: 2560, 0, 0, 0, 0, 0, 0 zio_data_buf_2560: 2560, 0, 0, 0, 0, 0, 0 zio_buf_3072: 3072, 0, 0, 0, 0, 0, 0 zio_data_buf_3072: 3072, 0, 0, 0, 0, 0, 0 zio_buf_3584: 3584, 0, 0, 0, 0, 0, 0 zio_data_buf_3584: 3584, 0, 0, 0, 0, 0, 0 zio_buf_4096: 4096, 0, 0, 65, 59646, 0, 0 zio_data_buf_4096: 4096, 0, 0, 15, 67606, 0, 0 zio_buf_5120: 5120, 0, 0, 0, 0, 0, 0 zio_data_buf_5120: 5120, 0, 0, 0, 0, 0, 0 zio_buf_6144: 6144, 0, 0, 0, 0, 0, 0 zio_data_buf_6144: 6144, 0, 0, 0, 0, 0, 0 zio_buf_7168: 7168, 0, 0, 2, 2, 0, 0 zio_data_buf_7168: 7168, 0, 0, 0, 0, 0, 0 zio_buf_8192: 8192, 0, 4, 14, 188935, 0, 0 zio_data_buf_8192: 8192, 0, 101645, 3153,13577591, 0, 0 zio_buf_10240: 10240, 0, 0, 0, 0, 0, 0 zio_data_buf_10240: 10240, 0, 0, 0, 0, 0, 0 zio_buf_12288: 12288, 0, 0, 11, 11091, 0, 0 zio_data_buf_12288: 12288, 0, 0, 0, 0, 0, 0 zio_buf_14336: 14336, 0, 0, 0, 0, 0, 0 zio_data_buf_14336: 14336, 0, 0, 0, 0, 0, 0 zio_buf_16384: 16384, 0, 125, 26, 133834, 0, 0 zio_data_buf_16384: 16384, 0, 0, 2, 3, 0, 0 zio_buf_20480: 20480, 0, 0, 11, 6941, 0, 0 zio_data_buf_20480: 20480, 0, 0, 0, 0, 0, 0 zio_buf_24576: 24576, 0, 0, 12, 69704, 0, 0 zio_data_buf_24576: 24576, 0, 0, 3, 3, 0, 0 zio_buf_28672: 28672, 0, 0, 13, 4626, 0, 0 zio_data_buf_28672: 28672, 0, 0, 0, 0, 0, 0 zio_buf_32768: 32768, 0, 0, 12, 56861, 0, 0 zio_data_buf_32768: 32768, 0, 0, 3, 3, 0, 0 zio_buf_40960: 40960, 0, 0, 12, 53025, 0, 0 zio_data_buf_40960: 40960, 0, 0, 4, 6, 0, 0 zio_buf_49152: 49152, 0, 1, 12, 49967, 0, 0 zio_data_buf_49152: 49152, 0, 0, 0, 0, 0, 0 zio_buf_57344: 57344, 0, 1, 9, 44222, 0, 0 zio_data_buf_57344: 57344, 0, 0, 0, 0, 0, 0 zio_buf_65536: 65536, 0, 0, 12, 43623, 0, 0 zio_data_buf_65536: 65536, 0, 0, 0, 0, 0, 0 zio_buf_81920: 81920, 0, 1, 11, 78182, 0, 0 zio_data_buf_81920: 81920, 0, 0, 0, 0, 0, 0 zio_buf_98304: 98304, 0, 3, 10, 71898, 0, 0 zio_data_buf_98304: 98304, 0, 0, 0, 0, 0, 0 zio_buf_114688: 114688, 0, 1, 12, 68527, 0, 0 zio_data_buf_114688: 114688, 0, 0, 0, 0, 0, 0 zio_buf_131072: 131072, 0, 1185, 62, 2063870, 0, 0 zio_data_buf_131072: 131072, 0, 0, 0, 0, 0, 0 zio_buf_163840: 163840, 0, 0, 0, 0, 0, 0 zio_data_buf_163840: 163840, 0, 0, 0, 0, 0, 0 zio_buf_196608: 196608, 0, 0, 0, 0, 0, 0 zio_data_buf_196608: 196608, 0, 0, 0, 0, 0, 0 zio_buf_229376: 229376, 0, 0, 0, 0, 0, 0 zio_data_buf_229376: 229376, 0, 0, 0, 0, 0, 0 zio_buf_262144: 262144, 0, 0, 0, 0, 0, 0 zio_data_buf_262144: 262144, 0, 0, 0, 0, 0, 0 zio_buf_327680: 327680, 0, 0, 0, 0, 0, 0 zio_data_buf_327680: 327680, 0, 0, 0, 0, 0, 0 zio_buf_393216: 393216, 0, 0, 0, 0, 0, 0 zio_data_buf_393216: 393216, 0, 0, 0, 0, 0, 0 zio_buf_458752: 458752, 0, 0, 0, 0, 0, 0 zio_data_buf_458752: 458752, 0, 0, 0, 0, 0, 0 zio_buf_524288: 524288, 0, 0, 0, 0, 0, 0 zio_data_buf_524288: 524288, 0, 0, 0, 0, 0, 0 zio_buf_655360: 655360, 0, 0, 0, 0, 0, 0 zio_data_buf_655360: 655360, 0, 0, 0, 0, 0, 0 zio_buf_786432: 786432, 0, 0, 0, 0, 0, 0 zio_data_buf_786432: 786432, 0, 0, 0, 0, 0, 0 zio_buf_917504: 917504, 0, 0, 0, 0, 0, 0 zio_data_buf_917504: 917504, 0, 0, 0, 0, 0, 0 zio_buf_1048576: 1048576, 0, 0, 0, 0, 0, 0 zio_data_buf_1048576: 1048576, 0, 0, 0, 0, 0, 0 zio_buf_1310720: 1310720, 0, 0, 0, 0, 0, 0 zio_data_buf_1310720: 1310720, 0, 0, 0, 0, 0, 0 zio_buf_1572864: 1572864, 0, 0, 0, 0, 0, 0 zio_data_buf_1572864: 1572864, 0, 0, 0, 0, 0, 0 zio_buf_1835008: 1835008, 0, 0, 0, 0, 0, 0 zio_data_buf_1835008: 1835008, 0, 0, 0, 0, 0, 0 zio_buf_2097152: 2097152, 0, 0, 0, 0, 0, 0 zio_data_buf_2097152: 2097152, 0, 0, 0, 0, 0, 0 zio_buf_2621440: 2621440, 0, 0, 0, 0, 0, 0 zio_data_buf_2621440: 2621440, 0, 0, 0, 0, 0, 0 zio_buf_3145728: 3145728, 0, 0, 0, 0, 0, 0 zio_data_buf_3145728: 3145728, 0, 0, 0, 0, 0, 0 zio_buf_3670016: 3670016, 0, 0, 0, 0, 0, 0 zio_data_buf_3670016: 3670016, 0, 0, 0, 0, 0, 0 zio_buf_4194304: 4194304, 0, 0, 0, 0, 0, 0 zio_data_buf_4194304: 4194304, 0, 0, 0, 0, 0, 0 zio_buf_5242880: 5242880, 0, 0, 0, 0, 0, 0 zio_data_buf_5242880: 5242880, 0, 0, 0, 0, 0, 0 zio_buf_6291456: 6291456, 0, 0, 0, 0, 0, 0 zio_data_buf_6291456: 6291456, 0, 0, 0, 0, 0, 0 zio_buf_7340032: 7340032, 0, 0, 0, 0, 0, 0 zio_data_buf_7340032: 7340032, 0, 0, 0, 0, 0, 0 zio_buf_8388608: 8388608, 0, 0, 0, 0, 0, 0 zio_data_buf_8388608: 8388608, 0, 0, 0, 0, 0, 0 zio_buf_10485760: 10485760, 0, 0, 0, 0, 0, 0 zio_data_buf_10485760: 10485760, 0, 0, 0, 0, 0, 0 zio_buf_12582912: 12582912, 0, 0, 0, 0, 0, 0 zio_data_buf_12582912: 12582912, 0, 0, 0, 0, 0, 0 zio_buf_14680064: 14680064, 0, 0, 0, 0, 0, 0 zio_data_buf_14680064: 14680064, 0, 0, 0, 0, 0, 0 zio_buf_16777216: 16777216, 0, 0, 0, 0, 0, 0 zio_data_buf_16777216: 16777216, 0, 0, 0, 0, 0, 0 lz4_ctx: 16384, 0, 0, 14, 349389, 0, 0 abd_chunk: 4096, 0, 805322, 271464,42895383, 0, 0 sa_cache: 152, 0, 0, 0, 0, 0, 0 dnode_t: 952, 0, 1250, 98, 2821, 0, 0 arc_buf_hdr_t_full: 344, 0, 727061, 117057,10272586, 0, 0 arc_buf_hdr_t_l2only: 88, 0, 0, 0, 0, 0, 0 arc_buf_t: 64, 0, 102849, 53267,13654973, 0, 0 dmu_buf_impl_t: 344, 0, 95433, 14314, 9647491, 0, 0 zil_lwb_cache: 320, 0, 641, 8863, 1349166, 0, 0 zil_zcw_cache: 80, 0, 46, 1326, 747269, 0, 0 zfs_znode_cache: 264, 0, 0, 0, 0, 0, 0 CTL IO: 672, 0, 116, 784,15894509, 0, 0 beio: 368, 0, 111, 509,15885991, 0, 0 icl_pdu: 80, 0, 116, 2481,34735588, 0, 0 iscsi_outstanding: 48, 0, 0, 0, 0, 0, 0 cfiscsi_data_wait: 72, 0, 0, 1265, 6621541, 0, 0 dtrace_state_cache: 16384, 0, 0, 3, 7, 0, 0 taskq_zone: 48, 0, 0, 0, 0, 0, 0 cblk0: 131072, 0, 0, 49, 157637, 0, 0 cblk1: 131072, 0, 0, 12, 159298, 0, 0 cblk2: 131072, 0, 2, 11, 159265, 0, 0 cblk3: 131072, 0, 2, 9, 159279, 0, 0 cblk4: 131072, 0, 1, 10, 158983, 0, 0 cblk5: 131072, 0, 0, 27, 157604, 0, 0 cblk6: 131072, 0, 0, 39, 157539, 0, 0 cblk7: 131072, 0, 1, 11, 166138, 0, 0 cblk8: 131072, 0, 2, 8, 159463, 0, 0 cblk9: 131072, 0, 2, 9, 159316, 0, 0 cblk10: 131072, 0, 1, 11, 159661, 0, 0 cblk11: 131072, 0, 2, 10, 159458, 0, 0 cblk12: 131072, 0, 0, 46, 157175, 0, 0 cblk13: 131072, 0, 1, 10, 159309, 0, 0 cblk14: 131072, 0, 2, 11, 159145, 0, 0 cblk15: 131072, 0, 0, 32, 167576, 0, 0 cblk16: 131072, 0, 2, 14, 165405, 0, 0 cblk17: 131072, 0, 2, 10, 158839, 0, 0 cblk18: 131072, 0, 2, 14, 162199, 0, 0 cblk19: 131072, 0, 2, 9, 158496, 0, 0 cblk20: 131072, 0, 0, 34, 167728, 0, 0 cblk21: 131072, 0, 0, 26, 157214, 0, 0 cblk22: 131072, 0, 1, 14, 160536, 0, 0 cblk23: 131072, 0, 2, 13, 162388, 0, 0 cblk24: 131072, 0, 1, 11, 162512, 0, 0 cblk25: 131072, 0, 1, 11, 159348, 0, 0 cblk26: 131072, 0, 2, 10, 158595, 0, 0 cblk27: 131072, 0, 1, 10, 159268, 0, 0 cblk28: 131072, 0, 0, 36, 157205, 0, 0 cblk29: 131072, 0, 0, 16, 159585, 0, 0 cblk30: 131072, 0, 1, 11, 158915, 0, 0 cblk31: 131072, 0, 2, 9, 158397, 0, 0 cblk32: 131072, 0, 0, 21, 157126, 0, 0 cblk33: 131072, 0, 0, 16, 164435, 0, 0 cblk34: 131072, 0, 1, 10, 159213, 0, 0 cblk35: 131072, 0, 2, 8, 158768, 0, 0 cblk36: 131072, 0, 1, 10, 159256, 0, 0 cblk37: 131072, 0, 1, 11, 165761, 0, 0 cblk38: 131072, 0, 0, 26, 167712, 0, 0 cblk39: 131072, 0, 2, 10, 159270, 0, 0 cblk40: 131072, 0, 2, 9, 158913, 0, 0 cblk41: 131072, 0, 0, 31, 167693, 0, 0 cblk42: 131072, 0, 1, 15, 162378, 0, 0 cblk43: 131072, 0, 2, 12, 159233, 0, 0 cblk44: 131072, 0, 0, 32, 167757, 0, 0 cblk45: 131072, 0, 2, 8, 159066, 0, 0 cblk46: 131072, 0, 1, 11, 160342, 0, 0 cblk47: 131072, 0, 2, 13, 161711, 0, 0 cblk48: 131072, 0, 0, 50, 157143, 0, 0 cblk49: 131072, 0, 2, 11, 162756, 0, 0 cblk50: 131072, 0, 2, 25, 162274, 0, 0 cblk51: 131072, 0, 1, 15, 162381, 0, 0 cblk52: 131072, 0, 0, 15, 160311, 0, 0 cblk53: 131072, 0, 2, 10, 158434, 0, 0 cblk54: 131072, 0, 2, 10, 159230, 0, 0 cblk55: 131072, 0, 2, 10, 159312, 0, 0 cblk56: 131072, 0, 2, 10, 158906, 0, 0 cblk57: 131072, 0, 2, 10, 159196, 0, 0 cblk58: 131072, 0, 0, 24, 157183, 0, 0 cblk59: 131072, 0, 2, 9, 159171, 0, 0 cblk60: 131072, 0, 0, 9, 157181, 0, 0 cblk61: 131072, 0, 1, 10, 158836, 0, 0 cblk62: 131072, 0, 0, 29, 157182, 0, 0 cblk63: 131072, 0, 1, 15, 162359, 0, 0 cblk64: 131072, 0, 2, 11, 159276, 0, 0 cblk65: 131072, 0, 0, 19, 167554, 0, 0 cblk66: 131072, 0, 1, 11, 158584, 0, 0 cblk67: 131072, 0, 1, 13, 159437, 0, 0 cblk68: 131072, 0, 0, 13, 162409, 0, 0 cblk69: 131072, 0, 1, 11, 159242, 0, 0 cblk70: 131072, 0, 1, 9, 158524, 0, 0 cblk71: 131072, 0, 0, 15, 162211, 0, 0 cblk72: 131072, 0, 0, 59, 167707, 0, 0 cblk73: 131072, 0, 2, 14, 162327, 0, 0 cblk74: 131072, 0, 1, 10, 158734, 0, 0 cblk75: 131072, 0, 1, 11, 158548, 0, 0 cblk76: 131072, 0, 0, 13, 157136, 0, 0 cblk77: 131072, 0, 2, 9, 162563, 0, 0 cblk78: 131072, 0, 0, 30, 157154, 0, 0 cblk79: 131072, 0, 1, 11, 166064, 0, 0 cblk80: 131072, 0, 0, 11, 159294, 0, 0 cblk81: 131072, 0, 0, 32, 167720, 0, 0 cblk82: 131072, 0, 1, 11, 162413, 0, 0 cblk83: 131072, 0, 0, 23, 167596, 0, 0 cblk84: 131072, 0, 2, 10, 158783, 0, 0 cblk85: 131072, 0, 0, 39, 157167, 0, 0 cblk86: 131072, 0, 1, 10, 159136, 0, 0 cblk87: 131072, 0, 0, 27, 167636, 0, 0 cblk88: 131072, 0, 2, 8, 158747, 0, 0 cblk89: 131072, 0, 1, 10, 158839, 0, 0 cblk90: 131072, 0, 0, 21, 157152, 0, 0 cblk91: 131072, 0, 0, 10, 167677, 0, 0 cblk92: 131072, 0, 0, 29, 157185, 0, 0 cblk93: 131072, 0, 1, 11, 158906, 0, 0 cblk94: 131072, 0, 0, 27, 157208, 0, 0 cblk95: 131072, 0, 2, 10, 158907, 0, 0 cblk96: 131072, 0, 1, 11, 162523, 0, 0 cblk97: 131072, 0, 2, 8, 158365, 0, 0 cblk98: 131072, 0, 0, 28, 157114, 0, 0 cblk99: 131072, 0, 1, 11, 166187, 0, 0 ------------------------------------------------------------------------ vmstat -i interrupt total rate irq4: uart0 5 0 irq6: fdc0 1 0 irq17: atapci1 4323947 40178 irq20: hpet0 uhci0 14963078 139038 irq23: uhci3 ehci0 2 0 irq256: igb0:rxq0 305820 2842 irq257: igb0:rxq1 28875 268 irq258: igb0:rxq2 16373 152 irq259: igb0:rxq3 21360 198 irq260: igb0:aq 3 0 irq289: t6nex0:evt 2 0 irq290: t6nex0:0a0 4431856 41181 irq291: t6nex0:0a1 4110493 38195 irq292: t6nex0:0a2 4060181 37727 irq293: t6nex0:0a3 4195117 38981 irq294: t6nex0:0a4 4697388 43648 irq295: t6nex0:0a5 4018159 37337 irq296: t6nex0:0a6 4126775 38346 irq297: t6nex0:0a7 4413716 41012 irq298: t6nex0:0A0 40071 372 irq299: t6nex0:0A1 2399 22 Total 53755621 499499 ------------------------------------------------------------------------ pstat -T 85/260795 files 0M/0M swap space ------------------------------------------------------------------------ pstat -s Device 512-blocks Used Avail Capacity ------------------------------------------------------------------------ iostat tty ada0 pass0 cbb0 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 3 53.99 151 7.94 0.00 0 0.00 30.42 6 0.16 0 0 8 1 91 ------------------------------------------------------------------------ ipcs -a Message Queues: T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME Semaphores: T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME ------------------------------------------------------------------------ ipcs -T msginfo: msgmax: 16384 (max characters in a message) msgmni: 40 (# of message queues) msgmnb: 2048 (max characters in a message queue) msgtql: 40 (max # of messages in system) msgssz: 8 (size of a message segment) msgseg: 2048 (# of message segments in system) shminfo: shmmax: 536870912 (max shared memory segment size) shmmin: 1 (min shared memory segment size) shmmni: 192 (max number of shared memory identifiers) shmseg: 128 (max shared memory segments per process) shmall: 131072 (max amount of shared memory in pages) seminfo: semmni: 50 (# of semaphore identifiers) semmns: 340 (# of semaphores in system) semmnu: 150 (# of undo structures in system) semmsl: 340 (max # of semaphores per id) semopm: 100 (max # of operations per semop call) semume: 50 (max # of undo entries per process) semusz: 632 (size in bytes of undo structure) semvmx: 32767 (semaphore maximum value) semaem: 16384 (adjust on exit max value) ------------------------------------------------------------------------ nfsstat Rpc Counts: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 0 0 0 0 Rpc Info: TimedOut Invalid X Replies Retries Requests 0 0 0 0 0 Cache Info: Attr Hits Attr Misses Lkup Hits Lkup Misses BioR Hits BioR Misses BioW Hits BioW Misses 0 0 0 0 0 0 0 0 BioRL Hits BioRL Misses BioD Hits BioD Misses DirE Hits DirE Misses Accs Hits Accs Misses 0 0 0 0 0 0 0 0 Server Info: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 0 0 0 0 Server Re-Failed: 140737488346832 Server Faults: 140737488346832 Server Write WriteOps WriteRPC Opsaved 0 0 0 Server Cache Inprog Idem Non-Idem Misses 0 0 0 0 ------------------------------------------------------------------------ netstat -s tcp: 29339027 packets sent 19255278 data packets (141016252366 bytes) 77 data packets (8512 bytes) retransmitted 6 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 10019797 ack-only packets (994363 delayed) 0 URG only packets 0 window probe packets 63867 window update packets 8 control packets 34513481 packets received 49138369 acks (for 141016252216 bytes) 226 duplicate acks 0 acks for unsent data 92121353 packets (143631167516 bytes) received in-sequence 302 completely duplicate packets (23584 bytes) 0 old duplicate packets 0 packets with some dup. data (0 bytes duped) 2 out-of-order packets (192 bytes) 0 packets (0 bytes) of data after window 0 window probes 0 window update packets 0 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 discarded due to memory problems 3 connection requests 107 connection accepts 0 bad connection attempts 0 listen queue overflows 0 ignored RSTs in the windows 109 connections established (including accepts) 0 times used RTT from hostcache 0 times used RTT variance from hostcache 0 times used slow-start threshold from hostcache 18 connections closed (including 4 drops) 5 connections updated cached RTT on close 5 connections updated cached RTT variance on close 1 connection updated cached ssthresh on close 1 embryonic connection dropped 21856269 segments updated rtt (of 18448029 attempts) 75 retransmit timeouts 1 connection dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 18 keepalive timeouts 16 keepalive probes sent 2 connections dropped by keepalive 13374155 correct ACK header predictions 12656689 correct data packet header predictions 107 syncache entries added 0 retransmitted 0 dupsyn 0 dropped 107 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 107 cookies sent 107 cookies received 5 hostcache entries added 0 bucket overflow 1 SACK recovery episode 1 segment rexmit in SACK recovery episodes 64 byte rexmits in SACK recovery episodes 16 SACK options (SACK blocks) received 1 SACK option (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window 0 packets with matching signature received 0 packets with bad signature received 0 times failed to make signature due to no SA 0 times unexpected signature received 0 times no signature provided by segment 0 Path MTU discovery black hole detection activations 0 Path MTU discovery black hole detection min MSS activations 0 Path MTU discovery black hole detection failures TCP connection count by state: 0 connections in CLOSED state 4 connections in LISTEN state 0 connections in SYN_SENT state 3 connections in SYN_RCVD state 97 connections in ESTABLISHED state 0 connections in CLOSE_WAIT state 2 connections in FIN_WAIT_1 state 18446744073709551614 connections in CLOSING state 0 connections in LAST_ACK state 0 connections in FIN_WAIT_2 state 0 connections in TIME_WAIT state udp: 52038 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 8 with no checksum 0 dropped due to no socket 51945 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 93 delivered 93 datagrams output 0 times multicast source filter matched ip: 34568050 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 packets reassembled ok 34568050 packets for this host 0 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 0 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 29339134 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 0 calls to icmp_error 0 errors not generated in response to an icmp message 0 messages with bad code fields 0 messages less than the minimum length 0 messages with bad checksum 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored Input histogram: destination unreachable: 33 0 message responses generated 0 invalid return addresses 0 no return routes ipsec: 0 inbound packets violated process security policy 0 inbound packets failed due to insufficient memory 0 invalid inbound packets 0 outbound packets violated process security policy 0 outbound packets with no SA available 0 outbound packets failed due to insufficient memory 0 outbound packets with no route available 0 invalid outbound packets 0 outbound packets with bundled SAs 0 mbufs coalesced during clone 0 clusters coalesced during clone 0 clusters copied during clone 0 mbufs inserted during makespace ah: 0 packets shorter than header shows 0 packets dropped; protocol family not supported 0 packets dropped; no TDB 0 packets dropped; bad KCR 0 packets dropped; queue full 0 packets dropped; no transform 0 replay counter wraps 0 packets dropped; bad authentication detected 0 packets dropped; bad authentication length 0 possible replay packets detected 0 packets in 0 packets out 0 packets dropped; invalid TDB 0 bytes in 0 bytes out 0 packets dropped; larger than IP_MAXPACKET 0 packets blocked due to policy 0 crypto processing failures 0 tunnel sanity check failures esp: 0 packets shorter than header shows 0 packets dropped; protocol family not supported 0 packets dropped; no TDB 0 packets dropped; bad KCR 0 packets dropped; queue full 0 packets dropped; no transform 0 packets dropped; bad ilen 0 replay counter wraps 0 packets dropped; bad encryption detected 0 packets dropped; bad authentication detected 0 possible replay packets detected 0 packets in 0 packets out 0 packets dropped; invalid TDB 0 bytes in 0 bytes out 0 packets dropped; larger than IP_MAXPACKET 0 packets blocked due to policy 0 crypto processing failures 0 tunnel sanity check failures ipcomp: 0 packets shorter than header shows 0 packets dropped; protocol family not supported 0 packets dropped; no TDB 0 packets dropped; bad KCR 0 packets dropped; queue full 0 packets dropped; no transform 0 replay counter wraps 0 packets in 0 packets out 0 packets dropped; invalid TDB 0 bytes in 0 bytes out 0 packets dropped; larger than IP_MAXPACKET 0 packets blocked due to policy 0 crypto processing failures 0 packets sent uncompressed; size < compr. algo. threshold 0 packets sent uncompressed; compression was useless arp: 27 ARP requests sent 10 ARP replies sent 255669 ARP requests received 1318 ARP replies received 256987 ARP packets received 61 total packets dropped due to no ARP entry 23 ARP entrys timed out 0 Duplicate IPs seen ip6: 0 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 0 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 5 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 0 multicast packets which we don't join Mbuf statistics: 0 one mbuf 0 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not contiguous 0 tunneling packets that can't find gif 0 packets discarded because of too many headers 0 failures of source address selection icmp6: 0 calls to icmp6_error 0 errors not generated in response to an icmp6 message 0 errors not generated because of rate limitation Output histogram: neighbor solicitation: 1 MLDv2 listener report: 1 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 0 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes ipsec6: 0 inbound packets violated process security policy 0 inbound packets failed due to insufficient memory 0 invalid inbound packets 0 outbound packets violated process security policy 0 outbound packets with no SA available 0 outbound packets failed due to insufficient memory 0 outbound packets with no route available 0 invalid outbound packets 0 outbound packets with bundled SAs 0 mbufs coalesced during clone 0 clusters coalesced during clone 0 clusters copied during clone 0 mbufs inserted during makespace rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output pfkey: 0 requests sent from userland 0 bytes sent from userland 0 messages with invalid length field 0 messages with invalid version field 0 messages with invalid message type field 0 messages too short 0 messages with memory allocation failure 0 messages with duplicate extension 0 messages with invalid extension type 0 messages with invalid sa type 0 messages with invalid address extension 0 requests sent to userland 0 bytes sent to userland 0 messages toward single socket 0 messages toward all sockets 0 messages toward registered sockets 0 messages with memory allocation failure ------------------------------------------------------------------------ netstat -m 4591/9324/13915 mbufs in use (current/cache/total) 1805/4387/6192/507330 mbuf clusters in use (current/cache/total/max) 1/4313 mbuf+clusters out of packet secondary zone in use (current/cache) 8608/2499/11107/253665 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/225480 9k jumbo clusters in use (current/cache/total/max) 2032/152/2184/169108 16k jumbo clusters in use (current/cache/total/max) 71701K/23533K/95234K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 sendfile syscalls 0 sendfile syscalls completed without I/O request 0 requests for I/O initiated by sendfile 0 pages read by sendfile as part of a request 0 pages were valid at time of a sendfile request 0 pages were valid and substituted to bogus page 0 pages were requested for read ahead by applications 0 pages were read ahead by sendfile 0 times sendfile encountered an already busy page 0 requests for sfbufs denied 0 requests for sfbufs delayed ------------------------------------------------------------------------ netstat -anA Active Internet connections (including servers) Tcpcb Proto Recv-Q Send-Q Local Address Foreign Address (state) fffff8001f01c350 udp4 0 0 *.514 *.* fffff8001f01c4f8 udp6 0 0 *.514 *.* ------------------------------------------------------------------------ netstat -aL Current listen queue sizes (qlen/incqlen/maxqlen) Proto Listen Local Address ------------------------------------------------------------------------ fstat fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0xc00000000 fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0xc00000000 fstat: can't read file 7 at 0x20007ffffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0xa00000000 fstat: can't read file 7 at 0x20007ffffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0xa00000000 fstat: can't read file 7 at 0x20007ffffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0xa00000000 fstat: can't read file 7 at 0x20007ffffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0x800000000 fstat: can't read file 7 at 0x20007ffffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0x800000000 fstat: can't read file 7 at 0x20007ffffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0x400000000 fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0xc00000000 fstat: can't read file 7 at 0x20007ffffffffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0x600000000 fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0x400000000 fstat: can't read file 7 at 0x20007ffffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read file 11 at 0x400000000 fstat: can't read file 13 at 0x20007ffffffffff fstat: can't read file 14 at 0x4000000001fffff fstat: can't read file 16 at 0x780000ffff fstat: can't read file 17 at 0x400000000 fstat: can't read file 19 at 0x20007ffffffffff fstat: can't read file 20 at 0x4000000001fffff fstat: can't read file 22 at 0x780000ffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 5 at 0x800000000 fstat: can't read file 7 at 0x20007ffffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 1 at 0x20007ffffffffff fstat: can't read file 2 at 0x4000000001fffff USER CMD PID FD MOUNT INUM MODE SZ|DV R/W root cat 2101 root / 2 drwxr-xr-x 1024 r root cat 2101 wd / 80256 drwxr-xr-x 512 r root cat 2101 text / 5619725 -r-xr-xr-x 23096 r root cat 2101 0* pipe fffff80005ba95f0 <-> fffff80005ba9758 0 rw root sh 2099 root / 2 drwxr-xr-x 1024 r root sh 2099 wd / 80256 drwxr-xr-x 512 r root sh 2099 text / 5619754 -r-xr-xr-x 166544 r root sh 2099 0* pipe fffff80005ba95f0 <-> fffff80005ba9758 0 rw root find 2098 root / 2 drwxr-xr-x 1024 r root find 2098 wd / 9572578 drwxr-xr-x 109056 r root find 2098 text / 5620015 -r-xr-xr-x 64056 r root find 2098 0 /dev 14 crw-rw-rw- null r root sh 2094 root / 2 drwxr-xr-x 1024 r root sh 2094 wd / 80256 drwxr-xr-x 512 r root sh 2094 text / 5619754 -r-xr-xr-x 166544 r root sh 2094 0 /dev 14 crw-rw-rw- null r root sh 2094 6 / 5618063 -rw------- 36 w root mail 2093 root / 2 drwxr-xr-x 1024 r root mail 2093 wd / 80256 drwxr-xr-x 512 r root mail 2093 text / 5620210 -r-xr-xr-x 109112 r root mail 2093 0* pipe fffff80005bb8be0 <-> fffff80005bb8d48 0 rw root sh 2092 root / 2 drwxr-xr-x 1024 r root sh 2092 wd / 80256 drwxr-xr-x 512 r root sh 2092 text / 5619754 -r-xr-xr-x 166544 r root sh 2092 0 /dev 14 crw-rw-rw- null r root sh 2084 root / 2 drwxr-xr-x 1024 r root sh 2084 wd / 80256 drwxr-xr-x 512 r root sh 2084 text / 5619754 -r-xr-xr-x 166544 r root sh 2084 0 /dev 14 crw-rw-rw- null r root sh 2084 6 / 5618042 -rw------- 48 w root lockf 2083 root / 2 drwxr-xr-x 1024 r root lockf 2083 wd / 80256 drwxr-xr-x 512 r root lockf 2083 text / 5620053 -r-xr-xr-x 14824 r root lockf 2083 0 /dev 14 crw-rw-rw- null r root sh 2081 root / 2 drwxr-xr-x 1024 r root sh 2081 wd / 80256 drwxr-xr-x 512 r root sh 2081 text / 5619754 -r-xr-xr-x 166544 r root sh 2081 0 /dev 14 crw-rw-rw- null r root sh 2081 6 / 5618042 -rw------- 48 w root sh 2080 root / 2 drwxr-xr-x 1024 r root sh 2080 wd / 80256 drwxr-xr-x 512 r root sh 2080 text / 5619754 -r-xr-xr-x 166544 r root sh 2080 0 /dev 14 crw-rw-rw- null r root sh 2080 6 / 5618042 -rw------- 48 w root mail 1994 root / 2 drwxr-xr-x 1024 r root mail 1994 wd / 80256 drwxr-xr-x 512 r root mail 1994 text / 5620210 -r-xr-xr-x 109112 r root mail 1994 0* pipe fffff80005dcd000 <-> fffff80005dcd168 0 rw root sh 1993 root / 2 drwxr-xr-x 1024 r root sh 1993 wd / 80256 drwxr-xr-x 512 r root sh 1993 text / 5619754 -r-xr-xr-x 166544 r root sh 1993 0* pipe fffff80005c738e8 <-> fffff80005c73a50 0 rw root sh 1984 root / 2 drwxr-xr-x 1024 r root sh 1984 wd / 80256 drwxr-xr-x 512 r root sh 1984 text / 5619754 -r-xr-xr-x 166544 r root sh 1984 0* pipe fffff80005c738e8 <-> fffff80005c73a50 0 rw root sh 1984 6* pipe fffff80005d7e168 <-> fffff80005d7e000 0 rw root lockf 1983 root / 2 drwxr-xr-x 1024 r root lockf 1983 wd / 80256 drwxr-xr-x 512 r root lockf 1983 text / 5620053 -r-xr-xr-x 14824 r root lockf 1983 0* pipe fffff80005c738e8 <-> fffff80005c73a50 0 rw root sh 1981 root / 2 drwxr-xr-x 1024 r root sh 1981 wd / 80256 drwxr-xr-x 512 r root sh 1981 text / 5619754 -r-xr-xr-x 166544 r root sh 1981 0* pipe fffff80005c738e8 <-> fffff80005c73a50 0 rw root sh 1981 6* pipe fffff80005d7e168 <-> fffff80005d7e000 0 rw root cron 1979 root / 2 drwxr-xr-x 1024 r root cron 1979 wd / 1685387 drwxr-x--- 512 r root cron 1979 text / 5620273 -r-xr-xr-x 51800 r root cron 1979 0 /dev 14 crw-rw-rw- null rw root ctld 1253 root / 2 drwxr-xr-x 1024 r root ctld 1253 wd / 2 drwxr-xr-x 1024 r root ctld 1253 text / 5620276 -r-xr-xr-x 129688 r root ctld 1253 0 /dev 14 crw-rw-rw- null rw root ctld 1253 6 /dev 14 crw-rw-rw- null rw root ftcleanup 1137 root / 2 drwxr-xr-x 1024 r root ftcleanup 1137 wd / 2 drwxr-xr-x 1024 r root ctl 872 root / 2 drwxr-xr-x 1024 r root ctl 872 wd / 2 drwxr-xr-x 1024 r root zfskern 859 root / 2 drwxr-xr-x 1024 r root zfskern 859 wd / 2 drwxr-xr-x 1024 r root getty 761 root / 2 drwxr-xr-x 1024 r root getty 761 wd / 2 drwxr-xr-x 1024 r root getty 761 text / 5619696 -r-xr-xr-x 35400 r root getty 761 ctty /dev 55 crw------- ttyu0 rw root getty 761 0 /dev 55 crw------- ttyu0 rw root getty 760 root / 2 drwxr-xr-x 1024 r root getty 760 wd / 2 drwxr-xr-x 1024 r root getty 760 text / 5619696 -r-xr-xr-x 35400 r root getty 760 ctty /dev 86 crw------- ttyv7 rw root getty 760 0 /dev 86 crw------- ttyv7 rw root getty 759 root / 2 drwxr-xr-x 1024 r root getty 759 wd / 2 drwxr-xr-x 1024 r root getty 759 text / 5619696 -r-xr-xr-x 35400 r root getty 759 ctty /dev 85 crw------- ttyv6 rw root getty 759 0 /dev 85 crw------- ttyv6 rw root getty 758 root / 2 drwxr-xr-x 1024 r root getty 758 wd / 2 drwxr-xr-x 1024 r root getty 758 text / 5619696 -r-xr-xr-x 35400 r root getty 758 ctty /dev 84 crw------- ttyv5 rw root getty 758 0 /dev 84 crw------- ttyv5 rw root getty 757 root / 2 drwxr-xr-x 1024 r root getty 757 wd / 2 drwxr-xr-x 1024 r root getty 757 text / 5619696 -r-xr-xr-x 35400 r root getty 757 ctty /dev 83 crw------- ttyv4 rw root getty 757 0 /dev 83 crw------- ttyv4 rw root getty 756 root / 2 drwxr-xr-x 1024 r root getty 756 wd / 2 drwxr-xr-x 1024 r root getty 756 text / 5619696 -r-xr-xr-x 35400 r root getty 756 ctty /dev 82 crw------- ttyv3 rw root getty 756 0 /dev 82 crw------- ttyv3 rw root getty 755 root / 2 drwxr-xr-x 1024 r root getty 755 wd / 2 drwxr-xr-x 1024 r root getty 755 text / 5619696 -r-xr-xr-x 35400 r root getty 755 ctty /dev 81 crw------- ttyv2 rw root getty 755 0 /dev 81 crw------- ttyv2 rw root getty 754 root / 2 drwxr-xr-x 1024 r root getty 754 wd / 2 drwxr-xr-x 1024 r root getty 754 text / 5619696 -r-xr-xr-x 35400 r root getty 754 ctty /dev 80 crw------- ttyv1 rw root getty 754 0 /dev 80 crw------- ttyv1 rw root getty 753 root / 2 drwxr-xr-x 1024 r root getty 753 wd / 2 drwxr-xr-x 1024 r root getty 753 text / 5619696 -r-xr-xr-x 35400 r root getty 753 ctty /dev 79 crw------- ttyv0 rw root getty 753 0 /dev 79 crw------- ttyv0 rw root cron 706 root / 2 drwxr-xr-x 1024 r root cron 706 wd / 1685387 drwxr-x--- 512 r root cron 706 text / 5620273 -r-xr-xr-x 51800 r root cron 706 0 /dev 14 crw-rw-rw- null rw smmsp sendmail 702 root / 2 drwxr-xr-x 1024 r smmsp sendmail 702 wd / 1685399 drwxrwx--- 512 r smmsp sendmail 702 text / 5620514 -r-xr-sr-x 735944 r smmsp sendmail 702 0 /dev 14 crw-rw-rw- null r root sendmail 699 root / 2 drwxr-xr-x 1024 r root sendmail 699 wd / 1685402 drwxr-xr-x 512 r root sendmail 699 text / 5620514 -r-xr-sr-x 735944 r root sendmail 699 0 /dev 14 crw-rw-rw- null r root sshd 696 root / 2 drwxr-xr-x 1024 r root sshd 696 wd / 2 drwxr-xr-x 1024 r root sshd 696 text / 5619974 -r-xr-xr-x 285472 r root sshd 696 0 /dev 14 crw-rw-rw- null rw root syslogd 571 root / 2 drwxr-xr-x 1024 r root syslogd 571 wd / 2 drwxr-xr-x 1024 r root syslogd 571 text / 5620343 -r-xr-xr-x 51784 r root syslogd 571 0 /dev 14 crw-rw-rw- null rw root syslogd 571 6 /dev 14 crw-rw-rw- null rw root syslogd 571 12 /dev 14 crw-rw-rw- null rw root syslogd 571 18* pipe fffff80005d42be0 <-> fffff80005d42d48 0 rw root devd 425 root / 2 drwxr-xr-x 1024 r root devd 425 wd / 2 drwxr-xr-x 1024 r root devd 425 text / 5619928 -r-xr-xr-x 1036400 r root devd 425 0 /dev 14 crw-rw-rw- null rw root devd 425 6 /dev 14 crw-rw-rw- null rw root adjkerntz 125 root / 2 drwxr-xr-x 1024 r root adjkerntz 125 wd / 2 drwxr-xr-x 1024 r root adjkerntz 125 text / 5619843 -r-xr-xr-x 14832 r root adjkerntz 125 0 /dev 14 crw-rw-rw- null rw root init 1 root / 2 drwxr-xr-x 1024 r root init 1 wd / 2 drwxr-xr-x 1024 r root init 1 text / 5619887 -r-xr-xr-x 1277680 r root kernel 0 root / 2 drwxr-xr-x 1024 r root kernel 0 wd / 2 drwxr-xr-x 1024 r ------------------------------------------------------------------------ dmesg Copyright (c) 1992-2018 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 12.0-CURRENT #8: Fri Feb 16 14:58:28 IST 2018 root@munnar:/usr/obj/usr/src/amd64.amd64/sys/MYKERNEL amd64 FreeBSD clang version 6.0.0 (branches/release_60 324090) (based on LLVM 6.0.0) WARNING: WITNESS option enabled, expect reduced performance. VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5405 @ 2.00GHz (2000.12-MHz K8-class CPU) Origin="GenuineIntel" Id=0x1067a Family=0x6 Model=0x17 Stepping=10 Features=0xbfebfbff Features2=0xc0ce33d AMD Features=0x20100800 AMD Features2=0x1 VT-x: (disabled in BIOS) HLT,PAUSE TSC: P-state invariant, performance statistics real memory = 8589934592 (8192 MB) avail memory = 8238014464 (7856 MB) Event timer "LAPIC" quality 100 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) random: unblocking device. ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard SMP: AP CPU #2 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #6 Launched! Timecounter "TSC" frequency 2000118732 Hz quality 1000 random: entropy device external interface netmap: loaded module [ath_hal] loaded module_register_init: MOD_LOAD (vesa, 0xffffffff81803590, 0) error 19 kbd1 at kbdmux0 nexus0 vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x71 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 24 at device 1.0 on pci0 pci1: on pcib1 pcib2: irq 26 at device 3.0 on pci0 pci2: on pcib2 pcib3: irq 26 at device 0.0 on pci2 pci3: on pcib3 pcib4: irq 26 at device 0.0 on pci3 pci4: on pcib4 pcib5: at device 0.3 on pci2 pci5: on pcib5 pcib6: irq 28 at device 5.0 on pci0 pci6: on pcib6 pci6: at device 0.0 (no driver attached) pci6: at device 0.1 (no driver attached) pci6: at device 0.2 (no driver attached) pci6: at device 0.3 (no driver attached) pci6: at device 0.4 (no driver attached) pci6: at device 0.5 (no driver attached) pci6: at device 0.6 (no driver attached) pcib7: irq 30 at device 7.0 on pci0 pci7: on pcib7 pci7: at device 0.0 (no driver attached) pci7: at device 0.1 (no driver attached) pci7: at device 0.2 (no driver attached) pci7: at device 0.3 (no driver attached) pci7: at device 0.4 (no driver attached) pci7: at device 0.5 (no driver attached) pci7: at device 0.6 (no driver attached) pcib8: irq 32 at device 9.0 on pci0 pci8: on pcib8 igb0: port 0x2000-0x201f mem 0xdb020000-0xdb03ffff,0xdb000000-0xdb01ffff,0xdb080000-0xdb083fff irq 32 at device 0.0 on pci8 igb0: attach_pre capping queues at 4 igb0: using 1024 tx descriptors and 1024 rx descriptors igb0: msix_init qsets capped at 4 igb0: pxm cpus: 8 queue msgs: 9 admincnt: 1 igb0: using 4 rx queues 4 tx queues igb0: Using MSIX interrupts with 5 vectors igb0: allocated for 4 tx_queues igb0: allocated for 4 rx_queues igb0: Ethernet address: 00:30:48:b9:93:b2 igb0: netmap queues/slots: TX 4/1024, RX 4/1024 igb1: port 0x2020-0x203f mem 0xdb060000-0xdb07ffff,0xdb040000-0xdb05ffff,0xdb084000-0xdb087fff irq 46 at device 0.1 on pci8 igb1: attach_pre capping queues at 4 igb1: using 1024 tx descriptors and 1024 rx descriptors igb1: msix_init qsets capped at 4 igb1: pxm cpus: 8 queue msgs: 9 admincnt: 1 igb1: using 4 rx queues 4 tx queues igb1: Using MSIX interrupts with 5 vectors igb1: allocated for 4 tx_queues igb1: allocated for 4 rx_queues igb1: Ethernet address: 00:30:48:b9:93:b3 igb1: netmap queues/slots: TX 4/1024, RX 4/1024 uhci0: port 0x1800-0x181f irq 20 at device 29.0 on pci0 uhci0: LegSup = 0x3000 usbus0 on uhci0 usbus0: 12Mbps Full Speed USB v1.0 uhci1: port 0x1820-0x183f irq 21 at device 29.1 on pci0 usbus1 on uhci1 usbus1: 12Mbps Full Speed USB v1.0 uhci2: port 0x1840-0x185f irq 22 at device 29.2 on pci0 usbus2 on uhci2 usbus2: 12Mbps Full Speed USB v1.0 uhci3: port 0x1860-0x187f irq 23 at device 29.3 on pci0 usbus3 on uhci3 usbus3: 12Mbps Full Speed USB v1.0 ehci0: mem 0xdb404000-0xdb4043ff irq 23 at device 29.7 on pci0 usbus4: EHCI version 1.0 usbus4 on ehci0 usbus4: 480Mbps High Speed USB v2.0 pcib9: at device 30.0 on pci0 pci9: on pcib9 vgapci0: port 0x3000-0x30ff mem 0xd0000000-0xd7ffffff,0xdb100000-0xdb10ffff irq 18 at device 1.0 on pci9 vgapci0: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1880-0x188f at device 31.1 on pci0 ata0: at channel 0 on atapci0 atapci1: port 0x18c0-0x18c7,0x18b8-0x18bb,0x18b0-0x18b7,0x1894-0x1897,0x18a0-0x18af mem 0xdb404400-0xdb4047ff irq 17 at device 31.2 on pci0 ata2: at channel 0 on atapci1 ata3: at channel 1 on atapci1 acpi_button0: on acpi0 acpi_syscontainer0: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (9600,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 orm0: at iomem 0xc0000-0xcafff,0xcb000-0xcbfff,0xcc000-0xccfff,0xcd000-0xcdfff,0xce000-0xcefff pnpid ORM0000 on isa0 Timecounters tick every 1.000 msec ugen1.1: at usbus1 ugen4.1: at usbus4 ugen3.1: at usbus3 ugen0.1: at usbus0 ugen2.1: at usbus2 uhub0: on usbus1 uhub1: on usbus0 uhub2: on usbus3 uhub3: on usbus2 uhub4: on usbus4 uhub0: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered ada0 at ata2 bus 0 scbus1 target 0 lun 0 ada0: ATA8-ACS SATA 2.x device ada0: Serial Number S24JJ9FC200111 ada0: 150.000MB/s transfers (SATA, UDMA5, PIO 8192bytes) ada0: 238475MB (488397168 512 byte sectors) ada0: quirks=0x1<4K> WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/ada0s1a [rw]... Setting hostuuid: 53d19f64-d663-a017-8922-003048b993b2. Setting hostid: 0x2f4c1e73. uhub4: 8 ports with 8 removable, self powered Starting file system checks: /dev/ada0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/ada0s1a: clean, 7340363 free (27267 frags, 914137 blocks, 0.1% fragmentation) Mounting local filesystems:. ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/perl5/5.24/mach/CORE 32-bit compatibility ldconfig path: /usr/lib32 Setting hostname: munnar. Setting up harvesting: [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED Feeding entropy: . ugen0.2: at usbus0 ukbd0 on uhub1 ukbd0: on usbus0 kbd2 at ukbd0 Link state changed to up igb0: link state changed to UP Starting Network: lo0 igb0 igb1. lo0: flags=8049 metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21 groups: lo igb0: flags=8843 metric 0 mtu 1500 options=e525bb ether 00:30:48:b9:93:b2 inet 10.193.184.94 netmask 0xfffffc00 broadcast 10.193.187.255 nd6 options=29 media: Ethernet autoselect (100baseTX ) status: active igb1: flags=8802 metric 0 mtu 1500 options=e525bb ether 00:30:48:b9:93:b3 nd6 options=29 media: Ethernet autoselect status: no carrier Starting devd. Starting Network: igb1. igb1: flags=8802 metric 0 mtu 1500 options=e525bb ether 00:30:48:b9:93:b3 nd6 options=29 media: Ethernet autoselect status: no carrier uhid0 on uhub1 uhid0: on usbus0 add host 127.0.0.1: gateway lo0 fib 0: route already in table add net default: gateway 10.193.184.5 add host ::1: gateway lo0 fib 0: route already in table add net fe80::: gateway ::1 add net ff02::: gateway ::1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 Creating and/or trimming log files. Starting syslogd. No core dumps found. Clearing /tmp (X related). Recovering vi editor sessions:. Updating motd:. Mounting late filesystems:. Configuring vt: blanktime. Performing sanity check on sshd configuration. Starting sshd. Starting sendmail_submit. Starting sendmail_msp_queue. Starting cron. Starting background file system checks in 60 seconds. Mon Feb 19 19:24:56 IST 2018 lock order reversal: 1st 0xfffffe0001c384c0 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3566 2nd 0xfffff80005b03a00 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:289 stack backtrace: #0 0xffffffff810e19d1 at witness_debugger+0xf1 #1 0xffffffff810e12a6 at witness_checkorder+0xf16 #2 0xffffffff81050bfc at _sx_xlock+0x19c #3 0xffffffff8153b50c at ufsdirhash_acquire+0x4c #4 0xffffffff8153b2ec at ufsdirhash_add+0x1c #5 0xffffffff8153fcbc at ufs_direnter+0xbac #6 0xffffffff8154e921 at ufs_makeinode+0x8c1 #7 0xffffffff81548865 at ufs_create+0x55 #8 0xffffffff8199c788 at VOP_CREATE_APV+0x228 #9 0xffffffff811a2639 at VOP_CREATE+0x59 #10 0xffffffff811a21bc at vn_open_cred+0x32c #11 0xffffffff811a1e7c at vn_open+0x4c #12 0xffffffff81194dd9 at kern_openat+0x2e9 #13 0xffffffff81195239 at sys_openat+0x69 #14 0xffffffff8172a48e at syscallenter+0x4ee #15 0xffffffff81729d3b at amd64_syscall+0x1b #16 0xffffffff816fa4cd at fast_syscall_common+0x101 Feb 19 20:22:11 munnar sshd[828]: error: PAM: Authentication error for root from rgupta-pc.asicdesigners.com ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Feb 19 20:26:23 munnar ctld[873]: failed to apply configuration; exiting t5nex0: mem 0xd9400000-0xd947ffff,0xd8000000-0xd8ffffff,0xd9688000-0xd9689fff irq 28 at device 0.4 on pci6 t5nex0: PCIe gen2 x8, 2 ports, 22 MSI-X interrupts, 55 eq, 21 iq t6nex0: mem 0xd9c00000-0xd9c7ffff,0xda000000-0xdaffffff,0xd9e88000-0xd9e89fff irq 30 at device 0.4 on pci7 t6nex0: PCIe gen2 x8, 2 ports, 22 MSI-X interrupts, 55 eq, 21 iq cxl0: on t5nex0 cxl0: Ethernet address: 00:07:43:28:cd:a0 cxl0: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE) cxl1: on t5nex0 cxl1: Ethernet address: 00:07:43:28:cd:a8 cxl1: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE) cc0: on t6nex0 cc0: Ethernet address: 00:07:43:3e:b5:c0 cc0: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE) cc1: on t6nex0 cc1: Ethernet address: 00:07:43:3e:b5:c8 cc1: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE) cc0: link state changed to UP Feb 19 20:27:12 munnar ctld[1133]: 102.1.1.2 (iqn.1994-05.com.redhat:ad152041b45a): read: Connection reset by peer lock order reversal: 1st 0xfffff80005b8b7c8 ufs (ufs) @ /usr/src/sys/kern/vfs_lookup.c:671 2nd 0xfffffe0001cf0bc0 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:282 3rd 0xfffff8005b5ecb78 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2607 stack backtrace: #0 0xffffffff810e19d1 at witness_debugger+0xf1 #1 0xffffffff810e12a6 at witness_checkorder+0xf16 #2 0xffffffff80ffa8b9 at __lockmgr_args+0xe49 #3 0xffffffff8153467f at _lockmgr_args+0x8f #4 0xffffffff815329e2 at ffs_lock+0xd2 #5 0xffffffff819a1ffb at VOP_LOCK1_APV+0x23b #6 0xffffffff811a4f43 at VOP_LOCK1+0x53 #7 0xffffffff811a2e51 at _vn_lock+0xd1 #8 0xffffffff81184497 at vget+0x137 #9 0xffffffff8116de11 at vfs_hash_get+0xf1 #10 0xffffffff8152bae9 at ffs_vgetf+0x69 #11 0xffffffff81515f8e at flush_pagedep_deps+0x65e #12 0xffffffff81515827 at softdep_sync_buf+0x437 #13 0xffffffff81533f8a at ffs_syncvnode+0x39a #14 0xffffffff814f0ef5 at ffs_truncate+0xda5 #15 0xffffffff8153ff45 at ufs_direnter+0xe35 #16 0xffffffff8154e921 at ufs_makeinode+0x8c1 #17 0xffffffff81548865 at ufs_create+0x55 Feb 19 21:09:03 munnar ctld[1254]: 102.1.1.2 (iqn.1994-05.com.redhat:ad152041b45a): read: Connection reset by peer witness_lock_list_get: witness exhausted ctl_datamove: tag 0x0051 on (0:88:0) aborted (0:4:0/1): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:4:0/1): Tag: 0x0011, type 1 (0:4:0/1): ctl_process_done: 98 seconds (0:102:0/99): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:102:0/99): Tag: 0x002b, type 1 (0:102:0/99): ctl_process_done: 99 seconds (0:81:0/78): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:42:0/39): WRITE(10). CDB: 2a 08 00 10 b9 c8 00 00 08 00 (0:81:0/78): Tag: 0x0041, type 1 (0:42:0/39): Tag: 0x0073, type 1 (0:42:0/39): ctl_process_done: 99 seconds (0:81:0/78): ctl_process_done: 99 seconds (0:33:0/30): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:56:0/53): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:33:0/30): Tag: 0x0073, type 1 (0:33:0/30): ctl_process_done: 108 seconds (0:56:0/53): Tag: 0x0069, type 1 (0:56:0/53): ctl_process_done: 108 seconds (0:25:0/22): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:25:0/22): Tag: 0x0078, type 1 (0:25:0/22): ctl_process_done: 108 seconds (0:97:0/94): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:97:0/94): Tag: 0x005f, type 1 (0:97:0/94): ctl_process_done: 128 seconds (0:6:0/3): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:6:0/3): Tag: 0x004c, type 1 (0:6:0/3): ctl_process_done: 129 seconds (0:80:0/77): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:80:0/77): Tag: 0x0067, type 1 (0:80:0/77): ctl_process_done: 130 seconds (0:24:0/21): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:24:0/21): Tag: 0x0075, type 1 (0:24:0/21): ctl_process_done: 131 seconds (0:15:0/12): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:18:0/15): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:15:0/12): Tag: 0x001b, type 1 (0:18:0/15): Tag: 0x0011, type 1 (0:18:0/15): ctl_process_done: 131 seconds (0:15:0/12): ctl_process_done: 132 seconds (0:29:0/26): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:29:0/26): Tag: 0x0030, type 1 (0:29:0/26): ctl_process_done: 133 seconds (0:40:0/37): WRITE(10). CDB: 2a 08 00 10 ba 50 00 00 08 00 (0:40:0/37): Tag: 0x0033, type 1 (0:71:0/68): WRITE(10). CDB: 2a 08 00 10 ad 00 00 00 08 00 (0:40:0/37): ctl_process_done: 140 seconds (0:28:0/25): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:71:0/68): Tag: 0x0007, type 1 (0:71:0/68): ctl_process_done: 140 seconds (0:28:0/25): Tag: 0x0002, type 1 (0:28:0/25): ctl_process_done: 138 seconds (0:67:0/64): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:67:0/64): Tag: 0x001d, type 1 (0:67:0/64): ctl_process_done: 141 seconds (0:3:0/0): WRITE(10). CDB: 2a 08 00 10 b2 e8 00 00 08 00 (0:38:0/35): WRITE(10). CDB: 2a 08 00 10 b2 b0 00 00 08 00 (0:3:0/0): Tag: 0x002c, type 1 (0:3:0/0): ctl_process_done: 131 seconds (0:45:0/42): WRITE(10). CDB: 2a 08 00 10 ac c8 00 00 08 00 (0:38:0/35): Tag: 0x0029, type 1 (0:45:0/42): Tag: 0x0019, type 1 (0:38:0/35): ctl_process_done: 113 seconds (0:45:0/42): ctl_process_done: 119 seconds (0:43:0/40): WRITE(10). CDB: 2a 08 00 10 b3 98 00 00 08 00 (0:43:0/40): Tag: 0x004b, type 1 (0:43:0/40): ctl_process_done: 118 seconds (0:91:0/88): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:91:0/88): Tag: 0x006f, type 1 (0:91:0/88): ctl_process_done: 114 seconds (0:35:0/32): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:35:0/32): Tag: 0x005e, type 1 (0:35:0/32): ctl_process_done: 114 seconds (0:17:0/14): WRITE(10). CDB: 2a 08 00 10 b5 50 00 00 08 00 (0:17:0/14): Tag: 0x0060, type 1 (0:17:0/14): ctl_process_done: 114 seconds (0:55:0/52): WRITE(10). CDB: 2a 08 00 10 b5 38 00 00 08 00 (0:55:0/52): Tag: 0x002d, type 1 (0:55:0/52): ctl_process_done: 104 seconds (0:101:0/98): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:22:0/19): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:101:0/98): Tag: 0x0002, type 1 (0:101:0/98): ctl_process_done: 113 seconds (0:22:0/19): Tag: 0x006c, type 1 (0:22:0/19): ctl_process_done: 113 seconds (0:36:0/33): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:53:0/50): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:36:0/33): Tag: 0x0050, type 1 (0:53:0/50): Tag: 0x0014, type 1 (0:36:0/33): ctl_process_done: 113 seconds (0:53:0/50): ctl_process_done: 113 seconds (0:84:0/81): WRITE(10). CDB: 2a 08 00 10 b7 b0 00 00 08 00 (0:84:0/81): Tag: 0x002f, type 1 (0:84:0/81): ctl_process_done: 108 seconds (0:58:0/55): WRITE(10). CDB: 2a 08 00 10 b5 30 00 00 08 00 (0:58:0/55): Tag: 0x0041, type 1 (0:58:0/55): ctl_process_done: 108 seconds (0:64:0/61): WRITE(10). CDB: 2a 08 00 10 b2 20 00 00 08 00 (0:64:0/61): Tag: 0x0009, type 1 (0:64:0/61): ctl_process_done: 109 seconds (0:9:0/6): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:9:0/6): Tag: 0x007c, type 1 (0:9:0/6): ctl_process_done: 109 seconds (0:49:0/46): WRITE(10). CDB: 2a 08 00 10 b7 88 00 00 08 00 (0:54:0/51): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:49:0/46): Tag: 0x0032, type 1 (0:54:0/51): Tag: 0x0024, type 1 (0:49:0/46): ctl_process_done: 109 seconds (0:54:0/51): ctl_process_done: 106 seconds (0:23:0/20): WRITE(10). CDB: 2a 08 00 10 b8 d8 00 00 08 00 (0:23:0/20): Tag: 0x002f, type 1 (0:23:0/20): ctl_process_done: 104 seconds (0:76:0/73): WRITE(10). CDB: 2a 08 00 10 ad d0 00 00 08 00 (0:93:0/90): WRITE(10). CDB: 2a 08 00 10 b7 20 00 00 08 00 (0:76:0/73): Tag: 0x001e, type 1 (0:76:0/73): ctl_process_done: 107 seconds (0:88:0/85): WRITE(10). CDB: 2a 08 00 10 b5 58 00 00 08 00 (0:93:0/90): Tag: 0x0031, type 1 (0:88:0/85): Tag: 0x004a, type 1 (0:88:0/85): ctl_process_done: 95 seconds (0:72:0/69): WRITE(10). CDB: 2a 08 00 10 b5 b8 00 00 08 00 (0:93:0/90): ctl_process_done: 107 seconds (0:72:0/69): Tag: 0x0055, type 1 (0:27:0/24): WRITE(10). CDB: 2a 08 00 10 b2 e8 00 00 08 00 (0:72:0/69): ctl_process_done: 116 seconds (0:27:0/24): Tag: 0x001b, type 1 (0:74:0/71): WRITE(10). CDB: 2a 08 00 10 a9 80 00 00 08 00 (0:27:0/24): ctl_process_done: 116 seconds (0:74:0/71): Tag: 0x0041, type 1 (0:74:0/71): ctl_process_done: 96 seconds (0:7:0/4): WRITE(10). CDB: 2a 08 00 10 a1 a0 00 00 08 00 (0:7:0/4): Tag: 0x0059, type 1 (0:7:0/4): ctl_process_done: 96 seconds (0:31:0/28): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:31:0/28): Tag: 0x0040, type 1 (0:31:0/28): ctl_process_done: 95 seconds (0:65:0/62): WRITE(10). CDB: 2a 08 00 10 b7 38 00 00 08 00 (0:65:0/62): Tag: 0x0070, type 1 (0:65:0/62): ctl_process_done: 94 seconds (0:42:0/39): WRITE(10). CDB: 2a 08 00 10 b9 c8 00 00 08 00 (0:42:0/39): Tag: 0x0027, type 1 (0:42:0/39): ctl_process_done: 118 seconds (0:42:0/39): WRITE(10). CDB: 2a 08 00 10 b9 f8 00 00 08 00 (0:39:0/36): WRITE(10). CDB: 2a 08 00 10 b8 88 00 00 08 00 (0:42:0/39): Tag: 0x0002, type 1 (0:39:0/36): Tag: 0x007f, type 1 (0:42:0/39): ctl_process_done: 94 seconds (0:39:0/36): ctl_process_done: 94 seconds (0:27:0/24): WRITE(10). CDB: 2a 08 00 10 b3 18 00 00 08 00 (0:27:0/24): Tag: 0x0046, type 1 (0:27:0/24): ctl_process_done: 94 seconds (0:83:0/80): WRITE(10). CDB: 2a 08 00 10 b5 98 00 00 08 00 (0:54:0/51): WRITE(10). CDB: 2a 08 00 10 b9 18 00 00 08 00 (0:83:0/80): Tag: 0x002c, type 1 (0:54:0/51): Tag: 0x0053, type 1 (0:83:0/80): ctl_process_done: 104 seconds (0:54:0/51): ctl_process_done: 103 seconds (0:96:0/93): WRITE(10). CDB: 2a 08 00 10 a7 78 00 00 08 00 (0:96:0/93): Tag: 0x0047, type 1 (0:96:0/93): ctl_process_done: 103 seconds (0:68:0/65): WRITE(10). CDB: 2a 08 00 10 ae 50 00 00 08 00 (0:68:0/65): Tag: 0x003a, type 1 (0:68:0/65): ctl_process_done: 106 seconds ctl_datamove: tag 0x001a on (0:21:0) aborted (0:98:0/95): WRITE(10). CDB: 2a 08 00 10 b5 20 00 00 08 00 (0:98:0/95): Tag: 0x0060, type 1 (0:98:0/95): ctl_process_done: 114 seconds (0:73:0/70): WRITE(10). CDB: 2a 00 00 00 90 00 00 08 00 00 (0:73:0/70): Tag: 0x0031, type 1 (0:73:0/70): ctl_process_done: 104 seconds (0:48:0/45): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:48:0/45): Tag: 0x0034, type 1 (0:48:0/45): ctl_process_done: 94 seconds (0:62:0/59): WRITE(10). CDB: 2a 00 00 00 98 00 00 08 00 00 (0:62:0/59): Tag: 0x002f, type 1 (0:62:0/59): ctl_process_done: 102 seconds (0:91:0/88): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:91:0/88): Tag: 0x0025, type 1 (0:91:0/88): ctl_process_done: 107 seconds (0:62:0/59): WRITE(10). CDB: 2a 00 00 00 70 00 00 08 00 00 (0:62:0/59): Tag: 0x0076, type 1 (0:62:0/59): ctl_process_done: 106 seconds (0:23:0/20): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:23:0/20): Tag: 0x0039, type 1 (0:23:0/20): ctl_process_done: 117 seconds (0:20:0/17): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:57:0/54): WRITE(10). CDB: 2a 00 00 00 78 00 00 08 00 00 (0:20:0/17): Tag: 0x0071, type 1 (0:57:0/54): Tag: 0x0015, type 1 (0:20:0/17): ctl_process_done: 91 seconds (0:57:0/54): ctl_process_done: 142 seconds (0:78:0/75): WRITE(10). CDB: 2a 00 00 00 c0 00 00 08 00 00 (0:78:0/75): Tag: 0x0043, type 1 (0:78:0/75): ctl_process_done: 136 seconds (0:97:0/94): WRITE(10). CDB: 2a 08 00 10 d6 48 00 00 08 00 (0:97:0/94): Tag: 0x001f, type 1 (0:97:0/94): ctl_process_done: 91 seconds (0:18:0/15): WRITE(10). CDB: 2a 00 00 00 f8 00 00 08 00 00 (0:18:0/15): Tag: 0x0035, type 1 (0:18:0/15): ctl_process_done: 92 seconds (0:73:0/70): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:73:0/70): Tag: 0x007d, type 1 (0:73:0/70): ctl_process_done: 139 seconds (0:16:0/13): WRITE(10). CDB: 2a 00 00 00 d0 00 00 08 00 00 (0:16:0/13): Tag: 0x0061, type 1 (0:16:0/13): ctl_process_done: 97 seconds (0:57:0/54): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:57:0/54): Tag: 0x000a, type 1 (0:57:0/54): ctl_process_done: 134 seconds (0:16:0/13): WRITE(10). CDB: 2a 00 00 00 d8 00 00 08 00 00 (0:16:0/13): Tag: 0x005a, type 1 (0:16:0/13): ctl_process_done: 107 seconds (0:18:0/15): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:18:0/15): Tag: 0x0039, type 1 (0:18:0/15): ctl_process_done: 95 seconds (0:77:0/74): WRITE(10). CDB: 2a 00 00 00 78 00 00 08 00 00 (0:77:0/74): Tag: 0x0018, type 1 (0:77:0/74): ctl_process_done: 91 seconds (0:58:0/55): WRITE(10). CDB: 2a 08 00 10 d2 58 00 00 08 00 (0:58:0/55): Tag: 0x0020, type 1 (0:58:0/55): ctl_process_done: 122 seconds (0:51:0/48): WRITE(10). CDB: 2a 08 00 10 ce 90 00 00 08 00 (0:51:0/48): Tag: 0x0042, type 1 (0:51:0/48): ctl_process_done: 104 seconds (0:95:0/92): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:95:0/92): Tag: 0x0034, type 1 (0:95:0/92): ctl_process_done: 98 seconds (0:65:0/62): WRITE(10). CDB: 2a 08 00 10 d4 f0 00 00 08 00 (0:65:0/62): Tag: 0x0014, type 1 (0:65:0/62): ctl_process_done: 172 seconds (0:65:0/62): WRITE(10). CDB: 2a 08 00 10 d4 f0 00 00 08 00 (0:65:0/62): Tag: 0x002f, type 1 (0:65:0/62): ctl_process_done: 104 seconds (0:58:0/55): WRITE(10). CDB: 2a 08 00 10 d2 58 00 00 08 00 (0:58:0/55): Tag: 0x006a, type 1 (0:58:0/55): ctl_process_done: 91 seconds (0:43:0/40): WRITE(10). CDB: 2a 00 00 00 78 00 00 08 00 00 (0:43:0/40): Tag: 0x001c, type 1 (0:43:0/40): ctl_process_done: 107 seconds (0:56:0/53): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:56:0/53): Tag: 0x0028, type 1 (0:56:0/53): ctl_process_done: 91 seconds ctl_datamove: tag 0x0036 on (0:46:0) aborted (0:56:0/53): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:56:0/53): Tag: 0x0052, type 1 (0:56:0/53): ctl_process_done: 91 seconds (0:76:0/73): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:76:0/73): Tag: 0x0016, type 1 (0:76:0/73): ctl_process_done: 91 seconds (0:52:0/49): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:52:0/49): Tag: 0x0011, type 1 (0:52:0/49): ctl_process_done: 94 seconds (0:64:0/61): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:64:0/61): Tag: 0x001a, type 1 (0:64:0/61): ctl_process_done: 94 seconds (0:10:0/7): WRITE(10). CDB: 2a 08 00 10 d6 c8 00 00 08 00 (0:10:0/7): Tag: 0x0052, type 1 (0:10:0/7): ctl_process_done: 120 seconds (0:87:0/84): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:87:0/84): Tag: 0x0070, type 1 (0:60:0/57): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:87:0/84): ctl_process_done: 96 seconds (0:60:0/57): Tag: 0x0016, type 1 (0:60:0/57): ctl_process_done: 97 seconds (0:5:0/2): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:5:0/2): Tag: 0x0010, type 1 (0:5:0/2): ctl_process_done: 98 seconds (0:85:0/82): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:85:0/82): Tag: 0x0032, type 1 (0:85:0/82): ctl_process_done: 103 seconds (0:38:0/35): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:7:0/4): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:7:0/4): WRITE(10). CDB: 2a 00 00 00 29 90 00 00 20 00 (0:38:0/35): Tag: 0x006d, type 1 (0:7:0/4): Tag: 0x0066, type 1 (0:7:0/4): Tag: 0x0010, type 1 (0:38:0/35): ctl_process_done: 101 seconds (0:6:0/3): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:7:0/4): ctl_datamove: 101 seconds (0:6:0/3): Tag: 0x0041, type 1 (0:7:0/4): ctl_process_done: 101 seconds (0:43:0/40): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 ctl_datamove: tag 0x0066 on (0:7:0) aborted (0:43:0/40): Tag: 0x002c, type 1 (0:6:0/3): ctl_process_done: 96 seconds (0:12:0/9): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:12:0/9): Tag: 0x0047, type 1 (0:43:0/40): ctl_process_done: 103 seconds (0:39:0/36): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:12:0/9): ctl_process_done: 103 seconds (0:92:0/89): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:39:0/36): Tag: 0x000a, type 1 (0:39:0/36): ctl_process_done: 104 seconds (0:92:0/89): Tag: 0x0019, type 1 (0:41:0/38): WRITE(10). CDB: 2a 08 00 10 cf 78 00 00 08 00 (0:92:0/89): ctl_process_done: 97 seconds (0:41:0/38): Tag: 0x001a, type 1 (0:41:0/38): ctl_process_done: 136 seconds (0:13:0/10): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:13:0/10): Tag: 0x0047, type 1 (0:13:0/10): ctl_process_done: 104 seconds (0:7:0/4): WRITE(10). CDB: 2a 00 00 00 29 90 00 00 20 00 (0:7:0/4): Tag: 0x0066, type 1 (0:7:0/4): ctl_process_done: 102 seconds (0:47:0/44): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:47:0/44): Tag: 0x0005, type 1 (0:47:0/44): ctl_process_done: 97 seconds (0:68:0/65): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:68:0/65): Tag: 0x003f, type 1 (0:68:0/65): ctl_process_done: 98 seconds (0:77:0/74): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:77:0/74): WRITE(10). CDB: 2a 00 00 00 9a 00 00 00 20 00 (0:77:0/74): Tag: 0x0050, type 1 (0:77:0/74): ctl_process_done: 112 seconds (0:77:0/74): Tag: 0x0075, type 1 (0:77:0/74): ctl_datamove: 112 seconds ctl_datamove: tag 0x0075 on (0:77:0) aborted (0:77:0/74): WRITE(10). CDB: 2a 00 00 00 9a 00 00 00 20 00 (0:77:0/74): Tag: 0x0075, type 1 (0:77:0/74): ctl_process_done: 112 seconds (0:40:0/37): WRITE(10). CDB: 2a 08 00 10 d5 a8 00 00 08 00 (0:40:0/37): Tag: 0x0008, type 1 (0:40:0/37): ctl_process_done: 145 seconds (0:90:0/87): WRITE(10). CDB: 2a 08 00 10 c9 a8 00 00 08 00 (0:90:0/87): Tag: 0x003a, type 1 (0:90:0/87): ctl_process_done: 145 seconds (0:75:0/72): WRITE(10). CDB: 2a 08 00 10 d2 90 00 00 08 00 (0:75:0/72): Tag: 0x004c, type 1 (0:75:0/72): ctl_process_done: 146 seconds (0:22:0/19): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:22:0/19): WRITE(10). CDB: 2a 00 00 00 4a d8 00 00 20 00 (0:22:0/19): Tag: 0x0072, type 1 (0:22:0/19): ctl_process_done: 105 seconds (0:22:0/19): Tag: 0x007c, type 1 (0:22:0/19): ctl_datamove: 105 seconds ctl_datamove: tag 0x007c on (0:22:0) aborted (0:22:0/19): WRITE(10). CDB: 2a 00 00 00 4a d8 00 00 20 00 (0:22:0/19): Tag: 0x007c, type 1 (0:22:0/19): ctl_process_done: 105 seconds (0:4:0/1): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:4:0/1): WRITE(10). CDB: 2a 00 00 00 91 60 00 00 20 00 (0:4:0/1): Tag: 0x0072, type 1 (0:4:0/1): Tag: 0x002d, type 1 (0:4:0/1): ctl_process_done: 114 seconds (0:4:0/1): ctl_datamove: 114 seconds ctl_datamove: tag 0x002d on (0:4:0) aborted (0:4:0/1): WRITE(10). CDB: 2a 00 00 00 91 60 00 00 20 00 (0:4:0/1): Tag: 0x002d, type 1 (0:4:0/1): ctl_process_done: 114 seconds (0:32:0/29): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:32:0/29): Tag: 0x0002, type 1 (0:32:0/29): ctl_process_done: 106 seconds (0:21:0/18): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:21:0/18): Tag: 0x0062, type 1 (0:21:0/18): ctl_process_done: 112 seconds (0:62:0/59): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:83:0/80): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:62:0/59): Tag: 0x007a, type 1 (0:83:0/80): Tag: 0x0016, type 1 (0:83:0/80): ctl_process_done: 120 seconds (0:78:0/75): WRITE(10). CDB: 2a 00 00 00 26 70 00 00 20 00 (0:45:0/42): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:34:0/31): WRITE(10). CDB: 2a 00 00 00 26 28 00 00 20 00 (0:100:0/97): WRITE(10). CDB: 2a 00 00 00 25 18 00 00 20 00 (0:62:0/59): ctl_process_done: 116 seconds (0:100:0/97): Tag: 0x003b, type 1 (0:73:0/70): WRITE(10). CDB: 2a 00 00 00 57 30 00 00 20 00 (0:78:0/75): Tag: 0x0036, type 1 (0:72:0/69): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:45:0/42): Tag: 0x0074, type 1 (0:34:0/31): Tag: 0x0064, type 1 (0:72:0/69): Tag: 0x0067, type 1 (0:100:0/97): ctl_datamove: 112 seconds (0:78:0/75): ctl_datamove: 112 seconds (0:45:0/42): ctl_process_done: 117 seconds (0:34:0/31): ctl_datamove: 112 seconds (0:72:0/69): ctl_process_done: 121 seconds (0:73:0/70): Tag: 0x004a, type 1 (0:33:0/30): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 ctl_datamove: tag 0x0036 on (0:78:0) aborted ctl_datamove: tag 0x003b on (0:100:0) aborted (0:54:0/51): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 ctl_datamove: tag 0x0064 on (0:34:0) aborted (0:33:0/30): Tag: 0x0079, type 1 (0:73:0/70): ctl_datamove: 111 seconds (0:54:0/51): Tag: 0x0023, type 1 ctl_datamove: tag 0x004a on (0:73:0) aborted (0:33:0/30): ctl_process_done: 115 seconds (0:73:0/70): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:54:0/51): ctl_process_done: 113 seconds (0:73:0/70): Tag: 0x000c, type 1 (0:73:0/70): ctl_process_done: 112 seconds (0:42:0/39): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:23:0/20): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:42:0/39): Tag: 0x0066, type 1 (0:42:0/39): ctl_process_done: 123 seconds ctl_datamove: tag 0x0052 on (0:14:0) aborted ctl_datamove: tag 0x000e on (0:91:0) aborted (0:23:0/20): Tag: 0x0042, type 1 (0:23:0/20): ctl_process_done: 110 seconds (0:31:0/28): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:78:0/75): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:78:0/75): Tag: 0x0011, type 1 (0:31:0/28): Tag: 0x006f, type 1 (0:31:0/28): ctl_process_done: 105 seconds (0:78:0/75): ctl_process_done: 114 seconds (0:34:0/31): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:55:0/52): WRITE(10). CDB: 2a 08 00 10 d2 d0 00 00 08 00 (0:34:0/31): Tag: 0x001e, type 1 (0:34:0/31): ctl_process_done: 114 seconds (0:100:0/97): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:55:0/52): Tag: 0x0013, type 1 (0:55:0/52): ctl_process_done: 105 seconds (0:100:0/97): Tag: 0x0050, type 1 (0:100:0/97): ctl_process_done: 114 seconds (0:82:0/79): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:37:0/34): WRITE(10). CDB: 2a 08 00 10 d7 08 00 00 08 00 (0:82:0/79): Tag: 0x0023, type 1 (0:37:0/34): Tag: 0x005f, type 1 (0:82:0/79): ctl_process_done: 107 seconds (0:80:0/77): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:37:0/34): ctl_process_done: 103 seconds (0:49:0/46): WRITE(10). CDB: 2a 08 00 10 d6 78 00 00 08 00 (0:80:0/77): Tag: 0x0011, type 1 (0:80:0/77): ctl_process_done: 107 seconds (0:74:0/71): WRITE(10). CDB: 2a 08 00 10 c6 48 00 00 08 00 (0:49:0/46): Tag: 0x003c, type 1 (0:74:0/71): Tag: 0x0051, type 1 (0:49:0/46): ctl_process_done: 99 seconds (0:74:0/71): ctl_process_done: 105 seconds (0:71:0/68): WRITE(10). CDB: 2a 08 00 10 ca 18 00 00 08 00 (0:48:0/45): WRITE(10). CDB: 2a 08 00 10 c9 18 00 00 08 00 (0:71:0/68): Tag: 0x0005, type 1 (0:48:0/45): Tag: 0x007f, type 1 (0:48:0/45): ctl_process_done: 105 seconds (0:28:0/25): WRITE(10). CDB: 2a 08 00 10 cf 90 00 00 08 00 (0:71:0/68): ctl_process_done: 92 seconds (0:28:0/25): Tag: 0x0052, type 1 (0:73:0/70): WRITE(10). CDB: 2a 00 00 00 57 30 00 00 20 00 (0:28:0/25): ctl_process_done: 105 seconds (0:70:0/67): WRITE(10). CDB: 2a 08 00 10 cf 20 00 00 08 00 (0:73:0/70): Tag: 0x004a, type 1 (0:73:0/70): ctl_process_done: 115 seconds (0:53:0/50): WRITE(10). CDB: 2a 08 00 10 c4 08 00 00 08 00 (0:70:0/67): Tag: 0x0001, type 1 ctl_datamove: tag 0x007a on (0:102:0) aborted (0:53:0/50): Tag: 0x001e, type 1 (0:70:0/67): ctl_process_done: 102 seconds (0:53:0/50): ctl_process_done: 93 seconds (0:29:0/26): WRITE(10). CDB: 2a 08 00 10 10 00 00 00 08 00 (0:50:0/47): WRITE(10). CDB: 2a 08 00 10 ce 30 00 00 08 00 (0:29:0/26): Tag: 0x004b, type 1 (0:29:0/26): ctl_process_done: 97 seconds (0:50:0/47): Tag: 0x0010, type 1 (0:50:0/47): ctl_process_done: 102 seconds (0:100:0/97): WRITE(10). CDB: 2a 00 00 00 25 18 00 00 20 00 (0:100:0/97): Tag: 0x003b, type 1 (0:100:0/97): ctl_process_done: 116 seconds (0:78:0/75): WRITE(10). CDB: 2a 00 00 00 26 70 00 00 20 00 (0:78:0/75): Tag: 0x0036, type 1 (0:78:0/75): ctl_process_done: 117 seconds (0:34:0/31): WRITE(10). CDB: 2a 00 00 00 26 28 00 00 20 00 (0:34:0/31): Tag: 0x0064, type 1 (0:34:0/31): ctl_process_done: 117 seconds (0:20:0/17): WRITE(10). CDB: 2a 08 00 10 ce f8 00 00 08 00 (0:20:0/17): Tag: 0x0010, type 1 (0:20:0/17): ctl_process_done: 94 seconds (0:14:0/11): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:14:0/11): Tag: 0x0040, type 1 (0:14:0/11): ctl_process_done: 91 seconds (0:14:0/11): WRITE(10). CDB: 2a 00 00 00 9b 60 00 00 20 00 (0:14:0/11): Tag: 0x0052, type 1 (0:14:0/11): ctl_process_done: 91 seconds (0:102:0/99): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 (0:102:0/99): Tag: 0x0053, type 1 (0:102:0/99): ctl_process_done: 91 seconds (0:102:0/99): WRITE(10). CDB: 2a 00 00 00 7c 40 00 00 20 00 (0:102:0/99): Tag: 0x007a, type 1 (0:102:0/99): ctl_process_done: 91 seconds (0:10:0/7): WRITE(10). CDB: 2a 08 00 10 d6 c8 00 00 08 00 (0:10:0/7): Tag: 0x002b, type 1 (0:10:0/7): ctl_process_done: 97 seconds panic: Memory modified after free 0xfffff80003ba8b80(32) val=0 @ 0xfffff80003ba8b80 cpuid = 5 time = 1519077207 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2c/frame 0xfffffe006210ee80 kdb_backtrace() at kdb_backtrace+0x53/frame 0xfffffe006210ef50 vpanic() at vpanic+0x268/frame 0xfffffe006210f020 kproc_shutdown() at kproc_shutdown/frame 0xfffffe006210f080 trash_ctor() at trash_ctor+0x6b/frame 0xfffffe006210f0c0 uma_zalloc_arg() at uma_zalloc_arg+0x284/frame 0xfffffe006210f200 bucket_alloc() at bucket_alloc+0x102/frame 0xfffffe006210f240 uma_zfree_arg() at uma_zfree_arg+0x647/frame 0xfffffe006210f330 uma_zfree() at uma_zfree+0x29/frame 0xfffffe006210f360 ctl_free_beio() at ctl_free_beio+0x80/frame 0xfffffe006210f390 ctl_complete_beio() at ctl_complete_beio+0x48/frame 0xfffffe006210f3b0 ctl_be_block_move_done() at ctl_be_block_move_done+0x266/frame 0xfffffe006210f430 cfiscsi_datamove_in() at cfiscsi_datamove_in+0x92d/frame 0xfffffe006210f510 cfiscsi_datamove() at cfiscsi_datamove+0x28/frame 0xfffffe006210f530 ctl_datamove() at ctl_datamove+0x498/frame 0xfffffe006210f7d0 ctl_be_block_dispatch_zvol() at ctl_be_block_dispatch_zvol+0x6c2/frame 0xfffffe006210f8e0 ctl_be_block_dispatch() at ctl_be_block_dispatch+0x626/frame 0xfffffe006210f9b0 ctl_be_block_worker() at ctl_be_block_worker+0x68d/frame 0xfffffe006210fa40 taskqueue_run_locked() at taskqueue_run_locked+0x246/frame 0xfffffe006210fab0 taskqueue_thread_loop() at taskqueue_thread_loop+0xc2/frame 0xfffffe006210fb30 fork_exit() at fork_exit+0x145/frame 0xfffffe006210fbb0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe006210fbb0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic ------------------------------------------------------------------------ kernel config options CONFIG_AUTOGENERATED ident MYKERNEL machine amd64 cpu HAMMER makeoptions WITH_CTF=1 makeoptions DEBUG=-g -O0 options KSTACK_PAGES=16 options KTR_ENTRIES=165536 options KTR_MASK=KTR_SPARE3 options KTR_COMPILE=KTR_SPARE3 options KTR options BPF_JITTER options XENHVM options USB_DEBUG options ATH_ENABLE_11N options AH_AR5416_INTERRUPT_MITIGATION options AH_SUPPORT_AR5416 options IEEE80211_SUPPORT_MESH options IEEE80211_AMPDU_AGE options IEEE80211_DEBUG options IXL_IW options SC_PIXEL_MODE options VESA options PCI_IOV options PCI_HP options ACPI_DMAR options EARLY_AP_STARTUP options SMP options MALLOC_DEBUG_MAXZONES=8 options WITNESS_SKIPSPIN options WITNESS options INVARIANT_SUPPORT options INVARIANTS options DEADLKRES options GDB options FULL_BUF_TRACKING options DDB options BUF_TRACKING options KDB_TRACE options KDB options RCTL options RACCT_DEFAULT_TO_DISABLED options RACCT options INCLUDE_CONFIG_FILE options DDB_CTF options KDTRACE_HOOKS options KDTRACE_FRAME options MAC options CAPABILITIES options CAPABILITY_MODE options AUDIT options HWPMC_HOOKS options KBD_INSTALL_CDEV options PRINTF_BUFR_SIZE=128 options _KPOSIX_PRIORITY_SCHEDULING options SYSVSEM options SYSVMSG options SYSVSHM options STACK options KTRACE options SCSI_DELAY=5000 options COMPAT_FREEBSD11 options COMPAT_FREEBSD10 options COMPAT_FREEBSD9 options COMPAT_FREEBSD7 options COMPAT_FREEBSD6 options COMPAT_FREEBSD5 options COMPAT_FREEBSD4 options COMPAT_FREEBSD32 options GEOM_LABEL options GEOM_RAID options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options MSDOSFS options NFS_ROOT options NFSLOCKD options NFSD options NFSCL options MD_ROOT options QUOTA options UFS_GJOURNAL options UFS_DIRHASH options UFS_ACL options SOFTUPDATES options FFS options SCTP options TCP_HHOOK options TCP_OFFLOAD options IPSEC_SUPPORT options IPSEC options INET6 options INET options PREEMPTION options SCHED_ULE options NEW_PCIB options GEOM_PART_MBR options GEOM_PART_EBR_COMPAT options GEOM_PART_EBR options GEOM_PART_BSD device isa device mem device io device uart_ns8250 device cpufreq device acpi device pci device fdc device ahci device ata device mvs device siis device ahc device ahd device esp device hptiop device isp device mpt device mps device mpr device sym device trm device adv device adw device aic device bt device isci device scbus device ch device da device sa device cd device pass device ses device amr device arcmsr device ciss device dpt device hptmv device hptnr device hptrr device hpt27xx device iir device ips device mly device twa device tws device aac device aacp device aacraid device ida device mfi device mlx device mrsas device pmspcv device twe device nvme device nvd device atkbdc device atkbd device psm device kbdmux device vga device splash device sc device vt device vt_vga device vt_efifb device agp device cbb device pccard device cardbus device uart device ppc device ppbus device lpt device ppi device puc device bxe device de device em device ix device ixv device ixl device ixlv device le device ti device txp device vx device miibus device ae device age device alc device ale device bce device bfe device bge device cas device dc device et device fxp device gem device hme device jme device lge device msk device nfe device nge device pcn device re device rl device sf device sge device sis device sk device ste device stge device tl device tx device vge device vr device wb device xl device wlan device wlan_wep device wlan_ccmp device wlan_tkip device wlan_amrr device an device ath device ath_pci device ath_hal device ath_rate_sample device ipw device iwi device iwn device malo device mwl device ral device wi device wpi device loop device random device padlock_rng device rdrand_rng device ether device vlan device tun device md device gif device firmware device bpf device uhci device ohci device ehci device xhci device usb device ukbd device umass device sound device snd_cmi device snd_csa device snd_emu10kx device snd_es137x device snd_hda device snd_ich device snd_via8233 device mmc device mmcsd device sdhci device virtio device virtio_pci device vtnet device virtio_blk device virtio_scsi device virtio_balloon device hyperv device xenpci device vmx device netmap device crypto ------------------------------------------------------------------------ ddb capture buffer