Attachment 196354 Details for Bug 230752 – Text dump of newnfs_request -> witness panic

Text dump of newnfs_request -> witness panic

core-with-nfsserver-1.txt (text/plain), 88.70 KB, created by Dimitry Andric on 2018-08-19 17:22:13 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Dimitry Andric

Created: 2018-08-19 17:22:13 UTC

Size: 88.70 KB

patch

obsolete

>vdim-fbsdhead-amd64 dumped core - see /var/crash/vmcore.2
>
>Sun Aug 19 16:08:06 CEST 2018
>
>FreeBSD vdim-fbsdhead-amd64 12.0-ALPHA2 FreeBSD 12.0-ALPHA2 #0 r338015: Sat Aug 18 17:36:21 CEST 2018     dim@vdim-fbsdhead-amd64:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64
>
>panic: excl->share
>
>GNU gdb (GDB) 8.1.1 [GDB v8.1.1 for FreeBSD]
>Copyright (C) 2018 Free Software Foundation, Inc.
>License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
>This is free software: you are free to change and redistribute it.
>There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>and "show warranty" for details.
>This GDB was configured as "x86_64-portbld-freebsd12.0".
>Type "show configuration" for configuration details.
>For bug reporting instructions, please see:
><http://www.gnu.org/software/gdb/bugs/>.
>Find the GDB manual and other documentation resources online at:
><http://www.gnu.org/software/gdb/documentation/>.
>For help, type "help".
>Type "apropos word" to search for commands related to "word"...
>Reading symbols from /boot/kernel/kernel...Reading symbols from /usr/lib/debug//boot/kernel/kernel.debug...done.
>done.
>
>Unread portion of the kernel message buffer:
>shared lock of (lockmgr) ufs @ /usr/src/sys/kern/vfs_lookup.c:671
>while exclusively locked from /usr/src/sys/kern/vfs_subr.c:2590
>panic: excl->share
>cpuid = 2
>time = 1534686985
>KDB: stack backtrace:
>db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0000583090
>vpanic() at vpanic+0x1b4/frame 0xfffffe00005830f0
>doadump() at doadump/frame 0xfffffe0000583170
>witness_checkorder() at witness_checkorder+0x3a8/frame 0xfffffe0000583200
>lockmgr_slock_hard() at lockmgr_slock_hard+0x6e/frame 0xfffffe0000583290
>__lockmgr_args() at __lockmgr_args+0x8db/frame 0xfffffe0000583330
>ffs_lock() at ffs_lock+0xa5/frame 0xfffffe0000583380
>VOP_LOCK1_APV() at VOP_LOCK1_APV+0xd9/frame 0xfffffe00005833b0
>_vn_lock() at _vn_lock+0x67/frame 0xfffffe0000583420
>lookup() at lookup+0x116/frame 0xfffffe00005834d0
>namei() at namei+0x4dd/frame 0xfffffe0000583590
>unp_connectat() at unp_connectat+0x155/frame 0xfffffe0000583820
>soconnectat() at soconnectat+0xe8/frame 0xfffffe0000583870
>clnt_vc_create() at clnt_vc_create+0x349/frame 0xfffffe00005839b0
>clnt_reconnect_call() at clnt_reconnect_call+0x239/frame 0xfffffe0000583a60
>newnfs_request() at newnfs_request+0x8fc/frame 0xfffffe0000583bd0
>nfsrv_getuser() at nfsrv_getuser+0x129/frame 0xfffffe0000583d70
>nfsrv_getgrpscred() at nfsrv_getgrpscred+0xca/frame 0xfffffe0000583dc0
>nfsd_excred() at nfsd_excred+0x10e/frame 0xfffffe0000583de0
>nfsrvd_dorpc() at nfsrvd_dorpc+0xd84/frame 0xfffffe0000583fc0
>nfssvc_program() at nfssvc_program+0x527/frame 0xfffffe0000584180
>svc_run_internal() at svc_run_internal+0xb79/frame 0xfffffe00005842c0
>svc_run() at svc_run+0x1ee/frame 0xfffffe0000584320
>nfsrvd_nfsd() at nfsrvd_nfsd+0x356/frame 0xfffffe0000584480
>nfssvc_nfsd() at nfssvc_nfsd+0x57b/frame 0xfffffe0000584950
>sys_nfssvc() at sys_nfssvc+0xdb/frame 0xfffffe0000584980
>amd64_syscall() at amd64_syscall+0x27f/frame 0xfffffe0000584ab0
>fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0000584ab0
>--- syscall (155, FreeBSD ELF64, sys_nfssvc), rip = 0x8002dee8a, rsp = 0x7fffffffe518, rbp = 0x7fffffffe7b0 ---
>KDB: enter: panic
>
>__curthread () at ./machine/pcpu.h:230
>230		__asm("movq %%gs:%1,%0" : "=r" (td)
>(kgdb) #0  __curthread () at ./machine/pcpu.h:230
>#1  doadump (textdump=<optimized out>)
>    at /usr/src/sys/kern/kern_shutdown.c:366
>#2  0xffffffff8044906b in db_dump (dummy=<optimized out>, 
>    dummy2=<unavailable>, dummy3=<unavailable>, dummy4=<unavailable>)
>    at /usr/src/sys/ddb/db_command.c:574
>#3  0xffffffff80448e39 in db_command (last_cmdp=<optimized out>, 
>    cmd_table=<optimized out>, dopager=1) at /usr/src/sys/ddb/db_command.c:481
>#4  0xffffffff80448bb4 in db_command_loop ()
>    at /usr/src/sys/ddb/db_command.c:534
>#5  0xffffffff8044bd7f in db_trap (type=<optimized out>, code=<optimized out>)
>    at /usr/src/sys/ddb/db_main.c:252
>#6  0xffffffff80be79d5 in kdb_trap (type=3, code=0, tf=<optimized out>)
>    at /usr/src/sys/kern/subr_kdb.c:693
>#7  0xffffffff81084c3a in trap (frame=0xfffffe0000582fc0)
>    at /usr/src/sys/amd64/amd64/trap.c:605
>#8  <signal handler called>
>#9  kdb_enter (why=0xffffffff8131179d "panic", msg=<optimized out>)
>    at /usr/src/sys/kern/subr_kdb.c:479
>#10 0xffffffff80b9d7b1 in vpanic (fmt=<optimized out>, ap=0xfffffe0000583130)
>    at /usr/src/sys/kern/kern_shutdown.c:852
>#11 0xffffffff80b9d550 in kassert_panic (fmt=0xffffffff812a3ed5 "excl->share")
>    at /usr/src/sys/kern/kern_shutdown.c:749
>#12 0xffffffff80c083a8 in witness_checkorder (lock=0xfffff80004190ba8, 
>    flags=1, file=0xffffffff8130d3f0 "/usr/src/sys/kern/vfs_lookup.c", 
>    line=<optimized out>, 
>    interlock=0xffffffff81f64e2c <w_locklistdata+264684>)
>    at /usr/src/sys/kern/subr_witness.c:1176
>#13 0xffffffff80b70c3e in lockmgr_slock_hard (lk=<optimized out>, 
>    flags=2106368, ilk=0xfffff80004190bd8, file=<optimized out>, 
>    line=<optimized out>, lwa=<optimized out>)
>    at /usr/src/sys/kern/kern_lock.c:567
>#14 0xffffffff80b71d5b in __lockmgr_args (lk=<optimized out>, 
>    flags=<optimized out>, ilk=<optimized out>, wmesg=<optimized out>, 
>    pri=<optimized out>, timo=<optimized out>, 
>    file=0xffffffff8130d3f0 "/usr/src/sys/kern/vfs_lookup.c", line=671)
>    at /usr/src/sys/kern/kern_lock.c:1195
>#15 0xffffffff80ec6a05 in _lockmgr_args (lk=0xfffff80004190ba8, 
>    flags=2106368, ilk=<optimized out>, wmesg=<optimized out>, prio=0, 
>    timo=0, file=<optimized out>, line=18) at /usr/src/sys/sys/lockmgr.h:104
>#16 ffs_lock (ap=0xfffffe00005833c8) at /usr/src/sys/ufs/ffs/ffs_vnops.c:428
>#17 0xffffffff81208059 in VOP_LOCK1_APV (
>    vop=0xffffffff81b62d50 <ffs_vnodeops2>, a=0xfffffe00005833c8)
>    at vnode_if.c:2087
>#18 0xffffffff80c86927 in VOP_LOCK1 (vp=<optimized out>, flags=2106368, 
>    file=<optimized out>, line=671) at ./vnode_if.h:859
>#19 _vn_lock (vp=0xfffff80004190b40, flags=2106368, 
>    file=0xffffffff8130d3f0 "/usr/src/sys/kern/vfs_lookup.c", line=671)
>    at /usr/src/sys/kern/vfs_vnops.c:1531
>#20 0xffffffff80c68fe6 in lookup (ndp=0xfffffe00005835a0)
>    at /usr/src/sys/kern/vfs_lookup.c:669
>#21 0xffffffff80c68aad in namei (ndp=0xfffffe00005835a0)
>    at /usr/src/sys/kern/vfs_lookup.c:450
>#22 0xffffffff80c49b85 in unp_connectat (fd=<optimized out>, 
>    so=<optimized out>, nam=<optimized out>, td=0xfffff800036f2000)
>    at /usr/src/sys/kern/uipc_usrreq.c:1554
>#23 0xffffffff80c3bc98 in soconnectat (fd=<optimized out>, 
>    so=<optimized out>, nam=0xfffff8000367d800, 
>    td=0xffffffff80b7d390 <_mtx_init+144>)
>    at /usr/src/sys/kern/uipc_socket.c:1230
>#24 0xffffffff80e611b9 in clnt_vc_create (so=0xfffff80004c316d0, 
>    raddr=0xfffff800035fc020, prog=553713921, vers=1, sendsz=4096, 
>    recvsz=4096, intrflag=0) at /usr/src/sys/rpc/clnt_vc.c:159
>#25 0xffffffff80e60439 in clnt_reconnect_connect (cl=0xfffff80003372840)
>    at /usr/src/sys/rpc/clnt_rc.c:193
>#26 clnt_reconnect_call (cl=0xfffff80003372840, ext=0xfffffe0000583ab0, 
>    proc=1, args=0xfffff800048dda00, resultsp=0xfffffe0000583c28, 
>    utimeout=...) at /usr/src/sys/rpc/clnt_rc.c:265
>#27 0xffffffff80a637ec in newnfs_request (nd=0xfffffe0000583c28, nmp=0x0, 
>    clp=0x0, nrp=0xffffffff82021a18 <nfsrv_nfsuserdsock>, vp=0x0, td=0x0, 
>    cred=0xfffff80003e65000, prog=553713921, vers=1, retsum=0x0, toplevel=0, 
>    xidp=0x0, dssep=0x0) at /usr/src/sys/fs/nfs/nfs_commonkrpc.c:818
>#28 0xffffffff80a6d5f9 in nfsrv_getuser (procnum=1, uid=<optimized out>, 
>    gid=<optimized out>, name=0x0, p=0xfffffe0000582ce0)
>    at /usr/src/sys/fs/nfs/nfs_commonsubs.c:3616
>#29 0xffffffff80a6d71a in nfsrv_getgrpscred (oldcred=0xfffff80003deec00)
>    at /usr/src/sys/fs/nfs/nfs_commonsubs.c:3150
>#30 0xffffffff80acf53e in nfsd_excred (nd=0xfffffe0000583ff8, 
>    exp=<optimized out>, credanon=0xfffffe0000582f80)
>    at /usr/src/sys/fs/nfsserver/nfs_nfsdport.c:2932
>#31 0xffffffff80aa8104 in nfsrvd_compound (nd=<optimized out>, isdgram=0, 
>    tag=0x10 <error: Cannot access memory at address 0x10>, 
>    taglen=<optimized out>, minorvers=<optimized out>, p=<optimized out>)
>    at /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:1008
>#32 nfsrvd_dorpc (nd=0xfffffe0000583ff8, isdgram=0, 
>    tag=0x10 <error: Cannot access memory at address 0x10>, taglen=7, 
>    minorvers=<optimized out>, p=0xfffff800036f2000)
>    at /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:560
>#33 0xffffffff80abc3a7 in nfs_proc (xid=<optimized out>, 
>    xprt=<optimized out>, nd=<optimized out>, rpp=<optimized out>)
>    at /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c:387
>#34 nfssvc_program (rqst=0xfffff8004d1d5800, xprt=0xfffff800035fb600)
>    at /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c:272
>#35 0xffffffff80e68499 in svc_executereq (rqstp=<optimized out>)
>    at /usr/src/sys/rpc/svc.c:1031
>#36 svc_run_internal (grp=<optimized out>, ismaster=1)
>    at /usr/src/sys/rpc/svc.c:1306
>#37 0xffffffff80e6785e in svc_run (pool=<optimized out>)
>    at /usr/src/sys/rpc/svc.c:1385
>#38 0xffffffff80abca06 in nfsrvd_nfsd (td=<optimized out>, 
>    args=0xfffffe0000584510) at /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c:542
>#39 0xffffffff80ad298b in nfssvc_nfsd (td=0xfffff800036f2000, 
>    uap=<optimized out>) at /usr/src/sys/fs/nfsserver/nfs_nfsdport.c:3451
>#40 0xffffffff80e45eeb in sys_nfssvc (td=0xfffff800036f2000, 
>    uap=0xfffff800036f23c0) at /usr/src/sys/nfs/nfs_nfssvc.c:111
>#41 0xffffffff810859ef in syscallenter (td=0xfffff800036f2000)
>    at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:135
>#42 amd64_syscall (td=0xfffff800036f2000, traced=0)
>    at /usr/src/sys/amd64/amd64/trap.c:1029
>#43 <signal handler called>
>#44 0x00000008002dee8a in ?? ()
>Backtrace stopped: Cannot access memory at address 0x7fffffffe518
>(kgdb) 
>
>------------------------------------------------------------------------
>ps -axlww
>
>UID  PID PPID CPU PRI NI    VSZ   RSS MWCHAN   STAT TT     TIME COMMAND
>  0    0    0   0 -16  0      0     0 swapin   DLs   -  0:00.04 [kernel]
>  0    1    0   0  52  0  10072  1052 wait     DLs   -  0:00.01 [init]
>  0    2    0   0 -16  0      0     0 crypto_w DL    -  0:00.00 [crypto]
>  0    3    0   0 -16  0      0     0 crypto_r DL    -  0:00.00 [crypto returns 0]
>  0    4    0   0 -16  0      0     0 crypto_r DL    -  0:00.00 [crypto returns 1]
>  0    5    0   0 -16  0      0     0 crypto_r DL    -  0:00.00 [crypto returns 2]
>  0    6    0   0 -16  0      0     0 crypto_r DL    -  0:00.00 [crypto returns 3]
>  0    7    0   0 -16  0      0     0 -        RL    -  0:00.04 [cam]
>  0    8    0   0 -16  0      0     0 idle     DL    -  0:00.00 [mpt_recovery0]
>  0    9    0   0 -16  0      0     0 -        DL    -  0:00.00 [soaiod1]
>  0   10    0   0 -16  0      0     0 audit_wo DL    -  0:00.00 [audit]
>  0   11    0   0 155  0      0     0 -        RL    -  1:18.28 [idle]
>  0   12    0   0 -56  0      0     0 -        WL    -  0:00.13 [intr]
>  0   13    0   0  -8  0      0     0 -        DL    -  0:00.05 [geom]
>  0   14    0   0 -68  0      0     0 -        DL    -  0:00.01 [usb]
>  0   15    0   0 -16  0      0     0 -        DL    -  0:00.00 [soaiod2]
>  0   16    0   0 -16  0      0     0 -        DL    -  0:00.00 [soaiod3]
>  0   17    0   0 -16  0      0     0 -        DL    -  0:00.00 [soaiod4]
>  0   18    0   0 -16  0      0     0 waiting_ DL    -  0:00.00 [sctp_iterator]
>  0   19    0   0 -16  0      0     0 -        DL    -  0:00.02 [rand_harvestq]
>  0   20    0   0 -16  0      0     0 psleep   DL    -  0:00.00 [pagedaemon]
>  0   21    0   0 -16  0      0     0 psleep   DL    -  0:00.00 [vmdaemon]
>  0   22    0   0 -16  0      0     0 qsleep   DL    -  0:00.00 [bufdaemon]
>  0   23    0   0  16  0      0     0 syncer   DL    -  0:00.00 [syncer]
>  0   24    0   0 -16  0      0     0 vlruwt   DL    -  0:00.00 [vnlru]
>  0  321    1   0  52  0  11488  2612 select   Ds    -  0:00.00 [dhclient]
>  0  324    1   0  52  0  11708  2740 select   Ds    -  0:00.00 [dhclient]
> 65  370    1   0  20  0  11872  2868 select   DCs   -  0:00.00 [dhclient]
>  0  442    1   0  20  0  10588  1536 select   Ds    -  0:00.00 [devd]
>  0  513    1   0  20  0  11500  2676 select   Ds    -  0:00.01 [syslogd]
>  0  524    1   0  20  0  11320  2508 select   Ds    -  0:00.01 [rpcbind]
>  0  633    1   0  52  0  11616  2512 pause    Ds    -  0:00.00 [nfsuserd]
>  0  634  633   0  20  0  11660  2560 ufs      D     -  0:00.00 [nfsuserd]
>  0  641    1   0  52  0  13412  3176 select   Ds    -  0:00.00 [mountd]
>  0  647    1   0  20  0  13116  2908 select   Ds    -  0:00.01 [nfsd]
>  0  650  647   0  52  0  11240  2496 -        R     -  0:00.00 [nfsd]
>  0  651    1   0  20  0 275328  3104 select   Ds    -  0:00.00 [rpc.statd]
>  0  654    1   0  52  0  13204  3096 rpcsvc   Ds    -  0:00.00 [rpc.lockd]
>123  697    1   0  20  0  18048 18112 select   Ds    -  0:00.10 [ntpd]
>  0  727    1   0  20  0  19244  6052 ufs      D     -  0:44.79 [vmtoolsd]
>  0  823    1   0  20  0  21612  8920 kqread   Ds    -  0:00.01 [master]
>125  825  823   0  20  0  21648  9004 kqread   D     -  0:00.01 [qmgr]
>  0  861    1   0  52  0  18304  7964 select   Ds    -  0:00.00 [sshd]
>  0  875    1   0  20  0  11520  2936 ufs      Ds    -  0:00.00 [cron]
>  0  921    1   0  52  0  10980  2544 ttyin    Ds+   -  0:00.00 [getty]
>  0  922    1   0  52  0  10980  2544 ttyin    Ds+   -  0:00.00 [getty]
>  0  923    1   0  52  0  10980  2544 ttyin    Ds+   -  0:00.00 [getty]
>  0  924    1   0  52  0  10980  2544 ttyin    Ds+   -  0:00.00 [getty]
>  0  925    1   0  52  0  10980  2544 ttyin    Ds+   -  0:00.00 [getty]
>  0  926    1   0  52  0  10980  2544 ttyin    Ds+   -  0:00.00 [getty]
>  0  927    1   0  52  0  10980  2544 ttyin    Ds+   -  0:00.00 [getty]
>  0  928    1   0  52  0  10980  2544 ttyin    Ds+   -  0:00.00 [getty]
>  0  929    1   0  52  0  10968  2540 ttyin    Ds+   -  0:00.00 [getty]
>125 1386  823   0  20  0  21588  8948 kqread   D     -  0:00.00 [pickup]
>
>------------------------------------------------------------------------
>vmstat -s
>
>  5447636 cpu context switches
>  1147010 device interrupts
>   311160 software interrupts
>   212586 traps
>  2705233 system calls
>       24 kernel threads created
>      794  fork() calls
>      680 vfork() calls
>        2 rfork() calls
>        0 swap pager pageins
>        0 swap pager pages paged in
>        0 swap pager pageouts
>        0 swap pager pages paged out
>     1023 vnode pager pageins
>     7462 vnode pager pages paged in
>        1 vnode pager pageouts
>        1 vnode pager pages paged out
>        0 page daemon wakeups
>        0 pages examined by the page daemon
>        0 clean page reclamation shortfalls
>        0 pages reactivated by the page daemon
>    55621 copy-on-write faults
>       55 copy-on-write optimized faults
>    98984 zero fill pages zeroed
>        0 zero fill pages prezeroed
>        0 intransit blocking page faults
>   173703 total VM faults taken
>      892 page faults requiring I/O
>        0 pages affected by kernel thread creation
>    47143 pages affected by  fork()
>    24359 pages affected by vfork()
>      114 pages affected by rfork()
>   289551 pages freed
>        0 pages freed by daemon
>        0 pages freed by exiting processes
>        0 pages active
>        0 pages inactive
>        0 pages in the laundry queue
>        0 pages wired down
>        0 pages free
>        0 bytes per page
>        0 total name lookups
>          cache hits (0% pos + 0% neg) system 0% per-directory
>          deletions 0%, falsehits 0%, toolong 0%
>
>------------------------------------------------------------------------
>vmstat -m
>
>         Type InUse MemUse HighUse Requests  Size(s)
>      CAM XPT    41     4K       -      295  32,64,128,256,512,1024,2048,65536
>      entropy     1     1K       -       49  32,4096
>      CAM DEV    10    20K       -       37  2048
>      CAM CCB     0     0K       -    20748  2048
>     CAM path    16     1K       -      113  32
>   CAM periph    14     4K       -     7395  16,32,64,128,256
>         ac97     2     1K       -        2  16,512
>       feeder    13     1K       -       15  32,128
>        mixer     1     4K       -        1  4096
>CAM I/O Scheduler     5     1K       -        5  128
>    CAM queue    14    13K       -      106  16,32,64,512,2048
>         UART     3     3K       -        3  16,1024
>      scsi_cd     0     0K       -        6  16
>          USB    35    81K       -       36  16,32,128,256,512,1024,4096,8192,32768
>       USBdev    35     6K       -       35  32,64,128,256,512
>CAM dev queue     4     1K       -        4  64
>      scsi_da     0     0K       -      361  16,64,256
>      ata_pci     1     1K       -        1  64
>        vtbuf    24  1968K       -       46  4096
>           vt    11     6K       -       11  512
>       DEVFS3   120    30K       -      159  256
>       DEVFS1   106    53K       -      138  512
>   DEVFS_RULE    56    27K       -       56  64,512
>        DEVFS    15     1K       -       16  16,128
>       DEVFSP     2     1K       -     2213  64
>NFSD srvcache     1     1K       -      837  128
>NFSD V4client     3     1K       -        5  128,256,512
> NFSD V4state     0     0K       -       16  256
> NFSD lckfile     1     1K       -       12  128,256
>  NFSD string     1     1K       -        1  16
>NFSD usrgroup     7    33K       -      185  128,8192
> NFSD session     1     1K       -        1  1024
>    pfs_nodes    20     5K       -       20  256
>  pfs_vncache     1     1K       -        1  64
>         GEOM   133    63K       -     2033  16,32,64,128,256,512,1024,2048,4096,8192,16384
>    raid_data     0     0K       -      306  32,128,256
>       isadev     6     1K       -        6  128
>  ddb_capture     1    64K       -        1  65536
>         cdev     4     1K       -        4  256
>     filedesc     9    37K       -       15  16,4096,8192
>        sigio     0     0K       -        1  64
>     filecaps     1     1K       -      240  16,32,64
>      kdtrace   242    51K       -     3137  64,256
>         kenv   101    12K       -      105  16,32,64,128,8192
>       kqueue    60    11K       -     1513  64,256,2048
>    proc-args    62     3K       -     1281  16,32,64,128,256
>        hhook    13     4K       -       13  256
>      ithread   172    26K       -      172  32,128,256
>       prison     6     1K       -        6  32
>       KTRACE   100    13K       -      100  128
>       linker   302   165K       -      348  16,32,64,128,256,512,1024,2048,4096,8192,16384
>        lockf    31     4K       -      263  64,128
>   loginclass     2     1K       -        2  64
>        cache     1     1K       -        1  32
>       devbuf  2175  3448K       -     2691  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
>         temp    20    17K       -    11117  16,32,64,128,256,512,1024,2048,16384,65536
>       module   514    65K       -      515  128
>     mtx_pool     2    16K       -        2  8192
>          osd     3     1K       -        9  16,32,64,128,256
>     pmchooks     1     1K       -        1  128
>          pmc     1     1K       -        1  64
>         pgrp    26     4K       -      146  128
>      session    26     4K       -      145  128
>         proc     2    32K       -        2  16384
>      subproc   136   257K       -     1583  512,4096
>         cred    85    22K       -     2282  256
>     acpiintr     1     1K       -        1  64
>       acpica  8521   806K       -    80146  16,32,64,128,256,512,1024,2048
>       plimit    26     7K       -     2127  256
>      uidinfo     6     5K       -       81  128,4096
>       dumper     1     1K       -        1  512
>     acpitask     1    64K       -        1  65536
>       sysctl     0     0K       -        1  64
>    sysctloid  4815   244K       -     4908  16,32,64,128
>    sysctltmp     0     0K       -    32548  16,64,256,1024
>       kbdmux     6    22K       -        6  16,512,1024,2048,16384
>      tidhash     1    32K       -        1  32768
>      callout     5  2184K       -        5  
>         umtx   452    57K       -      452  128
>     p1003.1b     1     1K       -        1  16
>         SWAP     1   548K       -        1  
>          bus  1306   117K       -    24981  16,32,64,128,256,512,1024,4096
>       bus-sc    80  1823K       -     7626  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
>      devstat    14    29K       -       14  32,4096
>        epoch     4     1K       -        4  128
> eventhandler   120    10K       -      120  64,128
>   gtaskqueue    34    35K       -       34  16,32,256,8192
>         kobj   353  1412K       -      718  4096
>      Per-cpu     1     1K       -        1  32
>         rman   699    74K       -      967  16,32,128
>         sbuf     0     0K       -     6432  16,32,64,128,256,512,1024,2048,4096,8192,16384
>    toponodes    20     3K       -       20  128
>    taskqueue    30     4K       -       30  16,32,256
>     terminal    11     3K       -       11  256
>       Unitno    23     2K       -      587  32,64
>         vmem     3    38K       -        4  2048,4096,32768
>     ioctlops     0     0K       -     4796  256,512,1024
>       select    23     3K       -       23  128
>          iov     0     0K       -    46694  16,64,128,256,512
>          msg     4    30K       -        4  2048,4096,8192,16384
>          sem     4   106K       -        4  2048,4096
>          shm     1    32K       -        1  32768
>          tty    13    13K       -       13  1024
>     mbuf_tag     0     0K       -       17  32
>        shmfd     1     8K       -        1  8192
>       soname    35     4K       -    49505  16,32,128
>          pcb    39  1692K       -       74  16,32,64,128,1024,2048,8192
>          acl     0     0K       -      612  4096
>     vfscache     4  2097K       -        4  256,16384,32768
>   cl_savebuf     0     0K       -       16  64
>  export_host     3     1K       -        3  256
>     vfs_hash     1  1024K       -        1  
>       vnodes     1     1K       -        1  256
>        mount    94     5K       -      228  16,32,64,128,256
>       statfs     0     0K       -     6035  4096
>  vnodemarker     0     0K       -     6979  512
>chacha20random     1     1K       -        1  1024
>          BPF    10    18K       -     6643  128,512,4096
>        ifnet     3     5K       -        3  128,2048
>       ifaddr    36    13K       -       36  16,32,64,128,256,512,2048,4096
>  ether_multi    19     2K       -       24  16,32,64,128
>        clone     7     1K       -        7  128
>        ipsec     1     1K       -        1  256
>      lltable    11     4K       -       15  256,512
>        iflib    14    82K       -       18  16,64,128,1024,4096,8192,16384,32768
>     routetbl    34     7K       -       38  32,64,128,256,512
>         vnet     1     1K       -        1  64
>    vnet_data     1   216K       -        1  
>vnet_data_free     1     1K       -        1  32
>         igmp     2     1K       -        2  128
>   in_mfilter     1     1K       -        1  1024
>     in_multi     2     1K       -        3  256
>  ip_moptions     2     1K       -        2  64,256
>encap_export_host     8     1K       -        8  64
>    sctp_a_it     0     0K       -        2  16
>     sctp_vrf     1     1K       -        1  64
>     sctp_ifa     4     1K       -        4  128
>     sctp_ifn     2     1K       -        2  128
>    sctp_iter     0     0K       -        2  256
>   tfo_ccache     1   128K       -        1  
>    hostcache     1    32K       -        1  32768
>          LRO     2    40K       -        2  8192,32768
>      tcpfunc     1     1K       -        1  64
>     syncache     1    68K       -        1  
>  in6_mfilter     2     2K       -        2  1024
>    in6_multi    16     2K       -       16  32,256
> ip6_moptions     4     1K       -        4  64,256
>       ip6opt     3     1K       -        3  256
>          mld     2     1K       -        2  128
>       ip6ndp     3     1K       -        3  64,256
>  inpcbpolicy    38     2K       -     2258  32
>     secasvar     1     1K       -        1  1024
>       sahead     1     1K       -        1  1024
>  ipsecpolicy     2     2K       -        2  256,1024
>    ipsec-saq     2     2K       -        2  1024
>          NLM     0     0K       -        1  32
>       crypto     2     2K       -        2  512,1024
>          rpc   102    31K       -     1178  16,32,64,128,256,512,1024,2048,4096
>audit_evclass   230     8K       -      285  32
>      pagedep     4   512K       -       69  256
>     inodedep    12  4100K       -      562  512
>    bmsafemap     4    32K       -      156  256,8192
>       newblk     4  8192K       -      172  256
>     indirdep     0     0K       -        3  128
>     freefrag     0     0K       -       39  128
>     freeblks     0     0K       -       64  256
>     freefile     1     1K       -       94  64
>       diradd     0     0K       -      403  128
>        mkdir     0     0K       -       14  128
>       dirrem     0     0K       -      394  128
>    newdirblk     0     0K       -        7  64
>     freework     4     1K       -       70  16,128
>        sbdep     0     0K       -       88  64
>     savedino     0     0K       -       73  256
>      softdep     4     2K       -        4  512
>  ufs_dirhash    81    18K       -       81  16,32,64,128,256,512
>    ufs_quota     1  1024K       -        1  
>    ufs_mount    12    74K       -       16  512,4096,8192,16384
>    vm_pgdata     1     1K       -        1  128
>      UMAHash     4    18K       -       10  512,1024,2048,4096,8192,16384
>      memdesc     1     4K       -        1  4096
>     pci_link     8     1K       -        8  64,128
>     atkbddev     2     1K       -        2  64
>      acpisem    33     5K       -       33  128
>      acpidev    71     5K       -       71  64
>      CAM SIM     4     1K       -        4  256
>       apmdev     1     1K       -        1  128
>   madt_table     0     0K       -        2  64,4096
>      io_apic     1     2K       -        1  2048
>   local_apic     1     8K       -        1  8192
>          MCA    13     2K       -       13  32,128
>         cpus     2     1K       -        2  32
>          msi    33     5K       -       33  128
>     nexusdev     5     1K       -        5  16
>  fdesc_mount     1     1K       -        1  16
>VMBlockFS hash     1     1K       -        1  128
>
>------------------------------------------------------------------------
>vmstat -z
>
>vmstat: memstat_kvm_uma: invalid address (0xfffff800bfedbe00)
>ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
>
>
>
>------------------------------------------------------------------------
>vmstat -i
>
>interrupt                          total       rate
>irq1: atkbd0                           2          0
>irq4: uart0                            4          0
>irq15: ata1                        14713         82
>irq17: mpt0 ehci0                   4497         25
>irq18: uhci0                       38676        215
>irq19: em0:irq0                  1089084       6063
>cpu0:timer                        385426       2146
>cpu2:timer                        175387        976
>cpu3:timer                        141704        789
>cpu1:timer                        199199       1109
>irq257: xhci0                         34          0
>Total                            2048726      11406
>
>------------------------------------------------------------------------
>pstat -T
>
>211/162607 files
>0M/0M swap space
>
>------------------------------------------------------------------------
>pstat -s
>
>Device          512-blocks     Used    Avail Capacity
>
>------------------------------------------------------------------------
>iostat
>
>       tty             da0              da1              da2             cpu
> tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
>   0     0 25.91   0  0.01   3.40   0  0.00  13.25   0  0.00   0  0  0  0 100
>
>------------------------------------------------------------------------
>ipcs -a
>
>Message Queues:
>T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME   
>
>Shared Memory:
>T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME   
>
>Semaphores:
>T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME   
>
>
>------------------------------------------------------------------------
>ipcs -T
>
>msginfo:
>	msgmax:        16384	(max characters in a message)
>	msgmni:           40	(# of message queues)
>	msgmnb:         2048	(max characters in a message queue)
>	msgtql:           40	(max # of messages in system)
>	msgssz:            8	(size of a message segment)
>	msgseg:         2048	(# of message segments in system)
>
>shminfo:
>	shmmax:    536870912	(max shared memory segment size)
>	shmmin:            1	(min shared memory segment size)
>	shmmni:          192	(max number of shared memory identifiers)
>	shmseg:          128	(max shared memory segments per process)
>	shmall:       131072	(max amount of shared memory in pages)
>
>seminfo:
>	semmni:           50	(# of semaphore identifiers)
>	semmns:          340	(# of semaphores in system)
>	semmnu:          150	(# of undo structures in system)
>	semmsl:          340	(max # of semaphores per id)
>	semopm:          100	(max # of operations per semop call)
>	semume:           50	(max # of undo entries per process)
>	semusz:          632	(size in bytes of undo structure)
>	semvmx:        32767	(semaphore maximum value)
>	semaem:        16384	(adjust on exit max value)
>
>
>------------------------------------------------------------------------
>nfsstat
>
>Rpc Counts:
>      Getattr      Setattr       Lookup     Readlink         Read        Write       Create       Remove
>            0            0            0            0            0            0            0            0
>       Rename         Link      Symlink        Mkdir        Rmdir      Readdir     RdirPlus       Access
>            0            0            0            0            0            0            0            0
>        Mknod       Fsstat       Fsinfo     PathConf       Commit
>            0            0            0            0            0
>Rpc Info:
>     TimedOut      Invalid    X Replies      Retries     Requests
>            0            0            0            0            0
>Cache Info:
>    Attr Hits  Attr Misses    Lkup Hits  Lkup Misses    BioR Hits  BioR Misses    BioW Hits  BioW Misses
>            0            0            0            0            0            0            0            0
>   BioRL Hits BioRL Misses    BioD Hits  BioD Misses    DirE Hits  DirE Misses    Accs Hits  Accs Misses
>            0            0            0            0            0            0            0            0
>Server Info:
>      Getattr      Setattr       Lookup     Readlink         Read        Write       Create       Remove
>            0            0            0            0            0            0            0            0
>       Rename         Link      Symlink        Mkdir        Rmdir      Readdir     RdirPlus       Access
>            0            0            0            0            0            0            0            0
>        Mknod       Fsstat       Fsinfo     PathConf       Commit
>            0            0            0            0            0
>Server Re-Failed:
>  140737488346832
>Server Faults:
>140737488346832
>Server Write 
>     WriteOps     WriteRPC      Opsaved
>            0            0            0
>Server Cache 
>       Inprog         Idem     Non-Idem       Misses
>            0            0            0            0
>
>------------------------------------------------------------------------
>netstat -s
>
>tcp:
>	1403 packets sent
>		838 data packets (168472 bytes)
>		0 data packets (0 bytes) retransmitted
>		0 data packets unnecessarily retransmitted
>		0 resends initiated by MTU discovery
>		556 ack-only packets (2 delayed)
>		0 URG only packets
>		0 window probe packets
>		5 window update packets
>		4 control packets
>	2390 packets received
>		836 acks (for 168481 bytes)
>		0 duplicate acks
>		0 acks for unsent data
>		1917 packets (1697696 bytes) received in-sequence
>		0 completely duplicate packets (0 bytes)
>		0 old duplicate packets
>		0 packets with some dup. data (0 bytes duped)
>		0 out-of-order packets (0 bytes)
>		0 packets (0 bytes) of data after window
>		0 window probes
>		0 window update packets
>		0 packets received after close
>		0 discarded for bad checksums
>		0 discarded for bad header offset fields
>		0 discarded because packet too short
>		0 discarded due to memory problems
>	0 connection requests
>	5 connection accepts
>	0 bad connection attempts
>	0 listen queue overflows
>	0 ignored RSTs in the windows
>	5 connections established (including accepts)
>		4 times used RTT from hostcache
>		4 times used RTT variance from hostcache
>		0 times used slow-start threshold from hostcache
>	41 connections closed (including 0 drops)
>		2 connections updated cached RTT on close
>		2 connections updated cached RTT variance on close
>		0 connections updated cached ssthresh on close
>	0 embryonic connections dropped
>	836 segments updated rtt (of 824 attempts)
>	0 retransmit timeouts
>		0 connections dropped by rexmit timeout
>	0 persist timeouts
>		0 connections dropped by persist timeout
>	0 Connections (fin_wait_2) dropped because of timeout
>	0 keepalive timeouts
>		0 keepalive probes sent
>		0 connections dropped by keepalive
>	458 correct ACK header predictions
>	1543 correct data packet header predictions
>	5 syncache entries added
>		0 retransmitted
>		0 dupsyn
>		0 dropped
>		5 completed
>		0 bucket overflow
>		0 cache overflow
>		0 reset
>		0 stale
>		0 aborted
>		0 badack
>		0 unreach
>		0 zone failures
>	5 cookies sent
>	5 cookies received
>	2 hostcache entries added
>		0 bucket overflow
>	0 SACK recovery episodes
>	0 segment rexmits in SACK recovery episodes
>	0 byte rexmits in SACK recovery episodes
>	0 SACK options (SACK blocks) received
>	0 SACK options (SACK blocks) sent
>	0 SACK scoreboard overflow
>	0 packets with ECN CE bit set
>	0 packets with ECN ECT(0) bit set
>	0 packets with ECN ECT(1) bit set
>	0 successful ECN handshakes
>	0 times ECN reduced the congestion window
>	0 packets with matching signature received
>	0 packets with bad signature received
>	0 times failed to make signature due to no SA
>	0 times unexpected signature received
>	0 times no signature provided by segment
>	0 Path MTU discovery black hole detection activations
>	0 Path MTU discovery black hole detection min MSS activations
>	0 Path MTU discovery black hole detection failures
>TCP connection count by state:
>	0 connections in CLOSED state
>	13 connections in LISTEN state
>	0 connections in SYN_SENT state
>	0 connections in SYN_RCVD state
>	1 connection  in ESTABLISHED state
>	0 connections in CLOSE_WAIT state
>	0 connections in FIN_WAIT_1 state
>	0 connections in CLOSING state
>	0 connections in LAST_ACK state
>	0 connections in FIN_WAIT_2 state
>	0 connections in TIME_WAIT state
>udp:
>	10469 datagrams received
>	0 with incomplete header
>	0 with bad data length field
>	0 with bad checksum
>	0 with no checksum
>	0 dropped due to no socket
>	10088 broadcast/multicast datagrams undelivered
>	0 dropped due to full socket buffers
>	0 not for hashed pcb
>	381 delivered
>	381 datagrams output
>	0 times multicast source filter matched
>ip:
>	14988 total packets received
>	0 bad header checksums
>	0 with size smaller than minimum
>	0 with data size < data length
>	0 with ip length > max ip packet size
>	0 with header length < data size
>	0 with data length < header length
>	0 with bad options
>	0 with incorrect version number
>	0 fragments received
>	0 fragments dropped (dup or out of space)
>	0 fragments dropped after timeout
>	0 packets reassembled ok
>	14987 packets for this host
>	0 packets for unknown/unsupported protocol
>	0 packets forwarded (0 packets fast forwarded)
>	1 packet not forwardable
>	0 packets received for unknown multicast group
>	0 redirects sent
>	1781 packets sent from this host
>	0 packets sent with fabricated ip header
>	0 output packets dropped due to no bufs, etc.
>	0 output packets discarded due to no route
>	0 output datagrams fragmented
>	0 fragments created
>	0 datagrams that can't be fragmented
>	0 tunneling packets that can't find gif
>	0 datagrams with bad address in header
>icmp:
>	0 calls to icmp_error
>	0 errors not generated in response to an icmp message
>	0 messages with bad code fields
>	0 messages less than the minimum length
>	0 messages with bad checksum
>	0 messages with bad length
>	0 multicast echo requests ignored
>	0 multicast timestamp requests ignored
>	0 message responses generated
>	0 invalid return addresses
>	0 no return routes
>ipsec:
>	0 inbound packets violated process security policy
>	0 inbound packets failed due to insufficient memory
>	0 invalid inbound packets
>	0 outbound packets violated process security policy
>	0 outbound packets with no SA available
>	0 outbound packets failed due to insufficient memory
>	0 outbound packets with no route available
>	0 invalid outbound packets
>	0 outbound packets with bundled SAs
>	0 spd cache hits
>	0 spd cache misses
>	0 clusters copied during clone
>	0 mbufs inserted during makespace
>ah:
>	0 packets shorter than header shows
>	0 packets dropped; protocol family not supported
>	0 packets dropped; no TDB
>	0 packets dropped; bad KCR
>	0 packets dropped; queue full
>	0 packets dropped; no transform
>	0 replay counter wraps
>	0 packets dropped; bad authentication detected
>	0 packets dropped; bad authentication length
>	0 possible replay packets detected
>	0 packets in
>	0 packets out
>	0 packets dropped; invalid TDB
>	0 bytes in
>	0 bytes out
>	0 packets dropped; larger than IP_MAXPACKET
>	0 packets blocked due to policy
>	0 crypto processing failures
>	0 tunnel sanity check failures
>esp:
>	0 packets shorter than header shows
>	0 packets dropped; protocol family not supported
>	0 packets dropped; no TDB
>	0 packets dropped; bad KCR
>	0 packets dropped; queue full
>	0 packets dropped; no transform
>	0 packets dropped; bad ilen
>	0 replay counter wraps
>	0 packets dropped; bad encryption detected
>	0 packets dropped; bad authentication detected
>	0 possible replay packets detected
>	0 packets in
>	0 packets out
>	0 packets dropped; invalid TDB
>	0 bytes in
>	0 bytes out
>	0 packets dropped; larger than IP_MAXPACKET
>	0 packets blocked due to policy
>	0 crypto processing failures
>	0 tunnel sanity check failures
>ipcomp:
>	0 packets shorter than header shows
>	0 packets dropped; protocol family not supported
>	0 packets dropped; no TDB
>	0 packets dropped; bad KCR
>	0 packets dropped; queue full
>	0 packets dropped; no transform
>	0 replay counter wraps
>	0 packets in
>	0 packets out
>	0 packets dropped; invalid TDB
>	0 bytes in
>	0 bytes out
>	0 packets dropped; larger than IP_MAXPACKET
>	0 packets blocked due to policy
>	0 crypto processing failures
>	0 packets sent uncompressed; size < compr. algo. threshold
>	0 packets sent uncompressed; compression was useless
>arp:
>	8 ARP requests sent
>	29 ARP replies sent
>	287 ARP requests received
>	7 ARP replies received
>	294 ARP packets received
>	0 total packets dropped due to no ARP entry
>	4 ARP entrys timed out
>	0 Duplicate IPs seen
>ip6:
>	4 total packets received
>	0 with size smaller than minimum
>	0 with data size < data length
>	0 with bad options
>	0 with incorrect version number
>	0 fragments received
>	0 fragments dropped (dup or out of space)
>	0 fragments dropped after timeout
>	0 fragments that exceeded limit
>	0 packets reassembled ok
>	4 packets for this host
>	0 packets forwarded
>	0 packets not forwardable
>	0 redirects sent
>	4 packets sent from this host
>	0 packets sent with fabricated ip header
>	0 output packets dropped due to no bufs, etc.
>	0 output packets discarded due to no route
>	0 output datagrams fragmented
>	0 fragments created
>	0 datagrams that can't be fragmented
>	0 packets that violated scope rules
>	0 multicast packets which we don't join
>	Input histogram:
>		UDP: 4
>	Mbuf statistics:
>		2 one mbuf
>		two or more mbuf:
>			lo0= 2
>		0 one ext mbuf
>		0 two or more ext mbuf
>	0 packets whose headers are not contiguous
>	0 tunneling packets that can't find gif
>	0 packets discarded because of too many headers
>	5 failures of source address selection
>	source addresses on a non-outgoing I/F
>		5 addresses scope=0xf
>	Source addresses selection rule applied:
>		5 same address
>icmp6:
>	0 calls to icmp6_error
>	0 errors not generated in response to an icmp6 message
>	0 errors not generated because of rate limitation
>	0 messages with bad code fields
>	0 messages < minimum length
>	0 bad checksums
>	0 messages with bad length
>	Histogram of error messages to be generated:
>		0 no route
>		0 administratively prohibited
>		0 beyond scope
>		0 address unreachable
>		0 port unreachable
>		0 packet too big
>		0 time exceed transit
>		0 time exceed reassembly
>		0 erroneous header field
>		0 unrecognized next header
>		0 unrecognized option
>		0 redirect
>		0 unknown
>	0 message responses generated
>	0 messages with too many ND options
>	0 messages with bad ND options
>	0 bad neighbor solicitation messages
>	0 bad neighbor advertisement messages
>	0 bad router solicitation messages
>	0 bad router advertisement messages
>	0 bad redirect messages
>	0 path MTU changes
>ipsec6:
>	0 inbound packets violated process security policy
>	0 inbound packets failed due to insufficient memory
>	0 invalid inbound packets
>	0 outbound packets violated process security policy
>	0 outbound packets with no SA available
>	0 outbound packets failed due to insufficient memory
>	0 outbound packets with no route available
>	0 invalid outbound packets
>	0 outbound packets with bundled SAs
>	0 spd cache hits
>	0 spd cache misses
>	0 clusters copied during clone
>	0 mbufs inserted during makespace
>rip6:
>	0 messages received
>	0 checksum calculations on inbound
>	0 messages with bad checksum
>	0 messages dropped due to no socket
>	0 multicast messages dropped due to no socket
>	0 messages dropped due to full socket buffers
>	0 delivered
>	0 datagrams output
>pfkey:
>	0 requests sent from userland
>	0 bytes sent from userland
>	0 messages with invalid length field
>	0 messages with invalid version field
>	0 messages with invalid message type field
>	0 messages too short
>	0 messages with memory allocation failure
>	0 messages with duplicate extension
>	0 messages with invalid extension type
>	0 messages with invalid sa type
>	0 messages with invalid address extension
>	0 requests sent to userland
>	0 bytes sent to userland
>	0 messages toward single socket
>	0 messages toward all sockets
>	0 messages toward registered sockets
>	0 messages with memory allocation failure
>
>------------------------------------------------------------------------
>netstat -m
>
>netstat: memstat_kvm_all: invalid address (0xfffff800bfedbe00)
>
>------------------------------------------------------------------------
>netstat -anA
>
>Active Internet connections (including servers)
>Tcpcb            Proto Recv-Q Send-Q Local Address      Foreign Address    (state)    
>fffff80005a30000 tcp4       0      0 *.111              *.*                LISTEN     
>fffff80005a303c0 tcp6       0      0 *.111              *.*                LISTEN     
>fffff800054fd988 udp6       0      0 *.*                *.*                
>fffff800054fe000 udp4       0      0 *.613              *.*                
>fffff800054fe3d0 udp4       0      0 *.111              *.*                
>fffff800054fe7a0 udp6       0      0 *.860              *.*                
>fffff800054fe988 udp6       0      0 *.111              *.*                
>fffff800053fc3d0 udp4       0      0 *.514              *.*                
>fffff800053fc5b8 udp6       0      0 *.514              *.*                
>
>------------------------------------------------------------------------
>netstat -aL
>
>Current listen queue sizes (qlen/incqlen/maxqlen)
>Proto Listen                           Local Address         
>tcp4  0/0/128                          *.sunrpc               
>tcp6  0/0/128                          *.sunrpc               
>
>------------------------------------------------------------------------
>fstat
>
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x600000000
>fstat: can't read file 7 at 0x20007ffffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 10 at 0x780000ffff
>fstat: can't read file 11 at 0x600000000
>fstat: can't read file 13 at 0x20007ffffffffff
>fstat: can't read file 14 at 0x4000000001fffff
>fstat: can't read file 16 at 0x780000ffff
>fstat: can't read file 17 at 0x600000000
>fstat: can't read file 19 at 0x20007ffffffffff
>fstat: can't read file 20 at 0x4000000001fffff
>fstat: can't read file 22 at 0x780000ffff
>fstat: can't read file 23 at 0x4e00000001
>fstat: can't read file 25 at 0x20007ffffffffff
>fstat: can't read file 26 at 0x4000000001fffff
>fstat: can't read file 28 at 0x780000ffff
>fstat: can't read file 29 at 0x800000001
>fstat: can't read file 31 at 0x20007ffffffffff
>fstat: can't read file 32 at 0x4000000001fffff
>fstat: can't read file 34 at 0x780000ffff
>fstat: can't read file 35 at 0x800000001
>fstat: can't read file 37 at 0x20007ffffffffff
>fstat: can't read file 38 at 0x4000000001fffff
>fstat: can't read file 40 at 0x780000ffff
>fstat: can't read file 41 at 0x400000001
>fstat: can't read file 43 at 0x20007ffffffffff
>fstat: can't read file 44 at 0x4000000001fffff
>fstat: can't read file 46 at 0x780000ffff
>fstat: can't read file 47 at 0x5200000001
>fstat: can't read file 49 at 0x20007ffffffffff
>fstat: can't read file 50 at 0x4000000001fffff
>fstat: can't read file 52 at 0x780000ffff
>fstat: can't read file 53 at 0x3600000000
>fstat: can't read file 55 at 0x20007ffffffffff
>fstat: can't read file 56 at 0x4000000001fffff
>fstat: can't read file 58 at 0x780000ffff
>fstat: can't read file 59 at 0xa00000000
>fstat: can't read file 61 at 0x20007ffffffffff
>fstat: can't read file 62 at 0x4000000001fffff
>fstat: can't read file 64 at 0x780000ffff
>fstat: can't read file 65 at 0x2a00000000
>fstat: can't read file 71 at 0x11800000001
>fstat: can't read file 77 at 0x11c00000001
>fstat: can't read file 83 at 0x3000000001
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x600000000
>fstat: can't read file 7 at 0x20007ffffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 10 at 0x780000ffff
>fstat: can't read file 11 at 0x600000000
>fstat: can't read file 13 at 0x20007ffffffffff
>fstat: can't read file 14 at 0x4000000001fffff
>fstat: can't read file 16 at 0x780000ffff
>fstat: can't read file 17 at 0x600000000
>fstat: can't read file 19 at 0x20007ffffffffff
>fstat: can't read file 20 at 0x4000000001fffff
>fstat: can't read file 22 at 0x780000ffff
>fstat: can't read file 23 at 0x4e00000001
>fstat: can't read file 25 at 0x20007ffffffffff
>fstat: can't read file 26 at 0x4000000001fffff
>fstat: can't read file 28 at 0x780000ffff
>fstat: can't read file 29 at 0x800000001
>fstat: can't read file 31 at 0x20007ffffffffff
>fstat: can't read file 32 at 0x4000000001fffff
>fstat: can't read file 34 at 0x780000ffff
>fstat: can't read file 35 at 0x800000001
>fstat: can't read file 37 at 0x20007ffffffffff
>fstat: can't read file 38 at 0x4000000001fffff
>fstat: can't read file 40 at 0x780000ffff
>fstat: can't read file 41 at 0x400000001
>fstat: can't read file 43 at 0x20007ffffffffff
>fstat: can't read file 44 at 0x4000000001fffff
>fstat: can't read file 46 at 0x780000ffff
>fstat: can't read file 47 at 0x5200000001
>fstat: can't read file 49 at 0x20007ffffffffff
>fstat: can't read file 50 at 0x4000000001fffff
>fstat: can't read file 52 at 0x780000ffff
>fstat: can't read file 53 at 0x3600000000
>fstat: can't read file 59 at 0x25c00000001
>fstat: can't read file 65 at 0x139400000001
>fstat: can't read file 67 at 0x20007ffffffffff
>fstat: can't read file 68 at 0x4000000001fffff
>fstat: can't read file 70 at 0x780000ffff
>fstat: can't read file 71 at 0x600000000
>fstat: can't read file 73 at 0x20007ffffffffff
>fstat: can't read file 74 at 0x4000000001fffff
>fstat: can't read file 76 at 0x780000ffff
>fstat: can't read file 77 at 0xa00000000
>fstat: can't read file 83 at 0x7000000001
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x600000000
>fstat: can't read file 7 at 0x20007ffffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 10 at 0x780000ffff
>fstat: can't read file 11 at 0x600000000
>fstat: can't read file 13 at 0x20007ffffffffff
>fstat: can't read file 14 at 0x4000000001fffff
>fstat: can't read file 16 at 0x780000ffff
>fstat: can't read file 17 at 0x600000000
>fstat: can't read file 19 at 0x20007ffffffffff
>fstat: can't read file 20 at 0x4000000001fffff
>fstat: can't read file 22 at 0x780000ffff
>fstat: can't read file 23 at 0x4a00000001
>fstat: can't read file 25 at 0x20007ffffffffff
>fstat: can't read file 26 at 0x4000000001fffff
>fstat: can't read file 28 at 0x780000ffff
>fstat: can't read file 29 at 0x600000001
>fstat: can't read file 31 at 0x20007ffffffffff
>fstat: can't read file 32 at 0x4000000001fffff
>fstat: can't read file 34 at 0x780000ffff
>fstat: can't read file 35 at 0x600000000
>fstat: can't read file 37 at 0x20007ffffffffff
>fstat: can't read file 38 at 0x4000000001fffff
>fstat: can't read file 40 at 0x780000ffff
>fstat: can't read file 41 at 0x200000001
>fstat: can't read file 43 at 0x20007ffffffffff
>fstat: can't read file 44 at 0x4000000001fffff
>fstat: can't read file 46 at 0x780000ffff
>fstat: can't read file 47 at 0x200000001
>fstat: can't read file 49 at 0x20007ffffffffff
>fstat: can't read file 50 at 0x4000000001fffff
>fstat: can't read file 52 at 0x780000ffff
>fstat: can't read file 53 at 0x200000001
>fstat: can't read file 55 at 0x20007ffffffffff
>fstat: can't read file 56 at 0x4000000001fffff
>fstat: can't read file 58 at 0x780000ffff
>fstat: can't read file 59 at 0x200000001
>fstat: can't read file 61 at 0x20007ffffffffff
>fstat: can't read file 62 at 0x4000000001fffff
>fstat: can't read file 64 at 0x780000ffff
>fstat: can't read file 65 at 0x2600000001
>fstat: can't read file 67 at 0x20007ffffffffff
>fstat: can't read file 68 at 0x4000000001fffff
>fstat: can't read file 70 at 0x780000ffff
>fstat: can't read file 71 at 0x200000001
>fstat: can't read file 73 at 0x20007ffffffffff
>fstat: can't read file 74 at 0x4000000001fffff
>fstat: can't read file 76 at 0x780000ffff
>fstat: can't read file 77 at 0x600000001
>fstat: can't read file 79 at 0x20007ffffffffff
>fstat: can't read file 80 at 0x4000000001fffff
>fstat: can't read file 82 at 0x780000ffff
>fstat: can't read file 83 at 0x2e00000001
>fstat: can't read file 85 at 0x20007ffffffffff
>fstat: can't read file 86 at 0x4000000001fffff
>fstat: can't read file 88 at 0x780000ffff
>fstat: can't read file 89 at 0x200000000
>fstat: can't read file 91 at 0x20007ffffffffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x400000000
>fstat: can't read file 7 at 0x20007ffffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 10 at 0x780000ffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x600000000
>fstat: can't read file 7 at 0x20007ffffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 10 at 0x780000ffff
>fstat: can't read file 11 at 0x600000000
>fstat: can't read file 13 at 0x20007ffffffffff
>fstat: can't read file 14 at 0x4000000001fffff
>fstat: can't read file 16 at 0x780000ffff
>fstat: can't read file 17 at 0x600000000
>fstat: can't read file 19 at 0x20007ffffffffff
>fstat: can't read file 20 at 0x4000000001fffff
>fstat: can't read file 22 at 0x780000ffff
>fstat: can't read file 23 at 0xde00000001
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x400000000
>fstat: can't read file 7 at 0x20007ffffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x400000000
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x400000000
>fstat: can't read file 7 at 0x20007ffffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x400000000
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x400000000
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x400000000
>fstat: can't read file 7 at 0x20007ffffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 10 at 0x780000ffff
>fstat: can't read file 11 at 0x400000000
>fstat: can't read file 13 at 0x20007ffffffffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x400000000
>fstat: can't read file 7 at 0x20007ffffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 10 at 0x780000ffff
>fstat: can't read file 11 at 0x400000000
>fstat: can't read file 13 at 0x20007ffffffffff
>fstat: can't read file 14 at 0x4000000001fffff
>fstat: can't read file 16 at 0x780000ffff
>fstat: can't read file 17 at 0x400000000
>fstat: can't read file 19 at 0x20007ffffffffff
>fstat: can't read file 20 at 0x4000000001fffff
>fstat: can't read file 22 at 0x780000ffff
>fstat: can't read file 23 at 0x2e00000001
>fstat: can't read file 25 at 0x20007ffffffffff
>fstat: can't read file 26 at 0x4000000001fffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x800000000
>fstat: can't read file 7 at 0x20007ffffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 1 at 0x200000000000000
>fstat: can't read file 2 at 0x400000000000000
>fstat: can't read file 5 at 0x800000000
>fstat: can't read file 7 at 0x200000000000002
>fstat: can't read file 8 at 0x400000000000000
>fstat: can't read file 11 at 0x400000000
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x800000000
>fstat: can't read file 7 at 0x20007ffffffffff
>fstat: can't read file 8 at 0x4000000001fffff
>fstat: can't read file 10 at 0x780000ffff
>fstat: can't read file 1 at 0x20007ffffffffff
>fstat: can't read file 2 at 0x4000000001fffff
>fstat: can't read file 4 at 0x780000ffff
>fstat: can't read file 5 at 0x800000000
>USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
>postfix  pickup      1386 root /             2 drwxr-xr-x    1024  r
>postfix  pickup      1386   wd /        9149230 drwxr-xr-x     512  r
>postfix  pickup      1386 text /        5955150 -rwxr-xr-x   19864  r
>postfix  pickup      1386    0 /dev         14 crw-rw-rw-    null rw
>postfix  pickup      1386    6 /dev         14 crw-rw-rw-    null rw
>postfix  pickup      1386   12 /dev         14 crw-rw-rw-    null rw
>postfix  pickup      1386   18* pipe fffff80004ea35f0 <-> fffff80004ea3758    100 rw
>postfix  pickup      1386   24* pipe fffff80004ea3758 <-> fffff80004ea35f0      0 rw
>postfix  pickup      1386   30* pipe fffff8000461ad48 <-> fffff8000461abe0      0 rw
>postfix  pickup      1386   36* local stream fffff8000486a700
>postfix  pickup      1386   42* local dgram fffff80004849c00 <-> fffff8000484cb00
>postfix  pickup      1386   54* pipe fffff8004d10fbe0 <-> fffff8004d10fd48      0 rw
>postfix  pickup      1386   60* pipe fffff8004d10fd48 <-> fffff8004d10fbe0      0 rw
>root     getty        929 root /             2 drwxr-xr-x    1024  r
>root     getty        929   wd /             2 drwxr-xr-x    1024  r
>root     getty        929 text /        3692098 -r-xr-xr-x   36048  r
>root     getty        929 ctty /dev         57 crw-------   ttyu0 rw
>root     getty        929    0 /dev         57 crw-------   ttyu0 rw
>root     getty        928 root /             2 drwxr-xr-x    1024  r
>root     getty        928   wd /             2 drwxr-xr-x    1024  r
>root     getty        928 text /        3692098 -r-xr-xr-x   36048  r
>root     getty        928 ctty /dev         90 crw-------   ttyv7 rw
>root     getty        928    0 /dev         90 crw-------   ttyv7 rw
>root     getty        927 root /             2 drwxr-xr-x    1024  r
>root     getty        927   wd /             2 drwxr-xr-x    1024  r
>root     getty        927 text /        3692098 -r-xr-xr-x   36048  r
>root     getty        927 ctty /dev         89 crw-------   ttyv6 rw
>root     getty        927    0 /dev         89 crw-------   ttyv6 rw
>root     getty        926 root /             2 drwxr-xr-x    1024  r
>root     getty        926   wd /             2 drwxr-xr-x    1024  r
>root     getty        926 text /        3692098 -r-xr-xr-x   36048  r
>root     getty        926 ctty /dev         88 crw-------   ttyv5 rw
>root     getty        926    0 /dev         88 crw-------   ttyv5 rw
>root     getty        925 root /             2 drwxr-xr-x    1024  r
>root     getty        925   wd /             2 drwxr-xr-x    1024  r
>root     getty        925 text /        3692098 -r-xr-xr-x   36048  r
>root     getty        925 ctty /dev         87 crw-------   ttyv4 rw
>root     getty        925    0 /dev         87 crw-------   ttyv4 rw
>root     getty        924 root /             2 drwxr-xr-x    1024  r
>root     getty        924   wd /             2 drwxr-xr-x    1024  r
>root     getty        924 text /        3692098 -r-xr-xr-x   36048  r
>root     getty        924 ctty /dev         86 crw-------   ttyv3 rw
>root     getty        924    0 /dev         86 crw-------   ttyv3 rw
>root     getty        923 root /             2 drwxr-xr-x    1024  r
>root     getty        923   wd /             2 drwxr-xr-x    1024  r
>root     getty        923 text /        3692098 -r-xr-xr-x   36048  r
>root     getty        923 ctty /dev         85 crw-------   ttyv2 rw
>root     getty        923    0 /dev         85 crw-------   ttyv2 rw
>root     getty        922 root /             2 drwxr-xr-x    1024  r
>root     getty        922   wd /             2 drwxr-xr-x    1024  r
>root     getty        922 text /        3692098 -r-xr-xr-x   36048  r
>root     getty        922 ctty /dev         84 crw-------   ttyv1 rw
>root     getty        922    0 /dev         84 crw-------   ttyv1 rw
>root     getty        921 root /             2 drwxr-xr-x    1024  r
>root     getty        921   wd /             2 drwxr-xr-x    1024  r
>root     getty        921 text /        3692098 -r-xr-xr-x   36048  r
>root     getty        921 ctty /dev         83 crw-------   ttyv0 rw
>root     getty        921    0 /dev         83 crw-------   ttyv0 rw
>root     cron         875 root /             2 drwxr-xr-x    1024  r
>root     cron         875   wd /        9150743 drwxr-x---     512  r
>root     cron         875 text /        3693767 -r-xr-xr-x   52264  r
>root     cron         875    0 /dev         14 crw-rw-rw-    null rw
>root     sshd         861 root /             2 drwxr-xr-x    1024  r
>root     sshd         861   wd /             2 drwxr-xr-x    1024  r
>root     sshd         861 text /        3692242 -r-xr-xr-x  302984  r
>root     sshd         861    0 /dev         14 crw-rw-rw-    null rw
>postfix  qmgr         825 root /             2 drwxr-xr-x    1024  r
>postfix  qmgr         825   wd /        9149230 drwxr-xr-x     512  r
>postfix  qmgr         825 text /        5955148 -rwxr-xr-x   74720  r
>postfix  qmgr         825    0 /dev         14 crw-rw-rw-    null rw
>postfix  qmgr         825    6 /dev         14 crw-rw-rw-    null rw
>postfix  qmgr         825   12 /dev         14 crw-rw-rw-    null rw
>postfix  qmgr         825   18* pipe fffff80004ea35f0 <-> fffff80004ea3758    100 rw
>postfix  qmgr         825   24* pipe fffff80004ea3758 <-> fffff80004ea35f0      0 rw
>postfix  qmgr         825   30* pipe fffff80004424168 <-> fffff80004424000      0 rw
>postfix  qmgr         825   36* local stream fffff8000486ab00
>postfix  qmgr         825   42* local dgram fffff8000484b300 <-> fffff8000484cb00
>postfix  qmgr         825   66* pipe fffff800044245f0 <-> fffff80004424758      0 rw
>postfix  qmgr         825   72* pipe fffff80004424758 <-> fffff800044245f0      0 rw
>root     master       823 root /             2 drwxr-xr-x    1024  r
>root     master       823   wd /        9149230 drwxr-xr-x     512  r
>root     master       823 text /        5955146 -rwxr-xr-x   48912  r
>root     master       823    0 /dev         14 crw-rw-rw-    null rw
>root     master       823    6 /dev         14 crw-rw-rw-    null rw
>root     master       823   12 /dev         14 crw-rw-rw-    null rw
>root     master       823   18* local dgram fffff8000489b700 <-> fffff8000484cb00
>root     master       823   24 /dev         14 crw-rw-rw-    null rw
>root     master       823   30* pipe fffff80004ea32f8 <-> fffff80004ea3460      0 rw
>root     master       823   36 /dev         14 crw-rw-rw-    null rw
>root     master       823   42 /dev         14 crw-rw-rw-    null rw
>root     master       823   48 /dev         14 crw-rw-rw-    null rw
>root     master       823   54 /dev         14 crw-rw-rw-    null rw
>root     master       823   60 /        9229490 -rw-------      33 rw
>root     master       823   66 /        9710992 -rw-------      33 rw
>root     master       823   72* pipe fffff80004ea38e8 <-> fffff80004ea3a50      0 rw
>root     master       823   78* internet stream tcp fffff80004b9bb40
>root     master       823   90* pipe fffff8000461b000 <-> fffff8000461b168      0 rw
>root     vmtoolsd     727 root /             2 drwxr-xr-x    1024  r
>root     vmtoolsd     727   wd /             2 drwxr-xr-x    1024  r
>root     vmtoolsd     727 text /        4351249 -rwxr-xr-x   52184  r
>root     vmtoolsd     727    0 /dev         14 crw-rw-rw-    null  r
>root     vmtoolsd     727    6 /dev         14 crw-rw-rw-    null  w
>ntpd     ntpd         697 root /             2 drwxr-xr-x    1024  r
>ntpd     ntpd         697   wd /             2 drwxr-xr-x    1024  r
>ntpd     ntpd         697 text /        3694313 -r-xr-xr-x  855168  r
>ntpd     ntpd         697    0 /dev         14 crw-rw-rw-    null  r
>ntpd     ntpd         697    6 /dev         14 crw-rw-rw-    null  r
>ntpd     ntpd         697   12 /dev         14 crw-rw-rw-    null  r
>ntpd     ntpd         697   18* local dgram fffff80004860900 <-> fffff80004890100
>root     rpc.lockd    654 root /             2 drwxr-xr-x    1024  r
>root     rpc.lockd    654   wd /             2 drwxr-xr-x    1024  r
>root     rpc.lockd    654 text /        3693955 -r-xr-xr-x   73200  r
>root     rpc.lockd    654    0 /dev         14 crw-rw-rw-    null rw
>root     rpc.statd    651 root /             2 drwxr-xr-x    1024  r
>root     rpc.statd    651   wd /             2 drwxr-xr-x    1024  r
>root     rpc.statd    651 text /        3693957 -r-xr-xr-x   32056  r
>root     rpc.statd    651    0 /dev         14 crw-rw-rw-    null rw
>root     rpc.statd    651    6 /dev         14 crw-rw-rw-    null rw
>root     nfsd         650 root /             2 drwxr-xr-x    1024  r
>root     nfsd         650   wd /             2 drwxr-xr-x    1024  r
>root     nfsd         650 text /        3693824 -r-xr-xr-x   31672  r
>root     nfsd         650    0 /dev         14 crw-rw-rw-    null rw
>root     nfsd         647 root /             2 drwxr-xr-x    1024  r
>root     nfsd         647   wd /             2 drwxr-xr-x    1024  r
>root     nfsd         647 text /        3693824 -r-xr-xr-x   31672  r
>root     nfsd         647    0 /dev         14 crw-rw-rw-    null rw
>root     nfsd         647    6 /dev         14 crw-rw-rw-    null rw
>root     mountd       641 root /             2 drwxr-xr-x    1024  r
>root     mountd       641   wd /             2 drwxr-xr-x    1024  r
>root     mountd       641 text /        3693816 -r-xr-xr-x   52512  r
>root     mountd       641    0 /dev         14 crw-rw-rw-    null rw
>root     mountd       641    6 /dev         14 crw-rw-rw-    null rw
>root     nfsuserd     634 root /             2 drwxr-xr-x    1024  r
>root     nfsuserd     634   wd /             2 drwxr-xr-x    1024  r
>root     nfsuserd     634 text /        3693858 -r-xr-xr-x   23664  r
>root     nfsuserd     634    0 /dev         14 crw-rw-rw-    null rw
>root     nfsuserd     634    6 /dev         14 crw-rw-rw-    null rw
>root     nfsuserd     633 root /             2 drwxr-xr-x    1024  r
>root     nfsuserd     633   wd /             2 drwxr-xr-x    1024  r
>root     nfsuserd     633 text /        3693858 -r-xr-xr-x   23664  r
>root     nfsuserd     633    0 /dev         14 crw-rw-rw-    null rw
>root     rpcbind      524 root /             2 drwxr-xr-x    1024  r
>root     rpcbind      524   wd /             2 drwxr-xr-x    1024  r
>root     rpcbind      524 text /        3693942 -r-xr-xr-x   52176  r
>root     rpcbind      524    0 /dev         14 crw-rw-rw-    null rw
>root     rpcbind      524    6 /dev         14 crw-rw-rw-    null rw
>root     rpcbind      524   12 /dev         14 crw-rw-rw-    null rw
>root     syslogd      513 root /             2 drwxr-xr-x    1024  r
>root     syslogd      513   wd /             2 drwxr-xr-x    1024  r
>root     syslogd      513 text /        3693975 -r-xr-xr-x   56528  r
>root     syslogd      513    0 /dev         14 crw-rw-rw-    null rw
>root     syslogd      513    6 /dev         14 crw-rw-rw-    null rw
>root     syslogd      513   12 /dev         14 crw-rw-rw-    null rw
>root     syslogd      513   18* pipe fffff80004486000 <-> fffff80004486168      0 rw
>root     syslogd      513   24* pipe fffff80004486168 <-> fffff80004486000      0 rw
>root     devd         442 root /             2 drwxr-xr-x    1024  r
>root     devd         442   wd /             2 drwxr-xr-x    1024  r
>root     devd         442 text /        3692120 -r-xr-xr-x  1086024  r
>root     devd         442    0 /dev         14 crw-rw-rw-    null rw
>root     devd         442    6 /dev         14 crw-rw-rw-    null rw
>_dhcp    dhclient     370 root /             2 drwxr-xr-x    1024  r
>_dhcp    dhclient     370   wd /             2 drwxr-xr-x    1024  r
>_dhcp    dhclient     370 text /        3691964 -r-xr-xr-x  107048  r
>_dhcp    dhclient     370    0 /dev         14 crw-rw-rw-    null rw
>_dhcp    dhclient     370    6 /dev         14 crw-rw-rw-    null rw
>root     dhclient     324 root /             2 drwxr-xr-x    1024  r
>root     dhclient     324   wd /             2 drwxr-xr-x    1024  r
>root     dhclient     324 text /        3691964 -r-xr-xr-x  107048  r
>root     dhclient     324    0 /dev         14 crw-rw-rw-    null rw
>root     dhclient     324    6 /dev         14 crw-rw-rw-    null rw
>root     dhclient     321 root /             2 drwxr-xr-x    1024  r
>root     dhclient     321   wd /             2 drwxr-xr-x    1024  r
>root     dhclient     321 text /        3691964 -r-xr-xr-x  107048  r
>root     dhclient     321    0 /dev         14 crw-rw-rw-    null rw
>root     dhclient     321    6 /dev         14 crw-rw-rw-    null rw
>root     init           1 root /             2 drwxr-xr-x    1024  r
>root     init           1   wd /             2 drwxr-xr-x    1024  r
>root     init           1 text /        3692038 -r-xr-xr-x  1351720  r
>root     kernel         0 root /             2 drwxr-xr-x    1024  r
>root     kernel         0   wd /             2 drwxr-xr-x    1024  r
>
>------------------------------------------------------------------------
>dmesg
>
>---<<BOOT>>---
>Copyright (c) 1992-2018 The FreeBSD Project.
>Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>	The Regents of the University of California. All rights reserved.
>FreeBSD is a registered trademark of The FreeBSD Foundation.
>FreeBSD 12.0-ALPHA2 #0 r338015: Sat Aug 18 17:36:21 CEST 2018
>    dim@vdim-fbsdhead-amd64:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
>FreeBSD clang version 7.0.0 (branches/release_70 339999) (based on LLVM 7.0.0svn)
>WARNING: WITNESS option enabled, expect reduced performance.
>VT(vga): text 80x25
>CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz (4000.00-MHz K8-class CPU)
>  Origin="GenuineIntel"  Id=0x306c3  Family=0x6  Model=0x3c  Stepping=3
>  Features=0xf83fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,SS>
>  Features2=0xfffa3203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV>
>  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
>  AMD Features2=0x21<LAHF,ABM>
>  Structured Extended Features=0x27ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG>
>  XSAVE Features=0x1<XSAVEOPT>
>  TSC: P-state invariant
>Hypervisor: Origin = "VMwareVMware"
>real memory  = 5368709120 (5120 MB)
>avail memory = 5122158592 (4884 MB)
>Event timer "LAPIC" quality 600
>ACPI APIC Table: <PTLTD  	 APIC  >
>FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
>FreeBSD/SMP: 4 package(s) x 1 core(s)
>random: unblocking device.
>MADT: Forcing active-low polarity and level trigger for SCI
>ioapic0 <Version 2.0> irqs 0-23 on motherboard
>Launching APs: 2 3 1
>random: entropy device external interface
>netmap: loaded module
>[ath_hal] loaded
>module_register_init: MOD_LOAD (vesa, 0xffffffff81107950, 0) error 19
>random: registering fast source Intel Secure Key RNG
>random: fast provider: "Intel Secure Key RNG"
>kbd1 at kbdmux0
>nexus0
>vtvga0: <VT VGA driver> on motherboard
>cryptosoft0: <software crypto> on motherboard
>acpi0: <INTEL 440BX> on motherboard
>acpi0: Power Button (fixed)
>hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
>Timecounter "HPET" frequency 14318180 Hz quality 950
>cpu0: <ACPI CPU> on acpi0
>attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
>Timecounter "i8254" frequency 1193182 Hz quality 0
>Event timer "i8254" frequency 1193182 Hz quality 100
>atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
>atrtc0: registered as a time-of-day clock, resolution 1.000000s
>Event timer "RTC" frequency 32768 Hz quality 0
>Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
>acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
>pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
>pci0: <ACPI PCI bus> on pcib0
>pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
>pci1: <ACPI PCI bus> on pcib1
>isab0: <PCI-ISA bridge> at device 7.0 on pci0
>isa0: <ISA bus> on isab0
>atapci0: <Intel PIIX4 UDMA33 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1060-0x106f at device 7.1 on pci0
>ata0: <ATA channel> at channel 0 on atapci0
>ata1: <ATA channel> at channel 1 on atapci0
>pci0: <bridge> at device 7.3 (no driver attached)
>vgapci0: <VGA-compatible display> port 0x1070-0x107f mem 0xe8000000-0xefffffff,0xfe000000-0xfe7fffff irq 16 at device 15.0 on pci0
>vgapci0: Boot video device
>mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xfeba0000-0xfebbffff,0xfebc0000-0xfebdffff irq 17 at device 16.0 on pci0
>mpt0: MPI Version=1.2.0.0
>pcib2: <ACPI PCI-PCI bridge> at device 17.0 on pci0
>pci2: <ACPI PCI bus> on pcib2
>uhci0: <UHCI (generic) USB controller> port 0x2080-0x209f irq 18 at device 0.0 on pci2
>usbus0 on uhci0
>usbus0: 12Mbps Full Speed USB v1.0
>em0: <Intel(R) PRO/1000 Network Connection> port 0x2000-0x203f mem 0xfd5c0000-0xfd5dffff,0xfdff0000-0xfdffffff irq 19 at device 1.0 on pci2
>em0: attach_pre capping queues at 1
>em0: using 1024 tx descriptors and 1024 rx descriptors
>em0: allocated for 1 tx_queues
>em0: allocated for 1 rx_queues
>em0: Ethernet address: 00:0c:29:19:64:3d
>Link state changed to up
>em0: link state changed to UP
>em0: netmap queues/slots: TX 1/1024, RX 1/1024
>pcm0: <AudioPCI ES1371-A> port 0x2040-0x207f irq 16 at device 2.0 on pci2
>pcm0: <Cirrus Logic CS4297A AC97 Codec>
>pcm0: <Playback: DAC1,DAC2 / Record: ADC>
>ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfd5ef000-0xfd5effff irq 17 at device 3.0 on pci2
>usbus1: EHCI version 1.0
>usbus1 on ehci0
>usbus1: 480Mbps High Speed USB v2.0
>pcib3: <ACPI PCI-PCI bridge> at device 21.0 on pci0
>pcib3: [GIANT-LOCKED]
>pci3: <ACPI PCI bus> on pcib3
>xhci0: <XHCI (generic) USB 3.0 controller> mem 0xfd4e0000-0xfd4fffff irq 18 at device 0.0 on pci3
>xhci0: 32 bytes context size, 64-bit DMA
>xhci0: Unable to map MSI-X table 
>usbus2 on xhci0
>usbus2: 5.0Gbps Super Speed USB v3.0
>pcib4: <ACPI PCI-PCI bridge> at device 21.1 on pci0
>pcib4: [GIANT-LOCKED]
>pcib5: <ACPI PCI-PCI bridge> at device 21.2 on pci0
>pcib5: [GIANT-LOCKED]
>pcib6: <ACPI PCI-PCI bridge> at device 21.3 on pci0
>pcib6: [GIANT-LOCKED]
>pcib7: <ACPI PCI-PCI bridge> at device 21.4 on pci0
>pcib7: [GIANT-LOCKED]
>pcib8: <ACPI PCI-PCI bridge> at device 21.5 on pci0
>pcib8: [GIANT-LOCKED]
>pcib9: <ACPI PCI-PCI bridge> at device 21.6 on pci0
>pcib9: [GIANT-LOCKED]
>pcib10: <ACPI PCI-PCI bridge> at device 21.7 on pci0
>pcib10: [GIANT-LOCKED]
>pcib11: <ACPI PCI-PCI bridge> at device 22.0 on pci0
>pcib11: [GIANT-LOCKED]
>pcib12: <ACPI PCI-PCI bridge> at device 22.1 on pci0
>pcib12: [GIANT-LOCKED]
>pcib13: <ACPI PCI-PCI bridge> at device 22.2 on pci0
>pcib13: [GIANT-LOCKED]
>pcib14: <ACPI PCI-PCI bridge> at device 22.3 on pci0
>pcib14: [GIANT-LOCKED]
>pcib15: <ACPI PCI-PCI bridge> at device 22.4 on pci0
>pcib15: [GIANT-LOCKED]
>pcib16: <ACPI PCI-PCI bridge> at device 22.5 on pci0
>pcib16: [GIANT-LOCKED]
>pcib17: <ACPI PCI-PCI bridge> at device 22.6 on pci0
>pcib17: [GIANT-LOCKED]
>pcib18: <ACPI PCI-PCI bridge> at device 22.7 on pci0
>pcib18: [GIANT-LOCKED]
>pcib19: <ACPI PCI-PCI bridge> at device 23.0 on pci0
>pcib19: [GIANT-LOCKED]
>pcib20: <ACPI PCI-PCI bridge> at device 23.1 on pci0
>pcib20: [GIANT-LOCKED]
>pcib21: <ACPI PCI-PCI bridge> at device 23.2 on pci0
>pcib21: [GIANT-LOCKED]
>pcib22: <ACPI PCI-PCI bridge> at device 23.3 on pci0
>pcib22: [GIANT-LOCKED]
>pcib23: <ACPI PCI-PCI bridge> at device 23.4 on pci0
>pcib23: [GIANT-LOCKED]
>pcib24: <ACPI PCI-PCI bridge> at device 23.5 on pci0
>pcib24: [GIANT-LOCKED]
>pcib25: <ACPI PCI-PCI bridge> at device 23.6 on pci0
>pcib25: [GIANT-LOCKED]
>pcib26: <ACPI PCI-PCI bridge> at device 23.7 on pci0
>pcib26: [GIANT-LOCKED]
>pcib27: <ACPI PCI-PCI bridge> at device 24.0 on pci0
>pcib27: [GIANT-LOCKED]
>pcib28: <ACPI PCI-PCI bridge> at device 24.1 on pci0
>pcib28: [GIANT-LOCKED]
>pcib29: <ACPI PCI-PCI bridge> at device 24.2 on pci0
>pcib29: [GIANT-LOCKED]
>pcib30: <ACPI PCI-PCI bridge> at device 24.3 on pci0
>pcib30: [GIANT-LOCKED]
>pcib31: <ACPI PCI-PCI bridge> at device 24.4 on pci0
>pcib31: [GIANT-LOCKED]
>pcib32: <ACPI PCI-PCI bridge> at device 24.5 on pci0
>pcib32: [GIANT-LOCKED]
>pcib33: <ACPI PCI-PCI bridge> at device 24.6 on pci0
>pcib33: [GIANT-LOCKED]
>pcib34: <ACPI PCI-PCI bridge> at device 24.7 on pci0
>pcib34: [GIANT-LOCKED]
>acpi_acad0: <AC Adapter> on acpi0
>atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
>atkbd0: <AT Keyboard> irq 1 on atkbdc0
>kbd0 at atkbd0
>atkbd0: [GIANT-LOCKED]
>psm0: <PS/2 Mouse> irq 12 on atkbdc0
>psm0: [GIANT-LOCKED]
>psm0: model IntelliMouse, device ID 3
>acpi_syscontainer0: <System Container> on acpi0
>uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
>uart0: console (115200,n,8,1)
>orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc9fff,0xca000-0xcafff,0xdc000-0xdffff,0xe0000-0xe7fff pnpid ORM0000 on isa0
>vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0
>Timecounters tick every 10.000 msec
>ugen1.1: <(0x15ad) EHCI root HUB> at usbus1
>ugen2.1: <0x15ad XHCI root HUB> at usbus2
>ugen0.1: <(0x15ad) UHCI root HUB> at usbus0
>uhub0: <(0x15ad) EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
>uhub1: <0x15ad XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus2
>uhub2: <(0x15ad) UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
>uhub2: 2 ports with 2 removable, self powered
>uhub1: 8 ports with 8 removable, self powered
>cd0 at ata1 bus 0 scbus1 target 0 lun 0
>cd0: <NECVMWar VMware IDE CDR10 1.00> Removable CD-ROM SCSI device
>cd0: Serial Number 10000000000000000001
>cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
>cd0: Attempt to query device size failed: NOT READY, Medium not present
>cd0: quirks=0x40<RETRY_BUSY>
>da0 at mpt0 bus 0 scbus2 target 0 lun 0
>da0: <VMware, VMware Virtual S 1.0> Fixed Direct Access SCSI-2 device
>da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit)
>da0: Command Queueing enabled
>da0: 102400MB (209715200 512 byte sectors)
>da0: quirks=0x140<RETRY_BUSY,STRICT_UNMAP>
>ugen2.2: <VMware VMware Virtual USB Mouse> at usbus2
>da1 at mpt0 bus 0 scbus2 target 1 lun 0
>da1: <VMware, VMware Virtual S 1.0> Fixed Direct Access SCSI-2 device
>da1: 320.000MB/s transfers (160.000MHz, offset 127, 16bit)
>da1: Command Queueing enabled
>da1: 8192MB (16777216 512 byte sectors)
>da1: quirks=0x140<RETRY_BUSY,STRICT_UNMAP>
>da2 at mpt0 bus 0 scbus2 target 2 lun 0
>da2: <VMware, VMware Virtual S 1.0> Fixed Direct Access SCSI-2 device
>da2: 320.000MB/s transfers (160.000MHz, offset 127, 16bit)
>da2: Command Queueing enabled
>da2: 163840MB (335544320 512 byte sectors)
>da2: quirks=0x140<RETRY_BUSY,STRICT_UNMAP>
>da3 at mpt0 bus 0 scbus2 target 3 lun 0
>da3: <VMware, VMware Virtual S 1.0> Fixed Direct Access SCSI-2 device
>da3: 320.000MB/s transfers (160.000MHz, offset 127, 16bit)
>da3: Command Queueing enabled
>da3: 102400MB (209715200 512 byte sectors)
>da3: quirks=0x140<RETRY_BUSY,STRICT_UNMAP>
>da4 at mpt0 bus 0 scbus2 target 4 lun 0
>da4: <VMware, VMware Virtual S 1.0> Fixed Direct Access SCSI-2 device
>da4: 320.000MB/s transfers (160.000MHz, offset 127, 16bit)
>da4: Command Queueing enabled
>da4: 102400MB (209715200 512 byte sectors)
>da4: quirks=0x140<RETRY_BUSY,STRICT_UNMAP>
>Trying to mount root from ufs:/dev/da0a [rw]...
>WARNING: WITNESS option enabled, expect reduced performance.
>ugen0.2: <vendor 0x0e0f VMware Virtual USB Hub> at usbus0
>uhub3 on uhub2
>uhub3: <VMware Virtual USB Hub> on usbus0
>uhub3: 7 ports with 7 removable, self powered
>uhub0: 6 ports with 6 removable, self powered
>Setting hostuuid: 564d2ee5-87a5-9616-8795-b415df19643d.
>Setting hostid: 0x4b303636.
>Starting file system checks:
>/dev/da0a: FILE SYSTEM CLEAN; SKIPPING CHECKS
>/dev/da0a: clean, 12195462 free (154358 frags, 1505138 blocks, 0.6% fragmentation)
>/dev/da3a: FILE SYSTEM CLEAN; SKIPPING CHECKS
>/dev/da3a: clean, 25389125 free (21 frags, 3173638 blocks, 0.0% fragmentation)
>/dev/da2a: FILE SYSTEM CLEAN; SKIPPING CHECKS
>/dev/da2a: clean, 12069529 free (2034225 frags, 1254413 blocks, 5.0% fragmentation)
>/dev/da4a: FILE SYSTEM CLEAN; SKIPPING CHECKS
>/dev/da4a: clean, 14780076 free (84 frags, 1847499 blocks, 0.0% fragmentation)
>Mounting local filesystems:.
>ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat /usr/local/lib/dovecot /usr/local/lib/gcc48 /usr/local/lib/gcc49 /usr/local/lib/gcc5 /usr/local/lib/gcc6 /usr/local/lib/gcc7 /usr/local/lib/gcc8 /usr/local/lib/gcc9 /usr/local/lib/nss /usr/local/lib/perl5/5.28/mach/CORE /usr/local/lib/samba4 /usr/local/llvm60/lib
>32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat
>Setting hostname: vdim-fbsdhead-amd64.
>Setting up harvesting: PURE_RDRAND,[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
>Feeding entropy: .
>Loading vmmemctl kernel module: 
>VMware memory control driver initialized
>done.
>Loading vmxnet kernel module: done.
>Loading vmblock kernel module: done.
>lo0: link state changed to UP
>Starting Network: lo0 em0.
>lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
>	inet6 ::1 prefixlen 128 
>	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
>	inet 127.0.0.1 netmask 0xff000000 
>	groups: lo 
>	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>	options=81009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER>
>	ether 00:0c:29:19:64:3d
>	media: Ethernet autoselect (1000baseT <full-duplex>)
>	status: active
>	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>Starting devd.
>Autoloading module: intpm.ko
>intsmb0: <Intel PIIX4 SMBUS Interface> port 0x1040-0x104f at device 7.3 on pci0
>intsmb0: intr SMI disabled revision 0
>smbus0: <System Management Bus> on intsmb0
>Starting dhclient.
>DHCPREQUEST on em0 to 255.255.255.255 port 67
>DHCPACK from 192.168.0.1
>bound to 192.168.0.142 -- renewal in 43200 seconds.
>Autoloading module: uhid.ko
>uhid0 on uhub1
>uhid0: <VMware> on usbus2
>uhid1 on uhub1
>uhid1: <VMware> on usbus2
>Autoloading module: uhid.ko
>add host 127.0.0.1: gateway lo0 fib 0: route already in table
>add host ::1: gateway lo0 fib 0: route already in table
>add net fe80::: gateway ::1
>add net ff02::: gateway ::1
>add net ::ffff:0.0.0.0: gateway ::1
>add net ::0.0.0.0: gateway ::1
>Creating and/or trimming log files.
>Starting syslogd.
>Setting date via ntp.
>19 Aug 09:47:02 ntpdate[521]: step time server 192.168.0.1 offset -3.015222 sec
>Starting rpcbind.
>No core dumps found.
>Clearing /tmp (X related).
>Starting nfsuserd.
>Starting mountd.
>Starting nfsd.
>Starting statd.
>Starting lockd.
>Updating motd:.
>Mounting late filesystems:.
>Security policy loaded: MAC/ntpd (mac_ntpd)
>Starting ntpd.
>Starting vmware_guestd.
>==Error==
>Config file /usr/local/etc/dovecot/dovecot.conf does not exist. If this is
>a new installation, please create the config files as outlined in
>	# pkg info -D dovecot
>postfix/postfix-script: starting the Postfix mail system
>/etc/rc: INFO: RSA host key exists.
>/etc/rc: INFO: ECDSA host key exists.
>/etc/rc: INFO: ED25519 host key exists.
>Performing sanity check on sshd configuration.
>Starting sshd.
>Configuring vt: blanktime.
>Starting cron.
>
>Sun Aug 19 09:47:08 CEST 2018
>lock order reversal:
> 1st 0xfffffe0000fa85c0 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3916
> 2nd 0xfffff80004e7aa00 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:289
>stack backtrace:
>#0 0xffffffff80c08fb3 at witness_debugger+0x73
>#1 0xffffffff80c08e34 at witness_checkorder+0xe34
>#2 0xffffffff80ba6848 at _sx_xlock+0x68
>#3 0xffffffff80ecbcfd at ufsdirhash_add+0x3d
>#4 0xffffffff80eceb48 at ufs_direnter+0x458
>#5 0xffffffff80ed7e00 at ufs_makeinode+0x600
>#6 0xffffffff80ed3ac4 at ufs_create+0x34
>#7 0xffffffff81205733 at VOP_CREATE_APV+0xd3
>#8 0xffffffff80acca2c at nfsvno_open+0x18c
>#9 0xffffffff80ac325f at nfsrvd_open+0xa5f
>#10 0xffffffff80aa806b at nfsrvd_dorpc+0xceb
>#11 0xffffffff80abc3a7 at nfssvc_program+0x527
>#12 0xffffffff80e68499 at svc_run_internal+0xb79
>#13 0xffffffff80e6785e at svc_run+0x1ee
>#14 0xffffffff80abca06 at nfsrvd_nfsd+0x356
>#15 0xffffffff80ad298b at nfssvc_nfsd+0x57b
>#16 0xffffffff80e45eeb at sys_nfssvc+0xdb
>#17 0xffffffff810859ef at amd64_syscall+0x27f
>shared lock of (lockmgr) ufs @ /usr/src/sys/kern/vfs_lookup.c:671
>while exclusively locked from /usr/src/sys/kern/vfs_subr.c:2590
>panic: excl->share
>cpuid = 2
>time = 1534686985
>KDB: stack backtrace:
>db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0000583090
>vpanic() at vpanic+0x1b4/frame 0xfffffe00005830f0
>doadump() at doadump/frame 0xfffffe0000583170
>witness_checkorder() at witness_checkorder+0x3a8/frame 0xfffffe0000583200
>lockmgr_slock_hard() at lockmgr_slock_hard+0x6e/frame 0xfffffe0000583290
>__lockmgr_args() at __lockmgr_args+0x8db/frame 0xfffffe0000583330
>ffs_lock() at ffs_lock+0xa5/frame 0xfffffe0000583380
>VOP_LOCK1_APV() at VOP_LOCK1_APV+0xd9/frame 0xfffffe00005833b0
>_vn_lock() at _vn_lock+0x67/frame 0xfffffe0000583420
>lookup() at lookup+0x116/frame 0xfffffe00005834d0
>namei() at namei+0x4dd/frame 0xfffffe0000583590
>unp_connectat() at unp_connectat+0x155/frame 0xfffffe0000583820
>soconnectat() at soconnectat+0xe8/frame 0xfffffe0000583870
>clnt_vc_create() at clnt_vc_create+0x349/frame 0xfffffe00005839b0
>clnt_reconnect_call() at clnt_reconnect_call+0x239/frame 0xfffffe0000583a60
>newnfs_request() at newnfs_request+0x8fc/frame 0xfffffe0000583bd0
>nfsrv_getuser() at nfsrv_getuser+0x129/frame 0xfffffe0000583d70
>nfsrv_getgrpscred() at nfsrv_getgrpscred+0xca/frame 0xfffffe0000583dc0
>nfsd_excred() at nfsd_excred+0x10e/frame 0xfffffe0000583de0
>nfsrvd_dorpc() at nfsrvd_dorpc+0xd84/frame 0xfffffe0000583fc0
>nfssvc_program() at nfssvc_program+0x527/frame 0xfffffe0000584180
>svc_run_internal() at svc_run_internal+0xb79/frame 0xfffffe00005842c0
>svc_run() at svc_run+0x1ee/frame 0xfffffe0000584320
>nfsrvd_nfsd() at nfsrvd_nfsd+0x356/frame 0xfffffe0000584480
>nfssvc_nfsd() at nfssvc_nfsd+0x57b/frame 0xfffffe0000584950
>sys_nfssvc() at sys_nfssvc+0xdb/frame 0xfffffe0000584980
>amd64_syscall() at amd64_syscall+0x27f/frame 0xfffffe0000584ab0
>fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0000584ab0
>--- syscall (155, FreeBSD ELF64, sys_nfssvc), rip = 0x8002dee8a, rsp = 0x7fffffffe518, rbp = 0x7fffffffe7b0 ---
>KDB: enter: panic
>
>------------------------------------------------------------------------
>kernel config
>
>options	CONFIG_AUTOGENERATED
>ident	GENERIC
>machine	amd64
>cpu	HAMMER
>makeoptions	WITH_CTF=1
>makeoptions	DEBUG=-g
>options	XENHVM
>options	USB_DEBUG
>options	ATH_ENABLE_11N
>options	AH_AR5416_INTERRUPT_MITIGATION
>options	AH_SUPPORT_AR5416
>options	IEEE80211_SUPPORT_MESH
>options	IEEE80211_AMPDU_AGE
>options	IEEE80211_DEBUG
>options	SC_PIXEL_MODE
>options	VESA
>options	PCI_IOV
>options	PCI_HP
>options	ACPI_DMAR
>options	EARLY_AP_STARTUP
>options	SMP
>options	NETDUMP
>options	ZSTDIO
>options	GZIO
>options	EKCD
>options	MALLOC_DEBUG_MAXZONES=8
>options	WITNESS_SKIPSPIN
>options	WITNESS
>options	INVARIANT_SUPPORT
>options	INVARIANTS
>options	DEADLKRES
>options	GDB
>options	FULL_BUF_TRACKING
>options	DDB
>options	BUF_TRACKING
>options	KDB_TRACE
>options	KDB
>options	RCTL
>options	RACCT_DEFAULT_TO_DISABLED
>options	RACCT
>options	INCLUDE_CONFIG_FILE
>options	DDB_CTF
>options	KDTRACE_HOOKS
>options	KDTRACE_FRAME
>options	MAC
>options	CAPABILITIES
>options	CAPABILITY_MODE
>options	AUDIT
>options	HWPMC_HOOKS
>options	KBD_INSTALL_CDEV
>options	PRINTF_BUFR_SIZE=128
>options	_KPOSIX_PRIORITY_SCHEDULING
>options	SYSVSEM
>options	SYSVMSG
>options	SYSVSHM
>options	STACK
>options	KTRACE
>options	SCSI_DELAY=5000
>options	COMPAT_FREEBSD11
>options	COMPAT_FREEBSD10
>options	COMPAT_FREEBSD9
>options	COMPAT_FREEBSD7
>options	COMPAT_FREEBSD6
>options	COMPAT_FREEBSD5
>options	COMPAT_FREEBSD4
>options	COMPAT_FREEBSD32
>options	EFIRT
>options	GEOM_LABEL
>options	GEOM_RAID
>options	PSEUDOFS
>options	PROCFS
>options	CD9660
>options	MSDOSFS
>options	NFS_ROOT
>options	NFSLOCKD
>options	NFSD
>options	NFSCL
>options	MD_ROOT
>options	QUOTA
>options	UFS_GJOURNAL
>options	UFS_DIRHASH
>options	UFS_ACL
>options	SOFTUPDATES
>options	FFS
>options	SCTP
>options	TCP_RFC7413
>options	TCP_HHOOK
>options	TCP_BLACKBOX
>options	TCP_OFFLOAD
>options	IPSEC_SUPPORT
>options	IPSEC
>options	INET6
>options	INET
>options	VIMAGE
>options	PREEMPTION
>options	SCHED_ULE
>options	NEW_PCIB
>options	GEOM_PART_GPT
>options	GEOM_PART_MBR
>options	GEOM_PART_EBR_COMPAT
>options	GEOM_PART_EBR
>options	GEOM_PART_BSD
>device	isa
>device	mem
>device	io
>device	uart_ns8250
>device	cpufreq
>device	acpi
>device	pci
>device	fdc
>device	ahci
>device	ata
>device	mvs
>device	siis
>device	ahc
>device	ahd
>device	esp
>device	hptiop
>device	isp
>device	mpt
>device	mps
>device	mpr
>device	sym
>device	trm
>device	adv
>device	adw
>device	aic
>device	bt
>device	isci
>device	ocs_fc
>device	scbus
>device	ch
>device	da
>device	sa
>device	cd
>device	pass
>device	ses
>device	amr
>device	arcmsr
>device	ciss
>device	dpt
>device	hptmv
>device	hptnr
>device	hptrr
>device	hpt27xx
>device	iir
>device	ips
>device	mly
>device	twa
>device	smartpqi
>device	tws
>device	aac
>device	aacp
>device	aacraid
>device	ida
>device	mfi
>device	mlx
>device	mrsas
>device	pmspcv
>device	twe
>device	nvme
>device	nvd
>device	atkbdc
>device	atkbd
>device	psm
>device	kbdmux
>device	vga
>device	splash
>device	sc
>device	vt
>device	vt_vga
>device	vt_efifb
>device	agp
>device	cbb
>device	pccard
>device	cardbus
>device	uart
>device	ppc
>device	ppbus
>device	lpt
>device	ppi
>device	puc
>device	bxe
>device	de
>device	em
>device	ix
>device	ixv
>device	ixl
>device	le
>device	ti
>device	txp
>device	vx
>device	miibus
>device	ae
>device	age
>device	alc
>device	ale
>device	bce
>device	bfe
>device	bge
>device	cas
>device	dc
>device	et
>device	fxp
>device	gem
>device	hme
>device	jme
>device	lge
>device	msk
>device	nfe
>device	nge
>device	pcn
>device	re
>device	rl
>device	sf
>device	sge
>device	sis
>device	sk
>device	ste
>device	stge
>device	tl
>device	tx
>device	vge
>device	vr
>device	wb
>device	xl
>device	wlan
>device	wlan_wep
>device	wlan_ccmp
>device	wlan_tkip
>device	wlan_amrr
>device	an
>device	ath
>device	ath_pci
>device	ath_hal
>device	ath_rate_sample
>device	ipw
>device	iwi
>device	iwn
>device	malo
>device	mwl
>device	ral
>device	wi
>device	wpi
>device	loop
>device	random
>device	padlock_rng
>device	rdrand_rng
>device	ether
>device	vlan
>device	tun
>device	md
>device	gif
>device	firmware
>device	bpf
>device	uhci
>device	ohci
>device	ehci
>device	xhci
>device	usb
>device	ukbd
>device	umass
>device	sound
>device	snd_cmi
>device	snd_csa
>device	snd_emu10kx
>device	snd_es137x
>device	snd_hda
>device	snd_ich
>device	snd_via8233
>device	mmc
>device	mmcsd
>device	sdhci
>device	virtio
>device	virtio_pci
>device	vtnet
>device	virtio_blk
>device	virtio_scsi
>device	virtio_balloon
>device	hyperv
>device	xenpci
>device	vmx
>device	netmap
>device	crypto
>
>------------------------------------------------------------------------
>ddb capture buffer
>
>

Actions: View

Attachments on bug 230752: 196354 | 196722