Bug 188388 - [panic] double fault / syscall sys_sys_exit
Summary: [panic] double fault / syscall sys_sys_exit
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: i386 (show other bugs)
Version: 9.2-RELEASE
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-i386 mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-04-09 07:20 UTC by Andrew Belashov
Modified: 2017-12-31 22:27 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Belashov 2014-04-09 07:20:00 UTC
The kernel regularly crashes with "panic: double fault" in syscall sys_sys_exit.
Some information from core.txt.18 follows:

localhost dumped core - see /var/crash/vmcore.18

Wed Apr  9 06:55:25 MSK 2014

FreeBSD localhost 9.2-RELEASE-p3 FreeBSD 9.2-RELEASE-p3 #4 r260982: Mon Apr  7 14:58:17 MSK 2014     bel@localhost:/usr/obj/usr/src/sys/ENIGMA  i386

panic: double fault

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:

Fatal double fault:
eip = 0xc0f65d4c
esp = 0xf66b6000
ebp = 0xf66b6ba4
cpuid = 1; apic id = 06
panic: double fault
cpuid = 1
KDB: stack backtrace:
db_trace_self_wrapper(c1181b97,0,c1357708,c71e7400,0,...) at db_trace_self_wrapper+0x26/frame 0xc13576dc
kdb_backtrace(c11c81a8,1,c11c8a47,c1357770,1,...) at kdb_backtrace+0x2a/frame 0xc1357738
panic(c11c8a47,6,6,0,0,...) at panic+0x16f/frame 0xc1357764
dblfault_handler() at dblfault_handler+0xa6/frame 0xc1357764
--- trap 0x17, eip = 0xc0f65d4c, esp = 0xf66b6000, ebp = 0xf66b6ba4 ---
Xpage(f5eb7000,1,c83b6000,f66b6bd4,c0d967fb,...) at Xpage/frame 0xf66b6ba4
pmap_release(c83b60b0,1000,bfc00000,c989c2f0,0,...) at pmap_release+0x27d/frame 0xf66b6bd4
vmspace_exit(c989c2f0,c9528000,1,0,3,...) at vmspace_exit+0xcb/frame 0xf66b6bf4
exit1(c989c2f0,0,f66b6cfc,c0f7ca03,c989c2f0,...) at exit1+0x82a/frame 0xf66b6c38
sys_sys_exit(c989c2f0,f66b6ccc,c87468d0,c1324f80,b,...) at sys_sys_exit+0x1d/frame 0xf66b6c48
syscall(f66b6d08) at syscall+0x443/frame 0xf66b6cfc
Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf66b6cfc
--- syscall (1, FreeBSD ELF32, sys_sys_exit), eip = 0x282be8bf, esp = 0xbfbfcbec, ebp = 0xbfbfcbf8 ---
Uptime: 1d2h4m41s
Physical memory: 2027 MB
Dumping 233 MB: 218 202 186 170 154 138 122 106 90 74 58 42 26 10

Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /boot/kernel/nullfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nullfs.ko
#0  doadump (textdump=1) at pcpu.h:249
249	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) #0  doadump (textdump=1) at pcpu.h:249
#1  0xc0aebf85 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:449
#2  0xc0aec282 in panic (fmt=<value optimized out>)
    at /usr/src/sys/kern/kern_shutdown.c:637
#3  0xc0f7bde6 in dblfault_handler () at /usr/src/sys/i386/i386/trap.c:1077
#4  0xc0f65d4c in Xprot () at /usr/src/sys/i386/i386/exception.s:133
#5  0x00010006 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) 

------------------------------------------------------------------------
ps -axl

  UID   PID  PPID CPU PRI NI    VSZ     RSS MWCHAN   STAT TT        TIME COMMAND
    0     0     0   0 -92  0      0       0 -        DLs  ??   352:41.30 [kerne
    0     1     0   0  20  0   8036 2860048 wait     DLs  ??     0:03.32 [init]
    0     2     0   0 -16  0      0       0 idle     DL   ??     0:00.00 [ciss_
    0     3     0   0 -16  0      0       0 -        DL   ??     0:02.16 [fdc0]
    0     4     0   0 -16  0      0       0 waiting_ DL   ??     0:00.00 [sctp_
    0     5     0   0 -16  0      0       0 ccb_scan DL   ??     0:00.00 [xpt_t
    0     6     0   0 -16  0      0       0 psleep   DL   ??     0:14.31 [paged
    0     7     0   0 -16  0      0       0 psleep   DL   ??     0:00.80 [vmdae
    0     8     0   0 155  0      0       0 pgzero   DL   ??     0:00.00 [pagez
    0     9     0   0 -16  0      0       0 psleep   DL   ??     0:04.13 [bufda
    0    10     0   0 -16  0      0       0 audit_wo DL   ??     0:00.00 [audit
    0    11     0   0 155  0      0       0 -        RL   ??  1900:49.21 [idle]
    0    12     0   0 -76  0      0       0 -        WL   ??    19:20.25 [intr]
    0    13     0   0  -8  0      0       0 -        DL   ??     2:53.62 [geom]
    0    14     0   0 -16  0      0       0 -        DL   ??     3:59.18 [yarro
    0    15     0   0 -68  0      0       0 -        DL   ??     0:02.54 [usb]
    0    16     0   0 -16  0      0       0 tzpoll   DL   ??     0:00.78 [acpi_
    0    17     0   0 -16  0      0       0 cooling  DL   ??     0:00.11 [acpi_
    0    18     0   0 -16  0      0       0 vlruwt   DL   ??     0:06.83 [vnlru
    0    19     0   0  16  0      0       0 syncer   DL   ??     1:03.83 [synce
    0    20     0   0 -16  0      0       0 sdflush  DL   ??     0:14.21 [softd
    0  1623     1   0  20  0  12132 1789520 select   Ds   ??     0:00.04 [devd]
    0  1917     1   0  20  0   9544 1787664 select   Ds   ??     1:41.06 [syslo
    0  1920     1   0 -52  0   9540  553616 nanslp   Ds   ??     0:05.61 [watch
   53  2067     1   0  20  0  44304 1985200 kqread   Ds   ??     1:01.14 [named
  801  2255     1   0  52  0   9852 2865616 wait     D    ??     0:00.34 [sh]
  801  2257     1   0  52  0   9852 1790448 wait     D    ??     0:00.36 [sh]
    0  2367     1   0  20  0  14388 1288112 uwait    Ds   ??     0:00.13 [bacul
    0  2613     1   0  26  0  13096 1284400 select   Ds   ??     0:00.00 [sshd]
   25  2619     1   0  20  0  11480 2864688 pause    Ds   ??     0:00.04 [sendm
    0  2623     1   0  20  0   9584 1986128 nanslp   Ds   ??     0:00.28 [cron]
    0  2847     1   0  20  0   9544  546192 select   Ds   ??     0:18.89 [syslo
   53  2853     1   0  20  0 142608  554544 kqread   Ds   ??    93:35.86 [named
    0  2905     1   0  52  0  13096  542480 select   Ds   ??     0:00.00 [sshd]
   25  2908     1   0  20  0  11480 2838096 pause    Ds   ??     0:00.04 [sendm
    0  2912     1   0  20  0   9584 2836240 nanslp   Ds   ??     0:00.27 [cron]
    0  3078     1   0  20  0   9544 1991696 select   Ds   ??     0:00.27 [syslo
   88  3129     1   0  52  0   9852 2840880 wait     DWs  ??     0:00.00 [sh]
    0  3144     1   0  52  0  13096 1791376 select   Ds   ??     0:00.00 [sshd]
   25  3169     1   0  20  0  11480 1797872 pause    Ds   ??     0:00.04 [sendm
    0  3182     1   0  20  0   9584 1993552 nanslp   Ds   ??     0:00.27 [cron]
   88  3464  3129   0  20  0 460568 1998192 sbwait   D    ??    20:47.57 [mysql
   58  3500 90598   0  25  0  57816 3474288 select   D    ??     6:01.61 [perl]
   58  3501 90598   0  25  0  53720 3403504 select   D    ??     1:44.78 [perl]
    0  3639     1   0  20  0   9544 2853552 select   Ds   ??     0:45.92 [syslo
    0  3686     1   0  52  0  10816 2848304 kqread   Ds   ??     0:00.00 [nscd]
    0  3697     1   0  20  0  41288 1995408 select   Ds   ??     8:30.76 [spama
    0  3704     1   0  20  0  13228 1290896 lockf    Ds   ??     0:11.22 [sasla
    0  3705  3704   0  20  0  13228 1798800 accept   D    ??     0:11.10 [sasla
    0  3706  3704   0  20  0  13228 2862832 lockf    D    ??     0:10.76 [sasla
    0  3708  3704   0  20  0  13228 2985552 lockf    D    ??     0:11.31 [sasla
    0  3709  3704   0  20  0  13228  541552 lockf    D    ??     0:11.28 [sasla
  785  3722     1   0  20  0 133016 2845520 select   Ds   ??     4:07.87 [perl5
    0  3746     1   0  30  0  11072 2971024 kqread   Ds   ??     1:58.02 [dovec
  143  3748  3746   0  20  0  11072 2970096 kqread   D    ??     0:28.98 [anvil
    0  3749  3746   0  20  0  11072 2968240 kqread   D    ??     0:27.49 [log]
    0  3752  3746   0  20  0  11072 2974736 kqread   D    ??     2:16.02 [confi
  143  3755  3746   0  20  0  11276 1793232 kqread   D    ??     1:20.98 [auth]
  106  3997     1   0  21  0 308044 2978448 uwait    Ds   ??    22:40.59 [clamd
  106  4005     1   0  20  0  40252 1796944 select   Ds   ??     2:35.60 [clama
  106  4009     1   0  52  0  22952 1285328 pause    Ds   ??     0:58.07 [fresh
    0  4015     1   0  52  0  13096 1992624 select   Ds   ??     0:00.00 [sshd]
    0  4018     1   0  20  0  14476 1987984 select   Ds   ??     1:15.53 [sendm
   25  4021     1   0  20  0  11608 2975664 pause    Ds   ??     0:00.03 [sendm
    0  4025     1   0  20  0   9584 1983344 nanslp   Ds   ??     0:00.32 [cron]
    0  4200     1   0  20  0   9544 2837168 select   Ds   ??     0:00.27 [syslo
    0  4256     1   0  30 10  67268 1990768 select   DNs  ??     0:12.63 [httpd
    0  4263     1   0  52  0  13096 1286256 select   Ds   ??     0:00.00 [sshd]
    0  4267     1   0  20  0   9584 2989264 nanslp   Ds   ??     0:00.27 [cron]
    0  4435     1   0  20  0   9544  545264 select   Ds   ??     0:00.27 [syslo
   80  4477  4256   0  30 10  79816  552688 sbwait   DN   ??     9:22.67 [httpd
    0  4478     1   0  20  0  62960 2991120 select   Ds   ??     0:03.38 [httpd
   80  4479  4256   0  31 10  75716 2997616 sbwait   DN   ??    10:08.72 [httpd
   80  4480  4256   0  30 10  79688 2990192 sbwait   DN   ??     9:46.00 [httpd
   80  4481  4256   0  30 10  79688 3017392 sbwait   DN   ??     9:52.88 [httpd
   80  4482  4256   0  30 10  75588 2972880 sbwait   DN   ??     9:50.03 [httpd
    0  4488     1   0  52  0  13096 3023888 select   Ds   ??     0:00.00 [sshd]
    0  4492     1   0  20  0   9584 2973808 nanslp   Ds   ??     0:00.27 [cron]
    0  4659     1   0  20  0   9544 2857264 select   Ds   ??     0:02.36 [syslo
65534  4686     1   0  20  0   9720 2979376 select   Ds   ??     0:06.25 [proft
    0  4706     1   0  52  0  13096 3019248 select   Ds   ??     0:00.00 [sshd]
   81  4708  4478   0  20  0  67056 2855408 accept   D    ??     0:00.42 [httpd
   81  4709  4478   0  52  0  62960 2995760 accept   DW   ??     0:00.00 [httpd
   81  4710  4478   0  20  0  67056 2986480 accept   D    ??     0:00.33 [httpd
   81  4711  4478   0  52  0  62960 1279760 accept   DW   ??     0:00.00 [httpd
   81  4712  4478   0  52  0  62960 2852624 accept   DW   ??     0:00.00 [httpd
   80  4714  4256   0  30 10  79684  548048 sbwait   DN   ??     9:57.19 [httpd
   25  4715     1   0  20  0  11608  543408 pause    Ds   ??     0:00.03 [sendm
    0  4719     1   0  20  0   9584 2992976 nanslp   Ds   ??     0:00.27 [cron]
    0  4759     1   0  20  0  11256 3022960 select   Ds   ??     0:08.38 [ntpd]
    0  4775     1   0  52  0   9608 2846448 select   Ds   ??     0:00.00 [inetd
    0  4792     1   0  20  0  11272 3024816 select   Ds   ??     0:01.09 [bsnmp
    0  4800     1   0  52  0   9544 1280688 ttyin    Ds+  ??     0:00.00 [getty
    0  4801     1   0  52  0   9544  555472 ttyin    Ds+  ??     0:00.00 [getty
    0  4802     1   0  52  0   9544 1283472 ttyin    Ds+  ??     0:00.00 [getty
    0  4803     1   0  52  0   9544 3022032 ttyin    Ds+  ??     0:00.00 [getty
    0  4804     1   0  52  0   9544 1281616 ttyin    Ds+  ??     0:00.00 [getty
    0  4805     1   0  52  0   9544 1989840 ttyin    Ds+  ??     0:00.00 [getty
    0  4806     1   0  52  0   9544 2987408 ttyin    Ds+  ??     0:00.00 [getty
    0  4807     1   0  52  0   9544 2994832 ttyin    Ds+  ??     0:00.00 [getty
    0  4808     1   0  52  0   9544 2839024 ttyin    Ds+  ??     0:00.00 [getty
   80  4811  4256   0  33 10  79812 3018320 lockf    DN   ??     9:50.97 [httpd
   80  4812  4256   0  30 10  75588 3020176 lockf    DN   ??     9:43.11 [httpd
   80  4817  4256   0  30 10  75588 1792304 sbwait   DN   ??     9:22.27 [httpd
   80  4828  4256   0  30 10  75588 3499952 select   DN   ??     9:23.41 [httpd
   81  5824  4478   0  20  0  67056  691792 accept   D    ??     0:00.31 [httpd
   81  6001  4478   0  20  0  67056 2847376 accept   D    ??     0:00.39 [httpd
   81  6002  4478   0  20  0  67056 3410608 accept   D    ??     0:00.38 [httpd
    0 13892  3746   0  20  0  12812 3413392 kqread   D    ??     0:00.05 [ssl-p
   80 18975  4256   0  31 10  71364 1795088 sbwait   DN   ??     0:29.06 [httpd
    0 21761  4018   0  20  0  14476 3486352 connec   D    ??     0:00.90 [sendm
    0 26054  4018   0  20  0  14476   22832 connec   D    ??     0:00.57 [sendm
32950 26548  4686   0  20  0  10020  548976 select   D    ??     0:00.04 [proft
   80 26556  4256   0  30 10  67268 3503664 sbwait   DN   ??     0:15.71 [httpd
   80 28226  4256   0  30 10  67268  734768 sbwait   DN   ??     0:10.25 [httpd
   81 30151  4478   0  20  0  67056 2844592 accept   D    ??     0:00.26 [httpd
    0 30188  4018   0  20  0  14476 2983696 connec   D    ??     0:00.35 [sendm
  144 31778  3746   0  20  0  12968  133808 kqread   D    ??     0:00.02 [pop3-
33471 31779  3746   0  20  0  11908 2988336 kqread   D    ??     0:00.01 [pop3]
    0 32184  4018   0  20  0  14476 3479856 select   D    ??     0:00.02 [sendm
    0 32185  4018   0  20  0  14476 3016464 select   D    ??     0:00.01 [sendm
   80 32201  4256   0  30 10  67268  693648 sbwait   DN   ??     0:02.69 [httpd
    0 32603  4018   0  20  0  14476  142160 select   D    ??     0:00.07 [sendm
    0 32661  2623   0  21  0   9584  194096 piperd   D    ??     0:00.00 [cron]
    0 32663 32661   0  32 10  10752 1293680 wait     DNs  ??     0:00.01 [bash]
    0 32673 32663   0  52 10  10752  143088 wait     DN   ??     1:24.51 [bash]
    0 32674 32663   0  30 10   9456 3418032 piperd   DN   ??     0:00.01 [mail]
32950 35664  4686   0  20  0  10020 2252912 select   D    ??     0:00.02 [proft
   80 39218  4256   0  31 10  75588 3026672 sbwait   DN   ??     2:36.96 [httpd
32950 39627  4686   0  20  0  10020 2251056 select   D    ??     0:00.02 [proft
32950 48195  4686   0  20  0  10020  549904 select   D    ??     0:00.02 [proft
   80 48697  4256   0  30 10  75716  747440 sbwait   DN   ??     7:37.82 [httpd
32950 49952  4686   0  20  0  10020 3422672 select   D    ??     0:00.02 [proft
32950 51075  4686   0  20  0  10020  188528 select   D    ??     0:00.02 [proft
32950 52088  4686   0  20  0  10020 3925936 select   D    ??     0:00.31 [proft
32950 52421  4686   0  20  0  10020  701072 select   D    ??     0:00.02 [proft
32950 53336  4686   0  20  0  10020 3494384 select   D    ??     0:00.02 [proft
32950 54302  4686   0  20  0  10020 1941616 select   D    ??     0:00.02 [proft
32950 54782  4686   0  20  0  10020 2191088 select   D    ??     0:00.02 [proft
  801 61449  2257   0  52  0   1560  722704 nanslp   D    ??     0:00.00 [sleep
 3596 63519  3746   0  20  0  11912  731056 kqread   D    ??     0:00.57 [pop3]
  801 64836  2255   0  52  0   1560 2188304 nanslp   D    ??     0:00.00 [sleep
   80 65467  4256   0  30 10  67268 3398864 sbwait   DN   ??     0:01.03 [httpd
   80 65591  4256   0  31 10  67268 3484496 sbwait   DN   ??     0:01.00 [httpd
    0 65800  4018   0  22  0  14476 3930576 select   D    ??     0:00.00 [sendm
    0 68507  3746   0  20  0  11276  146800 kqread   D    ??     0:00.00 [auth]
32950 69725  4686   0  20  0  10020 2245488 select   D    ??     0:00.00 [proft
   80 73803  4256   0  30 10  67268  745584 sbwait   DN   ??     0:01.05 [httpd
   80 73934  4256   0  30 10  67268 2981232 sbwait   DN   ??     0:00.00 [httpd
   80 73935  4256   0  30 10  67268  219440 lockf    DN   ??     0:01.08 [httpd
32950 74403  4686   0  20  0  10020   83728 select   D    ??     0:00.00 [proft
 1576 75337  3746   0  20  0  11912   87440 kqread   D    ??     0:00.00 [pop3]
  144 75559  3746   0  20  0  12968  550832 kqread   D    ??     0:00.00 [pop3-
15884 75659  3746   0  20  0  11884  192240 kqread   D    ??     0:00.00 [pop3]
32950 77833  4686   0  20  0  10020 2996688 select   D    ??     0:00.00 [proft
 5319 80175  3746   0  20  0  11924 2109168 kqread   D    ??     0:00.37 [pop3]
   80 81022  4256   0  30 10  67268 1935120 lockf    DN   ??     0:00.10 [httpd
 7751 81954  3746   0  20  0  11908  675408 kqread   D    ??     0:00.55 [pop3]
 3056 81956  3746   0  20  0  11912   21904 kqread   D    ??     0:00.55 [pop3]
   80 82164  4256   0  30 10  71364 3481712 sbwait   DN   ??     0:00.13 [httpd
   80 88857  4256   0  30 10  67268 3417104 lockf    DN   ??     0:00.00 [httpd
    0 89227  4018   0  21  0  14476 1557536 select   D    ??     0:00.00 [sendm
    0 89233 32673   0  98 10      0       0 -        RNE  ??     0:00.00 [bash]
    0 89926  2613   0  20  0  15852  742800 select   Ds   ??     0:00.35 [sshd]
 1005 89947 89926   0  21  0  15852 3487280 select   D    ??     0:03.74 [sshd]
    0 89948 89947   0  20  0  10088 3392368 wait     DWs  ??     0:00.00 [su]
    0 90018 89948   0  20  0  10752 2984624 ttyin    D+   ??     0:00.04 [bash]
    0 90598     1   0  20  0  41432 1987056 select   Ds   ??     0:07.73 [perl]
   80 93409  4256   0  30 10  75588  731984 sbwait   DN   ??     6:58.60 [httpd
   81 99797  4478   0  20  0  67056   92080 accept   D    ??     0:00.30 [httpd

------------------------------------------------------------------------
vmstat -s

2972408779 cpu context switches
588568432 device interrupts
111601679 software interrupts
214804395 traps
1606353512 system calls
       20 kernel threads created
   739539  fork() calls
    48250 vfork() calls
        0 rfork() calls
      394 swap pager pageins
      693 swap pager pages paged in
     7724 swap pager pageouts
    50113 swap pager pages paged out
   143716 vnode pager pageins
   967096 vnode pager pages paged in
    10487 vnode pager pageouts
    10902 vnode pager pages paged out
      468 page daemon wakeups
  7961594 pages examined by the page daemon
   137087 pages reactivated
 57584522 copy-on-write faults
   136884 copy-on-write optimized faults
 63122861 zero fill pages zeroed
  2435180 zero fill pages prezeroed
    14092 intransit blocking page faults
150920209 total VM faults taken
        0 pages affected by kernel thread creation
695477044 pages affected by  fork()
 46156572 pages affected by vfork()
        0 pages affected by rfork()
        0 pages cached
155024194 pages freed
        0 pages freed by daemon
        0 pages freed by exiting processes
   237349 pages active
   188328 pages inactive
    19767 pages in VM cache
    61262 pages wired down
     2876 pages free
     4096 bytes per page
221737715 total name lookups
          cache hits (89% pos + 6% neg) system 0% per-directory
          deletions 0%, falsehits 0%, toolong 0%

------------------------------------------------------------------------
vmstat -m

         Type InUse MemUse HighUse Requests  Size(s)
     acpiintr     1     1K       -        1  32
       USBdev    20     2K       -       20  32,64,128
       isadev     8     1K       -        8  64
          USB    35    12K       -       35  16,32,64,2048
       acpica  1219    60K       -    91218  16,32,64,128,256,512,1024
         cdev     8     1K       -        8  128
        sigio    15     1K       -    14205  32
     filedesc   270   119K       -  1356836  16,32,64,128,256,512,1024,2048,4096
      kdtrace   889   191K       -  1050931  64,256
         kenv    83     7K       -       88  16,32,64,128,4096
       kqueue    38    28K       -  1833885  128,256,1024,2048,4096
      entropy  1024    64K       -     1024  64
    proc-args   109     8K       -  2062457  16,32,64,128,256
        hhook     2     1K       -        2  128
      ithread   178    19K       -      178  16,64,128
       prison    12    13K       -       12  16,2048
    ciss_data    14    16K       -       16  16,128,256,512,1024,2048,4096
       KTRACE   100    13K       -      100  128
       kbdmux     6    18K       -        6  16,256,1024,2048
       linker   174    34K       -      188  16,32,256,1024,4096
        lockf   194    10K       - 54572695  32,64,128,256,512
   loginclass     2     1K       -     3712  64
       ip6ndp    16     2K       -       18  64,128
       ip6opt     0     0K       -       76  256
         temp    42   225K       -  7624434  16,32,64,128,256,512,1024,2048,4096
       devbuf  2370  1601K       -    10026  16,32,64,128,256,512,1024,2048,4096
          LED     4     1K       -        4  16,64
       module   535    34K       -      535  64,128
     mtx_pool     2     8K       -        2  4096
     pmchooks     1     1K       -        1  64
      subproc   787   835K       -   788429  256,4096
         proc     2     8K       -        2  4096
      session    62     4K       -    36007  64
         pgrp    63     4K       -    36051  64
         cred   684    65K       - 22924338  64,128
      uidinfo    23     3K       -   241928  64,1024
       plimit    35     9K       -   228376  256
      ata_pci     1     1K       -        1  32
     acpitask     1     2K       -        1  2048
    sysctltmp     0     0K       -   528618  16,32,64,128
    sysctloid  4097   128K       -     4180  16,32,64,128
       sysctl     0     0K       -   204754  16,32,64
      tidhash     1     8K       -        1  
      callout     1   256K       -        1  
         umtx  2060   194K       -     3070  64,128
     p1003.1b     1     1K       -        1  16
         SWAP     2   277K       -        2  64
       bus-sc    72   246K       -     3465  16,64,128,256,512,1024,2048,4096
          bus  1212    53K       -     5831  16,32,64,128,256,1024
      devstat     8    17K       -        8  16,4096
 eventhandler    94     5K       -       94  32,64,128
       feeder     7     1K       -        7  16
         kobj   375   750K       -      552  2048
CAM dev queue     4     1K       -        4  128
      Per-cpu     1     1K       -        1  16
      acpisem    19     3K       -       19  64,128
       DEVFS1   103    26K       -      109  256
       DEVFS3   952   119K       -      962  128
         rman   220    13K       -      668  16,32,64
       DEVFS2   103     4K       -      717  16,32
         sbuf     0     0K       -     1532  16,32,64,128,256,512,1024,2048,4096
   DEVFS_RULE    55    13K       -      107  32,256
        DEVFS   112     3K       -      113  16,64
        stack     0     0K       -        2  128
    taskqueue    21     1K       -       21  16,64
       Unitno    28     2K       -  1353611  16,64
          iov     0     0K       - 70132091  16,64,128,256
       select   892    56K       -     1368  64
     ioctlops     0     0K       -  1612374  16,32,64,128,256,512,1024,2048,4096
          msg     4    25K       -        4  1024,4096
          sem     4   101K       -        4  1024,4096
          shm     1    12K       -        1  
          tty    20    10K       -       22  512,1024
          pts     1     1K       -        1  128
         accf     4     1K       -        4  16,32
     mbuf_tag     0     0K       -       73  32
        shmfd     1     4K       -        1  4096
          pcb   117    80K       -  6481382  16,64,512,1024,2048,4096
       soname    54     5K       - 45379046  16,32,128
          acl     0     0K       -     1905  4096
       biobuf     0     0K       -    69446  2048
     vfscache     1   512K       -        1  
   cl_savebuf     0     0K       -   302161  32,64
     vfs_hash     1   256K       -        1  
       vnodes     2     1K       -        2  128
        mount   870    33K       -     1300  16,32,64,128,256,1024
  vnodemarker     0     0K       -    69945  512
          BPF    12     1K       -       12  64
  ether_multi    86     4K       -      102  16,32,64
       ifaddr   133    23K       -      133  16,32,64,128,256,512,2048
        ifnet    13    13K       -       13  64,1024
        clone     6    24K       -        8  16,128,4096
       arpcom     6     1K       -        6  16
      lltable    35     8K       -      117  128,256
         vlan     7     1K       -        7  64
     routetbl   110    12K       -     1093  16,32,64,128,256,512
         igmp    12     2K       -       12  128
     in_multi     2     1K       -        2  128
    sctp_iter     0     0K       -       11  256
     sctp_ifn     4     1K       -        4  128
     sctp_ifa    13     2K       -       13  128
     sctp_vrf     1     1K       -        1  64
    sctp_a_it     0     0K       -       11  16
    hostcache     1    16K       -        1  
     syncache     1    72K       -        1  
    in6_multi    45     6K       -       45  16,256
          mld    12     2K       -       12  128
     dummynet     3     2K       -        3  512
  IpFw/IpAcct    89    39K       -      172  16,32,64,128,256,512
     ipfw_tbl     3     1K       -        3  128
      NFS FHA     1     1K       -        1  1024
          rpc     2     1K       -        2  128
audit_evclass   180     3K       -      219  16
     savedino     0     0K       -    77317  256
     freework    13     2K       -   712881  16,128
    newdirblk     0     0K       -    69497  32
       dirrem    17     2K       -   498753  64
        mkdir     0     0K       -   138892  64
       diradd     4     1K       -   499431  64
     freefile    15     1K       -   412008  32
     freeblks    10     2K       -   412023  128
     freefrag     1     1K       -  1446595  64
     indirdep     3     1K       -    36968  64
       newblk    11    66K       -  4002716  128
    bmsafemap     9     5K       -   525579  128,4096
     inodedep    46   268K       -   619633  256
      pagedep    13    66K       -   223428  128
  ufs_dirhash   245   128K       -   260335  16,32,64,128,256,512,1024,2048,4096
    ufs_quota     1   256K       -        1  
    ufs_mount    19    58K       -       40  64,256,1024,2048,4096
    vm_pgdata     2    65K       -        2  64
      UMAHash     2     5K       -        7  256,512,1024,2048,4096
    pfs_nodes    21     3K       -       21  128
     atkbddev     2     1K       -        2  32
         GEOM   128    30K       -      742  16,32,64,128,256,512,1024,2048
     CAM path     9     1K       -      321  16
      CAM CCB   351   702K       -      480  2048
      scsi_cd     0     0K       -       10  16
      CAM DEV     6    12K       -      204  2048
      CAM XPT    31     2K       -      295  16,32,64,256,1024,2048
      scsi_da     0     0K       -      144  16
    CAM queue    18     5K       -     1217  16,32,64,128,256,2048
      acpidev    28     1K       -       28  32
      CAM SIM     6     1K       -        6  128
      memdesc     1     4K       -        1  4096
    raid_data     0     0K       -      114  16,128
         UART     6     3K       -        6  16,256,1024
     pci_link    16     2K       -       16  16,128
       apmdev     1     1K       -        1  64
md_nvidia_data     0     0K       -       18  512
   madt_table     0     0K       -        1  2048
  md_sii_data     0     0K       -       18  512
  ddb_capture     1    48K       -        1  
      io_apic     5     5K       -        5  1024
          MCA     4     1K       -        4  128
     nexusdev     4     1K       -        4  16
   CAM periph     8     1K       -      118  16,32,64,128
  nullfs_node  1093    35K       -  1210711  32
  nullfs_hash     1   256K       -        1  
 nullfs_mount    42     1K       -       42  16

------------------------------------------------------------------------
vmstat -z

ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               128,      0,      90,       0,      90,   0,   0
UMA Zones:              176,      0,      90,      10,      90,   0,   0
UMA Slabs:              284,      0,    1922,     192, 2174457,   0,   0
UMA RCntSlabs:          544,      0,    2092,      78,  167402,   0,   0
UMA Hash:               128,      0,       1,      29,       3,   0,   0
16 Bucket:               76,      0,       7,      93,     107,   0,   0
32 Bucket:              140,      0,      14,      70,     331,   0,   0
64 Bucket:              268,      0,      15,      41,     986,  11,   0
128 Bucket:             524,      0,     228,     206,   77727,42572,   0
VM OBJECT:              148,      0,   45403,   32155,21646187,   0,   0
MAP:                    140,      0,       8,      48,       8,   0,   0
KMAP ENTRY:              72,  57505,     342,    4004,11245721,   0,   0
MAP ENTRY:               72,      0,   11854,   12102,45842674,   0,   0
fakepg:                  72,      0,       0,       0,       0,   0,   0
mt_zone:               2060,      0,     332,      41,     332,   0,   0
16:                      16,      0,    3980,    1095,114470938,   0,   0
32:                      32,      0,    4053,    9846,17150155,   0,   0
64:                      64,      0,    6877,    1324,65101050,   0,   0
128:                    128,      0,    4876,    4604,22695902,   0,   0
256:                    256,      0,    2228,     697, 2048181,   0,   0
512:                    512,      0,     302,     546,  811439,   0,   0
1024:                  1024,      0,      78,     394, 4543463,   0,   0
2048:                  2048,      0,     778,     300,  165292,   0,   0
4096:                  4096,      0,     226,     383, 1111461,   0,   0
Files:                   56,      0,    1106,    1038,24061924,   0,   0
TURNSTILE:               72,      0,    1031,     289,    1536,   0,   0
rl_entry:                28,      0,     574,     188,     574,   0,   0
umtx pi:                 52,      0,       0,       0,       0,   0,   0
MAC labels:              20,      0,       0,       0,       0,   0,   0
PROC:                   724,      0,     169,     446,  787809,   0,   0
THREAD:                 752,      0,     718,     312,  263120,   0,   0
SLEEPQUEUE:              44,      0,    1031,     326,    1536,   0,   0
VMSPACE:                232,      0,     150,     377,  787790,   0,   0
cpuset:                  40,      0,      79,     197,      85,   0,   0
audit_record:           824,      0,       0,       0,       0,   0,   0
mbuf_packet:            256,      0,    1023,     622,879570562,   0,   0
mbuf:                   256,      0,     904,     610,4009153898,   0,   0
mbuf_cluster:          2048,  25600,    1411,     283,   90371,   0,   0
mbuf_jumbo_page:       4096,  12800,     873,     372,550903050,   0,   0
mbuf_jumbo_9k:         9216,  19200,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384,  12800,       0,       0,       0,   0,   0
mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   0
g_bio:                  152,      0,       0,    1196,15474785,   0,   0
ttyinq:                 152,      0,     195,      65,     345,   0,   0
ttyoutq:                256,      0,     103,      17,     183,   0,   0
ata_request:            208,      0,       0,     342,   31324,   0,   0
ata_composite:          180,      0,       0,       0,       0,   0,   0
vtnet_tx_hdr:            16,      0,       0,       0,       0,   0,   0
VNODE:                  284,      0,   69296,   29796, 7299305,   0,   0
VNODEPOLL:               60,      0,       0,       0,       0,   0,   0
NAMEI:                 1024,      0,       0,     312,87350294,   0,   0
S VFS Cache:             72,      0,   72304,   35339, 6384222,   0,   0
STS VFS Cache:           92,      0,       0,       0,       0,   0,   0
L VFS Cache:            292,      0,     268,     564,  137957,   0,   0
LTS VFS Cache:          312,      0,       0,       0,       0,   0,   0
DIRHASH:               1024,      0,    1375,     249,  304918,   0,   0
NCLNODE:                376,      0,       0,       0,       0,   0,   0
pipe:                   400,      0,      61,     319,  646606,   0,   0
Mountpoints:            676,      0,      56,      16,      56,   0,   0
ksiginfo:                80,      0,     603,     357,  124029,   0,   0
itimer:                 220,      0,       1,      35,       3,   0,   0
KNOTE:                   72,      0,     153,     642,96892315,   0,   0
socket:                 416,  25605,     329,    1174, 8563276,   0,   0
unpcb:                  172,  25622,     109,     420, 1209676,   0,   0
ipq:                     32,    904,       0,     339,   38679,   0,   0
udp_inpcb:              252,  25605,      29,     691, 6443554,   0,   0
udpcb:                    8,  25781,      29,     783, 6443554,   0,   0
tcp_inpcb:              252,  25605,     304,    1166,  909317,   0,   0
tcpcb:                  688,  25600,     153,     602,  909317,   0,   0
tcptw:                   52,   5184,     151,    1145,  441933,   0,   0
syncache:               120,  15360,       0,     192,  560088,   0,   0
hostcache:               76,  15400,    1178,    1672,   29269,   0,   0
tcpreass:                20,   1690,       0,     676,  179057,   0,   0
sackhole:                20,      0,       0,     507,16070165,   0,   0
sctp_ep:                936,  25600,       0,       0,       0,   0,   0
sctp_asoc:             1504,  40000,       0,       0,       0,   0,   0
sctp_laddr:              24,  80040,       0,     290,      12,   0,   0
sctp_raddr:             496,  80000,       0,       0,       0,   0,   0
sctp_chunk:              96, 400000,       0,       0,       0,   0,   0
sctp_readq:              76, 400000,       0,       0,       0,   0,   0
sctp_stream_msg_out:     68, 400008,       0,       0,       0,   0,   0
sctp_asconf:             24, 400055,       0,       0,       0,   0,   0
sctp_asconf_ack:         24, 400055,       0,       0,       0,   0,   0
ripcb:                  252,  25605,       0,      75,     717,   0,   0
rtentry:                108,      0,      73,      71,      73,   0,   0
IPFW dynamic rule:      108,  10008,       0,     216,     495,   0,   0
divcb:                  252,  25605,       0,       0,       0,   0,   0
selfd:                   28,      0,    1407,     498,30981920,   0,   0
SWAPMETA:               276, 131082,     231,     147,  714850,   0,   0
FFS inode:              116,      0,   68144,   32704, 6087668,   0,   0
FFS1 dinode:            128,      0,       0,       0,       0,   0,   0
FFS2 dinode:            256,      0,   68144,   29926, 6087660,   0,   0


------------------------------------------------------------------------
vmstat -i

interrupt                          total       rate
irq1: atkbd0                           8          0
irq4: uart0                            8          0
irq14: ata0                        93932       1806
irq19: uhci1                     4395093      84521
irq25: bge0                        10666        205
irq51: ciss0                     3853869      74112
irq78: em0                     580214856   11157978
cpu0:timer                     105383166    2026599
cpu1:timer                     105555870    2029920
Total                          799507468   15375143

------------------------------------------------------------------------
pstat -T

1106/12328 files
4M/1999M swap space

------------------------------------------------------------------------
pstat -s

Device          512-blocks     Used    Avail Capacity
/dev/da0s1b        4095744     8664  4087080     0%

------------------------------------------------------------------------
iostat

iostat: kvm_read(_tk_nin): invalid address (0x0)
iostat: disabling TTY statistics
             da0              da1              cd0             cpu
  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
 23.20 60581 1372.42  36.24 13045 461.71   0.00   1  0.00  14  2 22  1 61

------------------------------------------------------------------------
ipcs -a

Message Queues:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME   

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME   

Semaphores:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME   


------------------------------------------------------------------------
ipcs -T

msginfo:
	msgmax:        16384	(max characters in a message)
	msgmni:           40	(# of message queues)
	msgmnb:         2048	(max characters in a message queue)
	msgtql:           40	(max # of messages in system)
	msgssz:            8	(size of a message segment)
	msgseg:         2048	(# of message segments in system)

shminfo:
	shmmax:    536870912	(max shared memory segment size)
	shmmin:            1	(min shared memory segment size)
	shmmni:          192	(max number of shared memory identifiers)
	shmseg:          128	(max shared memory segments per process)
	shmall:       131072	(max amount of shared memory in pages)

seminfo:
	semmni:           50	(# of semaphore identifiers)
	semmns:          340	(# of semaphores in system)
	semmnu:          150	(# of undo structures in system)
	semmsl:          340	(max # of semaphores per id)
	semopm:          100	(max # of operations per semop call)
	semume:           50	(max # of undo entries per process)
	semusz:          616	(size in bytes of undo structure)
	semvmx:        32767	(semaphore maximum value)
	semaem:        16384	(adjust on exit max value)


------------------------------------------------------------------------
netstat -s

tcp:
	1344762042 packets sent
		1330051305 data packets (2794598390 bytes)
		3556716 data packets (733539251 bytes) retransmitted
		11387 data packets unnecessarily retransmitted
		11 resends initiated by MTU discovery
		9820577 ack-only packets (2089233 delayed)
		0 URG only packets
		70 window probe packets
		281140 window update packets
		1055352 control packets
	697563565 packets received
		653407961 acks (for 2754784958 bytes)
		31097110 duplicate acks
		278 acks for unsent data
		17475574 packets (1263754926 bytes) received in-sequence
		47250 completely duplicate packets (7908406 bytes)
		26561 old duplicate packets
		1627 packets with some dup. data (1126363 bytes duped)
		178836 out-of-order packets (240515401 bytes)
		101 packets (140405 bytes) of data after window
		0 window probes
		1001017 window update packets
		11952 packets received after close
		70 discarded for bad checksums
		0 discarded for bad header offset fields
		0 discarded because packet too short
		660 discarded due to memory problems
	325084 connection requests
	559084 connection accepts
	8 bad connection attempts
	392 listen queue overflows
	69786 ignored RSTs in the windows
	878193 connections established (including accepts)
	909013 connections closed (including 57388 drops)
		726117 connections updated cached RTT on close
		727515 connections updated cached RTT variance on close
		95341 connections updated cached ssthresh on close
	5811 embryonic connections dropped
	649828138 segments updated rtt (of 637289450 attempts)
	125108 retransmit timeouts
		1763 connections dropped by rexmit timeout
	104 persist timeouts
		0 connections dropped by persist timeout
	0 Connections (fin_wait_2) dropped because of timeout
	4018 keepalive timeouts
		69 keepalive probes sent
		3949 connections dropped by keepalive
	553157640 correct ACK header predictions
	10703244 correct data packet header predictions
	560088 syncache entries added
		3203 retransmitted
		2704 dupsyn
		0 dropped
		559084 completed
		0 bucket overflow
		0 cache overflow
		363 reset
		305 stale
		392 aborted
		0 badack
		5 unreach
		0 zone failures
	560088 cookies sent
	64 cookies received
	29289 hostcache entries added
		20 bucket overflow
	533427 SACK recovery episodes
	3319097 segment rexmits in SACK recovery episodes
	438777876 byte rexmits in SACK recovery episodes
	36271961 SACK options (SACK blocks) received
	159294 SACK options (SACK blocks) sent
	0 SACK scoreboard overflow
	0 packets with ECN CE bit set
	0 packets with ECN ECT(0) bit set
	0 packets with ECN ECT(1) bit set
	0 successful ECN handshakes
	0 times ECN reduced the congestion window
udp:
	16428995 datagrams received
	0 with incomplete header
	0 with bad data length field
	9 with bad checksum
	132967 with no checksum
	36420 dropped due to no socket
	0 broadcast/multicast datagrams undelivered
	0 dropped due to full socket buffers
	0 not for hashed pcb
	16392566 delivered
	18261421 datagrams output
	0 times multicast source filter matched
ip:
	714837936 total packets received
	0 bad header checksums
	0 with size smaller than minimum
	0 with data size < data length
	0 with ip length > max ip packet size
	0 with header length < data size
	0 with data length < header length
	0 with bad options
	0 with incorrect version number
	77603 fragments received
	5 fragments dropped (dup or out of space)
	51 fragments dropped after timeout
	38638 packets reassembled ok
	714031456 packets for this host
	687844 packets for unknown/unsupported protocol
	112 packets forwarded (0 packets fast forwarded)
	1745 packets not forwardable
	0 packets received for unknown multicast group
	0 redirects sent
	1363117825 packets sent from this host
	0 packets sent with fabricated ip header
	15 output packets dropped due to no bufs, etc.
	0 output packets discarded due to no route
	3808 output datagrams fragmented
	11414 fragments created
	0 datagrams that can't be fragmented
	0 tunneling packets that can't find gif
	0 datagrams with bad address in header
icmp:
	38165 calls to icmp_error
	0 errors not generated in response to an icmp message
	Output histogram:
		echo reply: 36350
		destination unreachable: 38165
	0 messages with bad code fields
	0 messages less than the minimum length
	2672 messages with bad checksum
	2 messages with bad length
	0 multicast echo requests ignored
	0 multicast timestamp requests ignored
	Input histogram:
		echo reply: 626
		destination unreachable: 687094
		routing redirect: 3
		echo: 36350
		time exceeded: 754
	36350 message responses generated
	0 invalid return addresses
	0 no return routes
igmp:
	0 messages received
	0 messages received with too few bytes
	0 messages received with wrong TTL
	0 messages received with bad checksum
	0 V1/V2 membership queries received
	0 V3 membership queries received
	0 membership queries received with invalid field(s)
	0 general queries received
	0 group queries received
	0 group-source queries received
	0 group-source queries dropped
	0 membership reports received
	0 membership reports received with invalid field(s)
	0 membership reports received for groups to which we belong
	0 V3 reports received without Router Alert
	0 membership reports sent
arp:
	364 ARP requests sent
	78 ARP replies sent
	2285 ARP requests received
	5130 ARP replies received
	15589 ARP packets received
	1858 total packets dropped due to no ARP entry
	82 ARP entrys timed out
	0 Duplicate IPs seen
ip6:
	0 total packets received
	0 with size smaller than minimum
	0 with data size < data length
	0 with bad options
	0 with incorrect version number
	0 fragments received
	0 fragments dropped (dup or out of space)
	0 fragments dropped after timeout
	0 fragments that exceeded limit
	0 packets reassembled ok
	0 packets for this host
	0 packets forwarded
	0 packets not forwardable
	0 redirects sent
	12 packets sent from this host
	0 packets sent with fabricated ip header
	0 output packets dropped due to no bufs, etc.
	168 output packets discarded due to no route
	0 output datagrams fragmented
	0 fragments created
	0 datagrams that can't be fragmented
	0 packets that violated scope rules
	0 multicast packets which we don't join
	Mbuf statistics:
		0 one mbuf
		0 one ext mbuf
		0 two or more ext mbuf
	0 packets whose headers are not contiguous
	0 tunneling packets that can't find gif
	0 packets discarded because of too many headers
	0 failures of source address selection
	Source addresses selection rule applied:
icmp6:
	0 calls to icmp6_error
	0 errors not generated in response to an icmp6 message
	0 errors not generated because of rate limitation
	Output histogram:
		neighbor solicitation: 2
	0 messages with bad code fields
	0 messages < minimum length
	0 bad checksums
	0 messages with bad length
	Histogram of error messages to be generated:
		0 no route
		0 administratively prohibited
		0 beyond scope
		0 address unreachable
		0 port unreachable
		0 packet too big
		0 time exceed transit
		0 time exceed reassembly
		0 erroneous header field
		0 unrecognized next header
		0 unrecognized option
		0 redirect
		0 unknown
	0 message responses generated
	0 messages with too many ND options
	0 messages with bad ND options
	0 bad neighbor solicitation messages
	0 bad neighbor advertisement messages
	0 bad router solicitation messages
	0 bad router advertisement messages
	0 bad redirect messages
	0 path MTU changes
rip6:
	0 messages received
	0 checksum calculations on inbound
	0 messages with bad checksum
	0 messages dropped due to no socket
	0 multicast messages dropped due to no socket
	0 messages dropped due to full socket buffers
	0 delivered
	0 datagrams output

------------------------------------------------------------------------
netstat -m

1927/1232/3159 mbufs in use (current/cache/total)
789/905/1694/25600 mbuf clusters in use (current/cache/total/max)
1023/622 mbuf+clusters out of packet secondary zone in use (current/cache)
873/372/1245/12800 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/19200 9k jumbo clusters in use (current/cache/total/max)
0/0/0/12800 16k jumbo clusters in use (current/cache/total/max)
5551K/3606K/9157K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
0 calls to protocol drain routines

------------------------------------------------------------------------
dmesg

Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.2-RELEASE-p3 #4 r260982: Mon Apr  7 14:58:17 MSK 2014
    bel@localhost:/usr/obj/usr/src/sys/ENIGMA i386
gcc version 4.2.1 20070831 patched [FreeBSD]
CPU: Intel(R) Xeon(TM) CPU 3.00GHz (3000.17-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf41  Family = 0xf  Model = 0x4  Stepping = 1
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x641d<SSE3,DTES64,MON,DS_CPL,CNXT-ID,CX16,xTPR>
  AMD Features=0x20000000<LM>
  TSC: P-state invariant
real memory  = 2147483648 (2048 MB)
avail memory = 2082476032 (1986 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <HP     00000083>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 2 package(s) x 1 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  6
ACPI Warning: Invalid length for Pm1aControlBlock: 32, using default 16 (20110527/tbfadt-638)
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic2 <Version 2.0> irqs 48-71 on motherboard
ioapic3 <Version 2.0> irqs 72-95 on motherboard
ioapic4 <Version 2.0> irqs 96-119 on motherboard
kbd1 at kbdmux0
acpi0: <HP P51> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-safe" frequency 3579545 Hz quality 850
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x908-0x90b on acpi0
pcib0: <ACPI Host-PCI bridge> on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci2: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 0.0 on pci2
pci3: <ACPI PCI bus> on pcib2
bge0: <HP NC7782 Gigabit Server Adapter, ASIC rev. 0x002100> mem 0xfddf0000-0xfddfffff irq 25 at device 1.0 on pci3
bge0: CHIP ID 0x00002100; ASIC REV 0x02; CHIP REV 0x21; PCI-X 133 MHz
miibus0: <MII bus> on bge0
brgphy0: <BCM5704 1000BASE-T media interface> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Ethernet address: 00:13:21:f2:41:51
bge1: <HP NC7782 Gigabit Server Adapter, ASIC rev. 0x002100> mem 0xfdde0000-0xfddeffff irq 26 at device 1.1 on pci3
bge1: CHIP ID 0x00002100; ASIC REV 0x02; CHIP REV 0x21; PCI-X 133 MHz
miibus1: <MII bus> on bge1
brgphy1: <BCM5704 1000BASE-T media interface> PHY 1 on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge1: Ethernet address: 00:13:21:f2:41:50
pcib3: <ACPI PCI-PCI bridge> at device 0.2 on pci2
pci4: <ACPI PCI bus> on pcib3
ciss0: <HP Smart Array 6i> port 0x4000-0x40ff mem 0xfdef0000-0xfdef1fff,0xfde80000-0xfdebffff irq 51 at device 3.0 on pci4
ciss0: PERFORMANT Transport
ciss0: got 0 MSI messages]
pcib4: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci5: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 0.0 on pci5
pci6: <ACPI PCI bus> on pcib5
em0: <Intel(R) PRO/1000 Legacy Network Connection 1.0.6> port 0x5000-0x503f mem 0xfdfe0000-0xfdffffff,0xfdf80000-0xfdfbffff irq 78 at device 2.0 on pci6
em0: Ethernet address: 00:11:0a:63:68:9c
em1: <Intel(R) PRO/1000 Legacy Network Connection 1.0.6> port 0x5040-0x507f mem 0xfdf60000-0xfdf7ffff irq 77 at device 2.1 on pci6
em1: Ethernet address: 00:11:0a:63:68:9d
pcib6: <ACPI PCI-PCI bridge> at device 0.2 on pci5
pci10: <ACPI PCI bus> on pcib6
uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0x3000-0x301f irq 16 at device 29.0 on pci0
usbus0 on uhci0
uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0x3020-0x303f irq 19 at device 29.1 on pci0
usbus1 on uhci1
uhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0x3040-0x305f irq 18 at device 29.2 on pci0
usbus2 on uhci2
uhci3: <Intel 82801EB (ICH5) USB controller USB-D> port 0x3060-0x307f irq 16 at device 29.3 on pci0
usbus3 on uhci3
ehci0: <Intel 82801EB/R (ICH5) USB 2.0 controller> mem 0xfbef0000-0xfbef03ff irq 23 at device 29.7 on pci0
usbus4: EHCI version 1.0
usbus4 on ehci0
pcib7: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib7
vgapci0: <VGA-compatible display> port 0x2000-0x20ff mem 0xfc000000-0xfcffffff,0xfbff0000-0xfbff0fff at device 3.0 on pci1
pci1: <base peripheral> at device 4.0 (no driver attached)
pci1: <base peripheral> at device 4.2 (no driver attached)
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH5 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x500-0x50f at device 31.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
acpi_tz0: <Thermal Zone> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model Generic PS/2 mouse, device ID 0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
fdc0: <floppy drive controller (FDE)> port 0x3f2-0x3f5 irq 6 drq 2 on acpi0
pmtimer0 on isa0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xcbfff,0xcc000-0xcd7ff,0xee000-0xeffff pnpid ORM0000 on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atrtc0: <AT realtime clock> at port 0x70 irq 8 on isa0
Event timer "RTC" frequency 32768 Hz quality 0
ppc0: parallel port not found.
uart1: <Non-standard ns8250 class UART with FIFOs> at port 0x2f8-0x2ff irq 3 on isa0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
p4tcc1: <CPU Frequency Thermal Control> on cpu1
Timecounters tick every 1.000 msec
ipfw2 (+ipv6) initialized, divert enabled, nat loadable, default to deny, logging disabled
DUMMYNET 0 with IPv6 initialized (100409)
load_dn_sched dn_sched FIFO loaded
load_dn_sched dn_sched PRIO loaded
load_dn_sched dn_sched QFQ loaded
load_dn_sched dn_sched RR loaded
load_dn_sched dn_sched WF2Q+ loaded
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen4.1: <Intel> at usbus4
uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
da0 at ciss0 bus 0 scbus0 target 0 lun 0
da0: <COMPAQ RAID 5 OK> Fixed Direct Access SCSI-4 device 
da0: 135.168MB/s transfers
da0: Command Queueing enabled
da0: 104183MB (213367680 512 byte sectors: 255H 32S/T 26148C)
da0: quirks=0x1<NO_SYNC_CACHE>
da1 at ciss0 bus 0 scbus0 target 1 lun 0
da1: <COMPAQ RAID 1(1+0) OK> Fixed Direct Access SCSI-4 device 
da1: 135.168MB/s transfers
da1: Command Queueing enabled
da1: 140006MB (286734240 512 byte sectors: 255H 32S/T 35139C)
da1: quirks=0x1<NO_SYNC_CACHE>
cd0 at ata0 bus 0 scbus3 target 0 lun 0
cd0: <COMPAQ CD-ROM SN-124 N104> Removable CD-ROM SCSI-0 device 
cd0: 16.700MB/s transfers (PIO4, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
SMP: AP CPU #1 Launched!
Timecounter "TSC-low" frequency 1500083839 Hz quality 1000
Root mount waiting for: usbus4
Root mount waiting for: usbus4
Root mount waiting for: usbus4
uhub4: 8 ports with 8 removable, self powered
Trying to mount root from ufs:/dev/da0s1a [rw]...
------------------------------------------------------------------------
kernel config

options	CONFIG_AUTOGENERATED
ident ENIGMA
machine	i386
cpu	I686_CPU
cpu	I586_CPU
cpu	I486_CPU
makeoptions	WITH_CTF=1
makeoptions	DEBUG=-g
options	KSTACK_PAGES=4
options	SW_WATCHDOG
options	KDB_UNATTENDED
options	DDB
options	ACCEPT_FILTER_HTTP
options	ROUTETABLES=4
options	SUIDDIR
options	DUMMYNET
options	IPDIVERT
options	IPFIREWALL_FORWARD
options	IPFIREWALL_VERBOSE
options	IPFIREWALL
options	USB_DEBUG
options	RDRAND_RNG
options	PADLOCK_RNG
options	AH_SUPPORT_AR5416
options	IEEE80211_SUPPORT_MESH
options	IEEE80211_AMPDU_AGE
options	IEEE80211_DEBUG
options	SC_PIXEL_MODE
options	VESA
options	AHD_REG_PRETTY_PRINT
options	AHC_REG_PRETTY_PRINT
options	ATA_STATIC_ID
options	ATA_CAM
options	SMP
options	DDB_CTF
options	KDB_TRACE
options	KDB
options	INCLUDE_CONFIG_FILE
options	KDTRACE_HOOKS
options	MAC
options	AUDIT
options	HWPMC_HOOKS
options	KBD_INSTALL_CDEV
options	PRINTF_BUFR_SIZE=128
options	_KPOSIX_PRIORITY_SCHEDULING
options	SYSVSEM
options	SYSVMSG
options	SYSVSHM
options	STACK
options	KTRACE
options	SCSI_DELAY=5000
options	COMPAT_FREEBSD7
options	COMPAT_FREEBSD6
options	COMPAT_FREEBSD5
options	COMPAT_FREEBSD4
options	GEOM_LABEL
options	GEOM_RAID
options	GEOM_PART_GPT
options	PSEUDOFS
options	PROCFS
options	CD9660
options	MSDOSFS
options	NFS_ROOT
options	NFSLOCKD
options	NFSD
options	NFSCL
options	MD_ROOT
options	QUOTA
options	UFS_GJOURNAL
options	UFS_DIRHASH
options	UFS_ACL
options	SOFTUPDATES
options	FFS
options	SCTP
options	TCP_OFFLOAD
options	INET6
options	INET
options	PREEMPTION
options	SCHED_ULE
options	NEW_PCIB
options	NATIVE
options	GEOM_PART_MBR
options	GEOM_PART_EBR_COMPAT
options	GEOM_PART_EBR
options	GEOM_PART_BSD
options	ISAPNP
device	isa
device	npx
device	mem
device	io
device	uart_ns8250
device	atpic
device	apic
device	cpufreq
device	acpi
device	eisa
device	pci
device	fdc
device	ahci
device	ata
device	mvs
device	siis
device	ahb
device	ahc
device	ahd
device	esp
device	hptiop
device	isp
device	mpt
device	sym
device	trm
device	adv
device	adw
device	aha
device	aic
device	bt
device	ncv
device	nsp
device	stg
device	isci
device	scbus
device	ch
device	da
device	sa
device	cd
device	pass
device	ses
device	amr
device	arcmsr
device	asr
device	ciss
device	dpt
device	hptmv
device	hptnr
device	hptrr
device	hpt27xx
device	iir
device	ips
device	mly
device	twa
device	tws
device	aac
device	aacp
device	aacraid
device	ida
device	mfi
device	mlx
device	pst
device	twe
device	atkbdc
device	atkbd
device	psm
device	kbdmux
device	vga
device	splash
device	sc
device	agp
device	pmtimer
device	cbb
device	pccard
device	cardbus
device	uart
device	ppc
device	ppbus
device	lpt
device	plip
device	ppi
device	puc
device	bxe
device	de
device	em
device	igb
device	ixgb
device	le
device	ti
device	txp
device	vx
device	miibus
device	ae
device	age
device	alc
device	ale
device	bce
device	bfe
device	bge
device	cas
device	dc
device	et
device	fxp
device	gem
device	hme
device	jme
device	lge
device	msk
device	nfe
device	nge
device	pcn
device	re
device	rl
device	sf
device	sge
device	sis
device	sk
device	ste
device	stge
device	tl
device	tx
device	vge
device	vr
device	vte
device	wb
device	xl
device	cs
device	ed
device	ex
device	ep
device	fe
device	ie
device	sn
device	xe
device	wlan
device	wlan_wep
device	wlan_ccmp
device	wlan_tkip
device	wlan_amrr
device	an
device	ath
device	ath_pci
device	ath_hal
device	ath_rate_sample
device	ipw
device	iwi
device	iwn
device	malo
device	mwl
device	ral
device	wi
device	wpi
device	loop
device	random
device	ether
device	vlan
device	tun
device	pty
device	md
device	gif
device	faith
device	firmware
device	bpf
device	uhci
device	ohci
device	ehci
device	xhci
device	usb
device	uhid
device	ukbd
device	ulpt
device	umass
device	ums
device	urio
device	u3g
device	uark
device	ubsa
device	uftdi
device	uipaq
device	uplcom
device	uslcom
device	uvisor
device	uvscom
device	aue
device	axe
device	cdce
device	cue
device	kue
device	rue
device	udav
device	rum
device	run
device	uath
device	upgt
device	ural
device	urtw
device	zyd
device	sound
device	snd_cmi
device	snd_csa
device	snd_emu10kx
device	snd_es137x
device	snd_hda
device	snd_ich
device	snd_uaudio
device	snd_via8233
device	virtio
device	virtio_pci
device	vtnet
device	virtio_blk
device	virtio_scsi
device	virtio_balloon

------------------------------------------------------------------------
ddb capture buffer

How-To-Repeat: The kernel config follows:
===
include GENERIC

ident           ENIGMA

options         IPFIREWALL
options         IPFIREWALL_VERBOSE
options         IPFIREWALL_FORWARD
options         IPDIVERT
options         DUMMYNET
options         SUIDDIR
options         ROUTETABLES=4
options         ACCEPT_FILTER_HTTP
options         DDB
options         KDB_UNATTENDED
options         SW_WATCHDOG
options         KSTACK_PAGES=4
===
Comment 1 Andrew Belashov 2014-04-16 10:35:24 UTC
Additional debug information. This is strange recursive Page Fault exceptio=
n. This is CPU bug?

The stack is flooded by Page Fault recursion (0xc0f65d4c - Is the address o=
f first instruction of Page Fault handler):
(kgdb) x/64w 0xf66b6000
0xf66b6000:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6010:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6020:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6030:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6040:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6050:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6060:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6070:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6080:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6090:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b60a0:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b60b0:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b60c0:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b60d0:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b60e0:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b60f0:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
[160 similar lines skipped]
(kgdb)
0xf66b6b00:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6b10:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6b20:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6b30:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6b40:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6b50:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6b60:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6b70:     0x00000000      0xc0f65d4c      0x00000020      0x00010006
0xf66b6b80:     0x00000002      0xc0f731d5      0x00000020      0x00010206
0xf66b6b90:     0xc83b6048      0xc989c2f0      0xf66b6ba4      0xc425cff0
0xf66b6ba0:     0xc83b60b0      0xf66b6bd4      0xc0f748dd      0xf5eb7000
0xf66b6bb0:     0x00000001      0xc83b6000      0xf66b6bd4      0xc0d967fb
0xf66b6bc0:     0xc83b6000      0x00000000      0xc83b60b0      0xc83b6000
0xf66b6bd0:     0xc9528000      0xf66b6bf4      0xc0d99eeb      0xc83b60b0
0xf66b6be0:     0x00001000      0xbfc00000      0xc989c2f0      0x00000000
0xf66b6bf0:     0xc9528000      0xf66b6c38      0xc0ab2b5a      0xc989c2f0
(kgdb) info symbol 0xc0f65d4c
Xpage in section .text
Xpage in section .text
(kgdb) x/3i Xpage
0xc0f65d4c <Xpage>:     push   $0xc
0xc0f65d4e <Xpage+2>:   jmp    0xc0f65d90 <alltraps>
0xc0f65d53 <Xpage+7>:   nop


With best regards,
 Andrew Belashov
Comment 2 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:59:13 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped