Bug 243646
| Summary: | panic: bad pte va 966000 pte 0 | ||
|---|---|---|---|
| Product: | Base System | Reporter: | Dirk Meyer <dinoex> |
| Component: | kern | Assignee: | freebsd-bugs (Nobody) <bugs> |
| Status: | New --- | ||
| Severity: | Affects Only Me | CC: | markj |
| Priority: | --- | Keywords: | crash |
| Version: | 12.1-RELEASE | ||
| Hardware: | amd64 | ||
| OS: | Any | ||
some Kernel modules removed:
FreeBSD netcom2.dinoex.sub.de 12.1-RELEASE-p1 FreeBSD 12.1-RELEASE-p1 r357113 GENERIC amd64
panic: bad pte va 1180000 pte fffff80081787bf8
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...
Unread portion of the kernel message buffer:
panic: bad pte va 1180000 pte fffff80081787bf8
cpuid = 1
time = 1580092419
KDB: stack backtrace:
#0 0xffffffff80c1d297 at kdb_backtrace+0x67
#1 0xffffffff80bd05cd at vpanic+0x19d
#2 0xffffffff80bd0423 at panic+0x43
#3 0xffffffff81099fb7 at pmap_remove_pages+0x8b7
#4 0xffffffff80f1171c at vmspace_exit+0x9c
#5 0xffffffff80b8a509 at exit1+0x5d9
#6 0xffffffff80b89f2d at sys_sys_exit+0xd
#7 0xffffffff810a8984 at amd64_syscall+0x364
#8 0xffffffff81082330 at fast_syscall_common+0x101
Uptime: 2h21m57s
Dumping 1413 out of 16235 MB:..2%..11%..21%..31%..41%..51%..62%..71%..81%..91%
Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/linprocfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/linprocfs.ko
Reading symbols from /boot/kernel/linux_common.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux_common.ko.debug...done.
done.
Loaded symbols for /boot/kernel/linux_common.ko
Reading symbols from /boot/kernel/linsysfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/linsysfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/linsysfs.ko
Reading symbols from /boot/kernel/ums.ko...Reading symbols from /usr/lib/debug//boot/kernel/ums.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ums.ko
Reading symbols from /boot/kernel/uhid.ko...Reading symbols from /usr/lib/debug//boot/kernel/uhid.ko.debug...done.
done.
Loaded symbols for /boot/kernel/uhid.ko
Reading symbols from /boot/kernel/linux.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux.ko.debug...done.
done.
Loaded symbols for /boot/kernel/linux.ko
Reading symbols from /boot/kernel/linux64.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux64.ko.debug...done.
done.
Loaded symbols for /boot/kernel/linux64.ko
Reading symbols from /boot/kernel/speaker.ko...Reading symbols from /usr/lib/debug//boot/kernel/speaker.ko.debug...done.
done.
Loaded symbols for /boot/kernel/speaker.ko
Reading symbols from /boot/kernel/sem.ko...Reading symbols from /usr/lib/debug//boot/kernel/sem.ko.debug...done.
done.
Loaded symbols for /boot/kernel/sem.ko
Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/nullfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/nullfs.ko
Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/tmpfs.ko
Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/fdescfs.ko
#0 doadump () at src/sys/amd64/include/pcpu.h:234
234 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (OFFSETOF_CURTHREAD));
(kgdb) #0 doadump () at src/sys/amd64/include/pcpu.h:234
#1 0xffffffff80bd01c8 in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:451
#2 0xffffffff80bd0629 in vpanic (fmt=<value optimized out>,
ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:877
#3 0xffffffff80bd0423 in panic (fmt=<value optimized out>)
at /usr/src/sys/kern/kern_shutdown.c:804
#4 0xffffffff81099fb7 in pmap_remove_pages (pmap=<value optimized out>)
at /usr/src/sys/amd64/amd64/pmap.c:6932
#5 0xffffffff80f1171c in vmspace_exit (td=0xfffff80315f12000)
at /usr/src/sys/vm/vm_map.c:409
#6 0xffffffff80b8a509 in exit1 (td=0xfffff80315f12000,
rval=<value optimized out>, signo=0) at /usr/src/sys/kern/kern_exit.c:396
#7 0xffffffff80b89f2d in sys_sys_exit (td=<value optimized out>,
uap=<value optimized out>) at /usr/src/sys/kern/kern_exit.c:175
#8 0xffffffff810a8984 in amd64_syscall (td=0xfffff80315f12000, traced=0)
at src/sys/amd64/amd64/../../kern/subr_syscall.c:135
#9 0xffffffff81082330 in fast_syscall_common ()
at /usr/src/sys/amd64/amd64/exception.S:581
#10 0x00000008029656aa in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language: auto; currently minimal
(kgdb)
------------------------------------------------------------------------
ps -axlww
UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND
0 0 0 0 -16 0 0 0 swapin DLs - 0:00.02 [kernel]
0 1 0 0 25 0 9960 896 wait DLs - 0:00.00 [init]
0 2 0 0 -16 0 0 0 crypto_w DL - 0:00.00 [crypto]
0 3 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 0]
0 4 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 1]
0 5 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 2]
0 6 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 3]
0 7 0 0 -16 0 0 0 - RL - 0:00.00 [cam]
0 8 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod1]
0 9 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod2]
0 10 0 0 -16 0 0 0 audit_wo DL - 0:00.00 [audit]
0 11 0 0 155 0 0 0 - RL - 3:43.29 [idle]
0 12 0 0 -56 0 0 0 - WL - 0:00.72 [intr]
0 13 0 0 -8 0 0 0 - DL - 0:00.00 [geom]
0 14 0 0 -16 0 0 0 seqstate DL - 0:00.00 [sequencer 00]
0 15 0 0 -68 0 0 0 - DL - 0:00.02 [usb]
0 16 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod3]
0 17 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod4]
0 18 0 0 -16 0 0 0 waiting_ DL - 0:00.00 [sctp_iterator]
0 19 0 0 -16 0 0 0 tzpoll DL - 0:00.00 [acpi_thermal]
0 20 0 0 -16 0 0 0 cooling DL - 0:00.00 [acpi_cooling1]
0 21 0 0 -16 0 0 0 - DL - 0:00.01 [rand_harvestq]
0 22 0 0 -16 0 0 0 psleep DL - 0:00.00 [pagedaemon]
0 23 0 0 -16 0 0 0 psleep DL - 0:00.00 [vmdaemon]
0 24 0 0 20 0 0 0 qsleep DL - 0:00.02 [bufdaemon]
0 25 0 0 20 0 0 0 vlruwt DL - 0:00.00 [vnlru]
0 26 0 0 16 0 0 0 syncer DL - 0:00.00 [syncer]
0 550 1 0 52 0 11048 2328 select Ds - 0:00.00 [moused]
0 579 1 0 20 0 11388 2620 select Ds - 0:00.00 [dhclient]
0 582 1 0 39 0 11612 2760 select Ds - 0:00.00 [dhclient]
65 628 1 0 20 0 11752 2876 select DCs - 0:00.00 [dhclient]
0 629 1 0 20 0 10456 1476 select Ds - 0:00.00 [devd]
0 637 1 0 20 0 16092 5500 select Ds - 0:00.01 [tincd]
0 711 1 0 20 0 10908 2216 select Ds - 0:00.00 [rtsold]
0 732 1 0 20 0 11400 2668 select Ds - 0:00.00 [syslogd]
0 946 1 0 52 0 11084 2452 select Ds - 0:00.00 [lpd]
25 982 1 0 20 0 16800 6368 pause Ds - 0:00.00 [sendmail]
0 995 1 0 20 0 17156 6896 select Ds - 0:00.01 [openvpn]
0 1006 1 0 20 0 17076 6984 select Ds - 0:00.01 [openvpn]
0 1048 1 0 20 0 19652 8372 select Ds - 0:00.00 [sshd]
0 1051 1 0 43 0 11464 2676 nanslp Ds - 0:00.00 [cron]
0 1127 1 0 52 0 11444 2600 select Ds - 0:00.00 [inetd]
0 1152 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty]
0 1153 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty]
0 1154 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty]
0 1155 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty]
0 1156 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty]
0 1157 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty]
0 1158 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty]
0 1159 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty]
0 1199 1048 0 20 0 20984 9536 select Ds - 0:00.00 [sshd]
0 1201 1199 0 20 0 14408 4652 pause Ds - 0:00.00 [csh]
0 1207 1199 0 20 0 14448 4720 ttyin Ds+ - 0:00.00 [csh]
0 1225 1201 0 21 0 13204 3868 select D+ - 0:00.00 [tmux]
0 1227 1 0 20 0 14048 5020 select Ds - 0:00.00 [tmux]
0 1228 1227 0 20 0 13608 4344 pause Ds - 0:00.01 [csh]
0 1356 1228 0 20 0 12604 3908 select D+ - 0:01.01 [sh]
0 1668 1356 0 52 0 13156 4180 nanslp D+ - 0:08.80 [sh]
0 4977 96509 0 52 0 11796 2988 piperd D - 0:00.00 [sh]
0 4980 4977 0 52 0 11796 2984 wait D - 0:00.00 [sh]
0 4981 4980 0 52 0 19864 9060 wait D - 0:00.00 [pkg]
0 4982 4980 0 52 0 10912 2228 piperd D - 0:00.00 [sed]
0 4983 4980 0 52 0 10764 2148 piperd DC - 0:00.00 [tee]
0 4984 4980 0 52 0 11052 2400 piperd D - 0:00.00 [wc]
0 4985 4981 0 33 0 28512 18072 - R - 0:00.00 [pkg]
0 22405 1356 0 26 0 12604 3832 piperd D+ - 0:00.23 [sh]
0 30973 1356 0 42 0 12604 3912 select D - 0:00.00 [sh]
0 32926 30973 0 43 0 12604 3908 wait D - 0:00.00 [sh]
65534 32927 32926 0 52 0 10480 1824 wait D - 0:00.00 [make]
65534 33025 32927 0 52 0 11648 3148 wait D - 0:00.00 [sh]
65534 33028 33025 0 52 0 11976 3220 wait D - 0:00.00 [sh]
65534 33135 33028 0 52 0 21020 11944 piperd D - 0:00.00 [qmake]
65534 33412 33135 0 52 0 10024 1320 wait D - 0:00.00 [make]
65534 33450 33412 0 52 0 72556 63576 wait D - 0:00.00 [clang++]
65534 33451 33450 0 52 0 66924 29948 - RE - 0:00.00 [ld.lld]
0 94762 1051 0 20 0 11464 2684 piperd D - 0:00.00 [cron]
0 94764 94762 0 22 0 11700 2968 wait Ds - 0:00.00 [sh]
0 94766 94764 0 22 0 10680 2128 wait D - 0:00.00 [lockf]
0 94767 94766 0 52 0 11700 2968 wait D - 0:00.00 [sh]
0 94775 94767 0 52 0 11752 2972 wait D - 0:00.00 [sh]
0 94776 94767 0 27 0 10892 2328 piperd D - 0:00.00 [mail]
0 96487 94775 0 52 0 11712 2972 wait D - 0:00.00 [sh]
0 96491 96487 0 52 0 11704 2968 wait D - 0:00.00 [sh]
0 96496 96491 0 52 0 10680 2128 wait D - 0:00.00 [lockf]
0 96498 96496 0 52 0 11704 2968 wait D - 0:00.00 [sh]
0 96509 96498 0 52 0 11752 2972 wait D - 0:00.00 [sh]
0 96510 96498 0 20 0 10892 2328 piperd D - 0:00.00 [mail]
------------------------------------------------------------------------
vmstat -s
23471056 cpu context switches
4280481 device interrupts
148337 software interrupts
158386910 traps
172407235 system calls
26 kernel threads created
706241 fork() calls
225962 vfork() calls
2 rfork() calls
0 swap pager pageins
0 swap pager pages paged in
0 swap pager pageouts
0 swap pager pages paged out
3724 vnode pager pageins
15594 vnode pager pages paged in
8582 vnode pager pageouts
167467 vnode pager pages paged out
0 page daemon wakeups
0 pages examined by the page daemon
0 clean page reclamation shortfalls
0 pages reactivated by the page daemon
23915075 copy-on-write faults
24024 copy-on-write optimized faults
126595742 zero fill pages zeroed
0 zero fill pages prezeroed
119 intransit blocking page faults
158210567 total VM faults taken
3238 page faults requiring I/O
0 pages affected by kernel thread creation
26828679 pages affected by fork()
46839162 pages affected by vfork()
114 pages affected by rfork()
193659777 pages freed
0 pages freed by daemon
0 pages freed by exiting processes
0 pages active
0 pages inactive
0 pages in the laundry queue
0 pages wired down
0 pages free
0 bytes per page
0 total name lookups
cache hits (0% pos + 0% neg) system 0% per-directory
deletions 0%, falsehits 0%, toolong 0%
------------------------------------------------------------------------
vmstat -m
Type InUse MemUse HighUse Requests Size(s)
CAM DEV 10 20K - 12 2048
entropy 1 1K - 41937 32,4096
CAM CCB 0 0K - 3841585 2048
CAM path 17 1K - 66 32
CAM periph 16 4K - 14171 16,32,64,128,256
acpi_perf 4 2K - 4 512
acpipwr 11 1K - 11 64
hdaa 14 57K - 14 256,512,1024,2048,16384,32768
hdac 3 2K - 3 128,256,1024
hdacc 3 1K - 3 32
feeder 21 2K - 27 32,128
mixer 4 16K - 4 4096
CAM I/O Scheduler 6 1K - 6 128
CAM queue 14 6K - 26 16,32,64,512
scsi_cd 0 0K - 6 16
USB 60 92K - 67 16,32,128,256,512,1024,2048,4096,8192,16384
USBdev 44 10K - 44 32,64,128,256,512,4096
CAM dev queue 4 1K - 4 64
scsi_da 0 0K - 60 16,256
vtbuf 24 4752K - 94 4096
vt 11 6K - 11 512
DEVFS2 131 5K - 262 16,32
DEVFS3 476 119K - 512 256
DEVFS1 131 66K - 170 512
DEVFS_RULE 65 31K - 65 64,512
DEVFS 53 2K - 54 16,128
DEVFSP 3 1K - 3 64
NFSD V4client 1 1K - 1 256
NFSD lckfile 1 1K - 1 256
NFSD session 1 1K - 1 1024
pfs_nodes 388 194K - 388 512
pfs_vncache 8 1K - 63 64
GEOM 139 27K - 1293 16,32,64,128,256,512,1024,2048,8192,16384,32768
raid_data 0 0K - 174 32,128,256
isadev 8 1K - 8 128
cdev 5 2K - 5 256
filedesc 1 4K - 68 32,64,4096,16384,32768
sigio 0 0K - 1 64
filecaps 4 1K - 52378 16,32,64
kdtrace 250 47K - 1882747 64,256
kenv 95 12K - 98 16,32,64,128,8192
kqueue 87 6K - 941658 64,256,512,2048
proc-args 82 5K - 535938 16,32,64,128,256
hhook 13 4K - 13 256
ithread 95 16K - 96 32,128,256
prison 26 18K - 26 16,32,256,4096
KTRACE 100 13K - 100 128
linker 384 1172K - 684 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
lockf 42 5K - 105108 64,128
loginclass 3 1K - 3 64
cache 1 1K - 1 32
devbuf 18066 37897K - 19638 16,32,64,128,256,512,1024,2048,4096,8192,65536
temp 50 18K - 14416436 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
evdev 5 5K - 5 1024
module 516 65K - 533 128
mtx_pool 2 72K - 2 8192,65536
osd 7 1K - 26 16,32,64,128,256
pmchooks 1 1K - 1 128
pmc 1 1K - 1 64
pgrp 33 5K - 37111 128
session 30 4K - 138 128
proc 2 64K - 2 32768
subproc 223 416K - 932397 512,4096
cred 62 16K - 73042 256
acpiintr 1 1K - 1 64
plimit 25 7K - 94735 256
uidinfo 6 9K - 64 128,8192
dumper 1 1K - 1 512
acpica 6964 688K - 195267 16,32,64,128,256,512,1024,2048,4096
sysctl 0 0K - 18 64
sysctloid 5190 265K - 6392 16,32,64,128
sysctltmp 0 0K - 72973 16,64,256,1024
acpitask 1 64K - 1 65536
tidhash 1 64K - 1 65536
callout 5 2184K - 5
umtx 524 66K - 524 128
p1003.1b 1 1K - 1 16
SWAP 1 232K - 1
bus 1379 120K - 42352 16,32,64,128,256,1024,4096
bus-sc 84 1910K - 3733 16,32,128,256,512,1024,2048,4096,8192,16384,32768,65536
acpisem 114 15K - 114 128
devstat 10 21K - 10 32,4096
epoch 4 1K - 4 128
eventhandler 126 11K - 127 64,128
gtaskqueue 34 35K - 34 16,32,256,8192
kobj 348 1392K - 657 4096
Per-cpu 1 1K - 1 32
kbdmux 7 22K - 7 16,512,1024,2048,16384
rman 243 28K - 670 32,128
sbuf 0 0K - 1094 16,32,64,128,256,4096,8192,16384,32768
toponodes 18 3K - 18 128
taskqueue 33 4K - 39 16,32,256
terminal 11 3K - 11 256
Unitno 29 2K - 34599 32,64
vmem 3 260K - 21 2048,4096,8192,16384,32768,65536
ioctlops 0 0K - 420 256,512,1024,2048
select 107 14K - 107 128
iov 1 1K - 10212321 16,32,64,128,256,512,4096
msg 4 30K - 4 2048,4096,8192,16384
sem 4 106K - 4 2048,4096
shm 1 32K - 1 32768
tty 15 15K - 15 1024
pts 3 1K - 3 256
mbuf_tag 0 0K - 200 32
shmfd 1 8K - 1 8192
soname 6 1K - 111707 16,32,128
pcb 32 2201K - 1077 16,32,64,128,1024,2048,8192
acl 0 0K - 102961 4096
vfscache 4 4145K - 4 256,16384,32768
cl_savebuf 0 0K - 13864 64
vfs_hash 1 2048K - 1
vnodes 4 1K - 1360 32,256
mount 476 25K - 43455 16,32,64,128,256,1024
statfs 0 0K - 1610811 4096
vnodemarker 0 0K - 5178 512
chacha20random 1 1K - 1 1024
BPF 13 18K - 13 128,512,4096
ifnet 6 11K - 6 128,2048
ifaddr 72 21K - 72 16,32,64,128,256,512,2048,4096
ether_multi 76 6K - 76 16,32,64,128
clone 7 1K - 7 128
ipsec 3 1K - 3 256
lltable 28 9K - 28 256,512
tun 3 1K - 3 256
routetbl 100 9K - 109 32,64,128,512
vnet 1 1K - 1 64
vnet_data 1 240K - 1
vnet_data_free 1 1K - 1 32
igmp 5 1K - 5 128
in_multi 5 2K - 5 256
encap_export_host 12 1K - 12 32,64
sctp_a_it 0 0K - 8 16
sctp_vrf 1 1K - 1 64
sctp_ifa 14 2K - 14 128
sctp_ifn 5 1K - 5 128
sctp_iter 0 0K - 8 256
tfo_ccache 1 128K - 1
hostcache 1 32K - 1 32768
tcpfunc 1 1K - 1 64
syncache 1 68K - 1
in6_multi 60 8K - 60 32,256
ip6opt 1 1K - 2 32,256
mld 5 1K - 5 128
ip6ndp 13 2K - 18 32,64,256
inpcbpolicy 20 1K - 1364 32
secasvar 1 1K - 1 1024
sahead 1 1K - 1 1024
ipsecpolicy 2 2K - 2 256,1024
ipsec-saq 2 2K - 2 1024
crypto 2 2K - 2 512,1024
rpc 2 8K - 2 4096
audit_evclass 230 8K - 285 32
pagedep 1316 841K - 170334 256
inodedep 18603 11349K - 1458476 512
bmsafemap 12 11K - 1112239 256,8192
newblk 6192 5644K - 1900939 256
indirdep 60 8K - 6079 128,32768
freefrag 2 1K - 61705 128
freeblks 323 81K - 1048986 256
freefile 16 1K - 1029900 64
diradd 18521 2316K - 1605777 128
mkdir 995 125K - 272436 128
dirrem 20 3K - 1577774 128
newdirblk 988 62K - 140455 64
freework 419 53K - 1268677 64,128
freedep 0 0K - 4730 64
jaddref 1 1K - 1878213 128
jremref 1 1K - 1845374 128
jmvref 0 0K - 23401 128
jnewblk 1 1K - 1900938 128
jfreefrag 0 0K - 61705 128
jseg 20524 2566K - 538964 128
jsegdep 25931 1621K - 5686230 64
sbdep 0 0K - 261 64
savedino 0 0K - 108216 256
jblocks 2 1K - 2 128,256
softdep 1 1K - 1 512
ufs_dirhash 8550 2295K - 602673 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
ufs_quota 1 2048K - 1
ufs_mount 3 41K - 5 512,8192,32768
vm_pgdata 1 1K - 1 128
UMAHash 5 96K - 23 512,1024,2048,4096,8192,16384,32768,65536
memdesc 1 4K - 1 4096
pci_link 16 2K - 16 64,128
atkbddev 2 1K - 2 64
acpidev 61 4K - 61 64
CAM SIM 4 1K - 4 256
CAM XPT 46 4K - 123 16,32,64,128,256,512,1024,2048,65536
apmdev 1 1K - 1 128
madt_table 0 0K - 2 64,4096
intr 4 400K - 4 65536
io_apic 1 2K - 1 2048
local_apic 1 8K - 1 8192
MCA 14 3K - 14 32,128,256
cpus 2 1K - 2 32
msi 15 2K - 15 128
nexusdev 5 1K - 5 16
linux 31 2K - 32 64
ksem 1 8K - 1 8192
nullfs_hash 1 2048K - 1
nullfs_node 718 45K - 5154 64
nullfs_mount 17 1K - 150 32
tmpfs mount 3 1K - 3 256
tmpfs name 5252 212K - 58195 16,32,64,128
fdesc_mount 2 1K - 2 16
------------------------------------------------------------------------
This looks a bit like a page table page is being freed while it is still in use. Is there something specific that triggers this? Which process is exiting during the crash? running poudriere bulk -J1 (In reply to Dirk Meyer from comment #3) Are you able to try running with INVARIANTS enabled? running gerneric Kernel only I could reproduce the problems until I disabled journaling on the root fs (ufs, softupdates) |
FreeBSD netcom2.dinoex.sub.de 12.1-RELEASE-p1 FreeBSD 12.1-RELEASE-p1 r357113 GENERIC amd64 panic: bad pte va 966000 pte 0 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: bad pte va 966000 pte 0 cpuid = 0 time = 1580083866 KDB: stack backtrace: #0 0xffffffff80c1d297 at kdb_backtrace+0x67 #1 0xffffffff80bd05cd at vpanic+0x19d #2 0xffffffff80bd0423 at panic+0x43 #3 0xffffffff81099fb7 at pmap_remove_pages+0x8b7 #4 0xffffffff80f1171c at vmspace_exit+0x9c #5 0xffffffff80b8a509 at exit1+0x5d9 #6 0xffffffff80b89f2d at sys_sys_exit+0xd #7 0xffffffff810a8984 at amd64_syscall+0x364 #8 0xffffffff81082330 at fast_syscall_common+0x101 Uptime: 14h50m29s Dumping 937 out of 16235 MB:..2%..11%..21%..31%..41%..52%..62%..72%..81%..91% Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/linprocfs.ko.debug...done. done. Loaded symbols for /boot/kernel/linprocfs.ko Reading symbols from /boot/kernel/linux_common.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux_common.ko.debug...done. done. Loaded symbols for /boot/kernel/linux_common.ko Reading symbols from /boot/kernel/linsysfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/linsysfs.ko.debug...done. done. Loaded symbols for /boot/kernel/linsysfs.ko Reading symbols from /boot/kernel/ums.ko...Reading symbols from /usr/lib/debug//boot/kernel/ums.ko.debug...done. done. Loaded symbols for /boot/kernel/ums.ko Reading symbols from /boot/kernel/radeonkms.ko...Reading symbols from /usr/lib/debug//boot/kernel/radeonkms.ko.debug...done. done. Loaded symbols for /boot/kernel/radeonkms.ko Reading symbols from /boot/kernel/drm2.ko...Reading symbols from /usr/lib/debug//boot/kernel/drm2.ko.debug...done. done. Loaded symbols for /boot/kernel/drm2.ko Reading symbols from /boot/kernel/iicbus.ko...Reading symbols from /usr/lib/debug//boot/kernel/iicbus.ko.debug...done. done. Loaded symbols for /boot/kernel/iicbus.ko Reading symbols from /boot/kernel/iic.ko...Reading symbols from /usr/lib/debug//boot/kernel/iic.ko.debug...done. done. Loaded symbols for /boot/kernel/iic.ko Reading symbols from /boot/kernel/iicbb.ko...Reading symbols from /usr/lib/debug//boot/kernel/iicbb.ko.debug...done. done. Loaded symbols for /boot/kernel/iicbb.ko Reading symbols from /boot/kernel/radeonkmsfw_BARTS_pfp.ko...Reading symbols from /usr/lib/debug//boot/kernel/radeonkmsfw_BARTS_pfp.ko.debug...done. done. Loaded symbols for /boot/kernel/radeonkmsfw_BARTS_pfp.ko Reading symbols from /boot/kernel/radeonkmsfw_BARTS_me.ko...Reading symbols from /usr/lib/debug//boot/kernel/radeonkmsfw_BARTS_me.ko.debug...done. done. Loaded symbols for /boot/kernel/radeonkmsfw_BARTS_me.ko Reading symbols from /boot/kernel/radeonkmsfw_BTC_rlc.ko...Reading symbols from /usr/lib/debug//boot/kernel/radeonkmsfw_BTC_rlc.ko.debug...done. done. Loaded symbols for /boot/kernel/radeonkmsfw_BTC_rlc.ko Reading symbols from /boot/kernel/radeonkmsfw_BARTS_mc.ko...Reading symbols from /usr/lib/debug//boot/kernel/radeonkmsfw_BARTS_mc.ko.debug...done. done. Loaded symbols for /boot/kernel/radeonkmsfw_BARTS_mc.ko Reading symbols from /boot/kernel/uhid.ko...Reading symbols from /usr/lib/debug//boot/kernel/uhid.ko.debug...done. done. Loaded symbols for /boot/kernel/uhid.ko Reading symbols from /boot/kernel/linux.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux.ko.debug...done. done. Loaded symbols for /boot/kernel/linux.ko Reading symbols from /boot/kernel/linux64.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux64.ko.debug...done. done. Loaded symbols for /boot/kernel/linux64.ko Reading symbols from /boot/kernel/speaker.ko...Reading symbols from /usr/lib/debug//boot/kernel/speaker.ko.debug...done. done. Loaded symbols for /boot/kernel/speaker.ko Reading symbols from /boot/modules/nvidia.ko...done. Loaded symbols for /boot/modules/nvidia.ko Reading symbols from /boot/kernel/sem.ko...Reading symbols from /usr/lib/debug//boot/kernel/sem.ko.debug...done. done. Loaded symbols for /boot/kernel/sem.ko Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/nullfs.ko.debug...done. done. Loaded symbols for /boot/kernel/nullfs.ko Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done. done. Loaded symbols for /boot/kernel/tmpfs.ko Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done. done. Loaded symbols for /boot/kernel/fdescfs.ko #0 doadump () at src/sys/amd64/include/pcpu.h:234 234 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (OFFSETOF_CURTHREAD)); (kgdb) #0 doadump () at src/sys/amd64/include/pcpu.h:234 #1 0xffffffff80bd01c8 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:451 #2 0xffffffff80bd0629 in vpanic (fmt=<value optimized out>, ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:877 #3 0xffffffff80bd0423 in panic (fmt=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:804 #4 0xffffffff81099fb7 in pmap_remove_pages (pmap=<value optimized out>) at /usr/src/sys/amd64/amd64/pmap.c:6932 #5 0xffffffff80f1171c in vmspace_exit (td=0xfffff8019974e5e0) at /usr/src/sys/vm/vm_map.c:409 #6 0xffffffff80b8a509 in exit1 (td=0xfffff8019974e5e0, rval=<value optimized out>, signo=0) at /usr/src/sys/kern/kern_exit.c:396 #7 0xffffffff80b89f2d in sys_sys_exit (td=<value optimized out>, uap=<value optimized out>) at /usr/src/sys/kern/kern_exit.c:175 #8 0xffffffff810a8984 in amd64_syscall (td=0xfffff8019974e5e0, traced=0) at src/sys/amd64/amd64/../../kern/subr_syscall.c:135 #9 0xffffffff81082330 in fast_syscall_common () at /usr/src/sys/amd64/amd64/exception.S:581 #10 0x0000000003e6794a in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) ------------------------------------------------------------------------ ps -axlww UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 0 0 0 -16 0 0 0 swapin DLs - 0:00.04 [kernel] 0 1 0 0 25 0 9960 896 wait DLs - 0:00.00 [init] 0 2 0 0 -16 0 0 0 crypto_w DL - 0:00.00 [crypto] 0 3 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 0] 0 4 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 1] 0 5 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 2] 0 6 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns 3] 0 7 0 0 -16 0 0 0 - RL - 0:00.43 [cam] 0 8 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod1] 0 9 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod2] 0 10 0 0 -16 0 0 0 audit_wo DL - 0:00.00 [audit] 0 11 0 0 155 0 0 0 - RL - 3539:05.53 [idle] 0 12 0 0 -56 0 0 0 - WL - 0:29.93 [intr] 0 13 0 0 -8 0 0 0 - DL - 0:00.00 [geom] 0 14 0 0 -16 0 0 0 seqstate DL - 0:00.00 [sequencer 00] 0 15 0 0 -68 0 0 0 - DL - 0:02.98 [usb] 0 16 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod3] 0 17 0 0 -16 0 0 0 - DL - 0:00.00 [soaiod4] 0 18 0 0 -16 0 0 0 waiting_ DL - 0:00.00 [sctp_iterator] 0 19 0 0 -16 0 0 0 tzpoll DL - 0:00.17 [acpi_thermal] 0 20 0 0 -16 0 0 0 cooling DL - 0:00.03 [acpi_cooling1] 0 21 0 0 -16 0 0 0 - DL - 0:02.89 [rand_harvestq] 0 22 0 0 -16 0 0 0 psleep DL - 0:00.87 [pagedaemon] 0 23 0 0 -16 0 0 0 psleep DL - 0:00.00 [vmdaemon] 0 24 0 0 20 0 0 0 qsleep DL - 0:01.14 [bufdaemon] 0 25 0 0 -16 0 0 0 vlruwt DL - 0:00.04 [vnlru] 0 26 0 0 16 0 0 0 syncer DL - 0:00.33 [syncer] 0 550 1 0 20 0 11048 2388 select Ds - 0:00.00 [moused] 0 579 1 0 20 0 11388 2620 select Ds - 0:00.00 [dhclient] 0 582 1 0 43 0 11612 2760 select Ds - 0:00.00 [dhclient] 65 628 1 0 20 0 11752 2876 select DCs - 0:00.00 [dhclient] 0 629 1 0 20 0 10456 1476 select Ds - 0:02.54 [devd] 0 637 1 0 20 0 16272 5500 select Ds - 0:00.32 [tincd] 0 711 1 0 20 0 10908 2216 select Ds - 0:00.01 [rtsold] 0 732 1 0 20 0 11400 2668 select Ds - 0:01.88 [syslogd] 0 946 1 0 52 0 11084 2452 select Ds - 0:00.00 [lpd] 25 982 1 0 20 0 16800 6368 pause Ds - 0:00.01 [sendmail] 0 995 1 0 20 0 17156 6912 select Ds - 0:00.24 [openvpn] 0 1006 1 0 20 0 17104 6956 select Ds - 0:00.35 [openvpn] 0 1048 1 0 20 0 19652 8372 select Ds - 0:00.00 [sshd] 0 1051 1 0 44 0 11464 2676 nanslp Ds - 0:00.07 [cron] 0 1127 1 0 52 0 11444 2600 select Ds - 0:00.00 [inetd] 0 1152 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty] 0 1153 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty] 0 1154 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty] 0 1155 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty] 0 1156 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty] 0 1157 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty] 0 1158 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty] 0 1159 1 0 52 0 10880 2288 ttyin Ds+ - 0:00.00 [getty] 0 4668 1048 0 20 0 21052 9584 select Ds - 0:00.02 [sshd] 0 4670 4668 0 20 0 14408 4656 pause Ds - 0:00.02 [csh] 0 4682 4668 0 20 0 13604 4332 ttyin Ds+ - 0:00.01 [csh] 0 4688 4670 0 21 0 13204 4100 select D+ - 0:00.00 [tmux] 0 4690 1 0 20 0 14096 5056 select Ds - 0:00.01 [tmux] 0 4691 4690 0 20 0 13608 4348 pause Ds - 0:00.01 [csh] 0 25060 4691 0 52 0 12932 3852 select D+ - 0:00.82 [sh] 0 25368 25060 0 52 0 12932 3720 nanslp D+ - 0:00.16 [sh] 0 45759 25060 0 27 0 12932 3844 piperd D+ - 0:00.00 [sh] 0 64142 25060 0 52 0 12932 3860 select D - 0:00.00 [sh] 0 64610 64142 0 52 0 12932 3856 wait D - 0:00.00 [sh] 65534 64611 64610 0 52 0 10304 1744 wait D - 0:00.00 [make] 65534 64652 64611 0 52 0 11644 3144 wait D - 0:00.00 [sh] 65534 64950 64652 0 52 0 12616 3668 wait D - 0:00.00 [sh] 65534 67594 64950 0 72 0 72556 51644 - RE - 0:00.00 [cpp] 65534 67595 64950 0 72 0 11388 532 - RE - 0:00.00 [egrep] ------------------------------------------------------------------------ vmstat -s 15831555 cpu context switches 2131396 device interrupts 768226 software interrupts 8267188 traps 14996492 system calls 26 kernel threads created 44730 fork() calls 22838 vfork() calls 2 rfork() calls 0 swap pager pageins 0 swap pager pages paged in 0 swap pager pageouts 0 swap pager pages paged out 2615 vnode pager pageins 10932 vnode pager pages paged in 167 vnode pager pageouts 1812 vnode pager pages paged out 0 page daemon wakeups 0 pages examined by the page daemon 0 clean page reclamation shortfalls 0 pages reactivated by the page daemon 2572938 copy-on-write faults 4867 copy-on-write optimized faults 5287614 zero fill pages zeroed 0 zero fill pages prezeroed 14 intransit blocking page faults 8296951 total VM faults taken 2407 page faults requiring I/O 0 pages affected by kernel thread creation 1745990 pages affected by fork() 1648727 pages affected by vfork() 114 pages affected by rfork() 10486245 pages freed 0 pages freed by daemon 0 pages freed by exiting processes 0 pages active 0 pages inactive 0 pages in the laundry queue 0 pages wired down 0 pages free 0 bytes per page 0 total name lookups cache hits (0% pos + 0% neg) system 0% per-directory deletions 0%, falsehits 0%, toolong 0% ------------------------------------------------------------------------ vmstat -m Type InUse MemUse HighUse Requests Size(s) CAM DEV 10 20K - 12 2048 entropy 1 1K - 1942 32,4096 CAM CCB 0 0K - 438443 2048 CAM path 17 1K - 65 32 CAM periph 16 4K - 87966 16,32,64,128,256 acpi_perf 4 2K - 4 512 acpipwr 11 1K - 11 64 hdaa 14 57K - 14 256,512,1024,2048,16384,32768 hdac 3 2K - 3 128,256,1024 hdacc 3 1K - 3 32 feeder 21 2K - 27 32,128 mixer 4 16K - 4 4096 CAM I/O Scheduler 6 1K - 6 128 CAM queue 14 6K - 26 16,32,64,512 scsi_cd 0 0K - 6 16 USB 60 92K - 67 16,32,128,256,512,1024,2048,4096,8192,16384 USBdev 44 10K - 44 32,64,128,256,512,4096 CAM dev queue 4 1K - 4 64 scsi_da 0 0K - 60 16,256 vtbuf 24 5712K - 70 4096 vt 11 6K - 11 512 DEVFS2 141 5K - 282 16,32 DEVFS3 515 129K - 709 256 DEVFS1 141 71K - 170 512 DEVFS_RULE 65 31K - 65 64,512 DEVFS 53 2K - 71 16,128 DEVFSP 3 1K - 3 64 NFSD V4client 1 1K - 1 256 NFSD lckfile 1 1K - 1 256 NFSD session 1 1K - 1 1024 pfs_nodes 388 194K - 388 512 pfs_vncache 9 1K - 13 64 GEOM 139 27K - 1293 16,32,64,128,256,512,1024,2048,8192,16384,32768 raid_data 0 0K - 174 32,128,256 isadev 8 1K - 8 128 cdev 5 2K - 5 256 filedesc 1 4K - 5 32,64,4096,16384,32768 sigio 0 0K - 1 64 filecaps 1 1K - 2506 16,32,64 kdtrace 206 40K - 136098 64,256 kenv 95 12K - 98 16,32,64,128,8192 kqueue 66 5K - 81287 64,256,512 proc-args 68 4K - 34506 16,32,64,128,256 hhook 13 4K - 13 256 ithread 96 17K - 96 32,128,256 prison 30 22K - 32 16,32,256,4096 KTRACE 100 13K - 100 128 linker 545 10577K - 682 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 lockf 40 5K - 4348 64,128 loginclass 3 1K - 3 64 cache 1 1K - 1 32 devbuf 18069 37898K - 20274 16,32,64,128,256,512,1024,2048,4096,8192,65536 temp 54 18K - 216655 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 evdev 5 5K - 5 1024 module 532 67K - 533 128 mtx_pool 2 72K - 2 8192,65536 osd 8 1K - 34 16,32,64,128,256 pmchooks 1 1K - 1 128 pmc 1 1K - 1 64 pgrp 32 4K - 747 128 session 29 4K - 675 128 proc 2 64K - 2 32768 subproc 184 323K - 67893 512,4096 cred 62 16K - 31160 256 acpiintr 1 1K - 1 64 plimit 24 6K - 56427 256 uidinfo 5 9K - 198 128,8192 dumper 1 1K - 1 512 acpica 6964 688K - 282579 16,32,64,128,256,512,1024,2048,4096 sysctl 0 0K - 27 64 sysctloid 6234 318K - 6392 16,32,64,128 sysctltmp 0 0K - 357537 16,64,256,1024 acpitask 1 64K - 1 65536 tidhash 1 64K - 1 65536 callout 5 2184K - 5 umtx 468 59K - 468 128 p1003.1b 1 1K - 1 16 SWAP 1 232K - 1 bus 1461 130K - 189623 16,32,64,128,256,1024,4096 bus-sc 111 1914K - 3733 16,32,128,256,512,1024,2048,4096,8192,16384,32768,65536 acpisem 114 15K - 114 128 devstat 10 21K - 10 32,4096 epoch 4 1K - 4 128 eventhandler 127 11K - 127 64,128 gtaskqueue 34 35K - 34 16,32,256,8192 kobj 357 1428K - 657 4096 Per-cpu 1 1K - 1 32 kbdmux 7 22K - 7 16,512,1024,2048,16384 rman 243 28K - 670 32,128 sbuf 0 0K - 1066 16,32,64,128,256,4096,8192,16384 toponodes 18 3K - 18 128 taskqueue 39 5K - 39 16,32,256 terminal 11 3K - 11 256 Unitno 31 2K - 113 32,64 vmem 3 260K - 13 2048,4096,8192,16384,32768,65536 ioctlops 0 0K - 1413 256,512,1024,2048 select 101 13K - 101 128 iov 0 0K - 759172 16,32,64,128,256,512,4096 msg 4 30K - 4 2048,4096,8192,16384 sem 4 106K - 4 2048,4096 shm 1 32K - 1 32768 tty 15 15K - 15 1024 pts 3 1K - 3 256 mbuf_tag 0 0K - 457 32 shmfd 1 8K - 1 8192 soname 6 1K - 396173 16,32,128 pcb 32 2201K - 2854 16,32,64,128,1024,2048,8192 acl 0 0K - 298 4096 vfscache 4 4145K - 4 256,16384,32768 cl_savebuf 0 0K - 533 64 vfs_hash 1 2048K - 1 vnodes 4 1K - 149 32,256 mount 476 25K - 20142 16,32,64,128,256,1024 statfs 0 0K - 95211 4096 vnodemarker 0 0K - 6732 512 chacha20random 1 1K - 1 1024 BPF 13 18K - 13 128,512,4096 ifnet 6 11K - 6 128,2048 ifaddr 72 21K - 72 16,32,64,128,256,512,2048,4096 ether_multi 76 6K - 76 16,32,64,128 clone 7 1K - 7 128 ipsec 3 1K - 3 256 lltable 28 9K - 28 256,512 tun 3 1K - 3 256 routetbl 100 9K - 109 32,64,128,512 vnet 1 1K - 1 64 vnet_data 1 240K - 1 vnet_data_free 1 1K - 1 32 igmp 5 1K - 5 128 in_multi 5 2K - 5 256 encap_export_host 12 1K - 12 32,64 sctp_a_it 0 0K - 9 16 sctp_vrf 1 1K - 1 64 sctp_ifa 14 2K - 14 128 sctp_ifn 5 1K - 5 128 sctp_iter 0 0K - 9 256 tfo_ccache 1 128K - 1 hostcache 1 32K - 1 32768 tcpfunc 1 1K - 1 64 syncache 1 68K - 1 in6_multi 60 8K - 60 32,256 ip6opt 1 1K - 2 32,256 mld 5 1K - 5 128 ip6ndp 13 2K - 18 32,64,256 inpcbpolicy 20 1K - 3622 32 secasvar 1 1K - 1 1024 sahead 1 1K - 1 1024 ipsecpolicy 2 2K - 2 256,1024 ipsec-saq 2 2K - 2 1024 crypto 2 2K - 2 512,1024 rpc 2 8K - 2 4096 audit_evclass 230 8K - 285 32 pagedep 517 641K - 7050 256 inodedep 5417 4756K - 79685 512 bmsafemap 10 11K - 69888 256,8192 newblk 1988 4593K - 200778 256 indirdep 23 3K - 850 128,32768 freefrag 3 1K - 2314 128 freeblks 200 50K - 58299 256 freefile 14 1K - 59177 64 diradd 5369 672K - 102743 128 mkdir 665 84K - 11638 128 dirrem 18 3K - 96801 128 newdirblk 491 31K - 6051 64 freework 307 39K - 80981 64,128 freedep 0 0K - 795 64 jaddref 1 1K - 114381 128 jremref 2 1K - 107327 128 jmvref 0 0K - 741 128 jnewblk 1 1K - 200777 128 jfreefrag 0 0K - 2314 128 jseg 26729 3342K - 39109 128 jsegdep 10920 683K - 424799 64 sbdep 0 0K - 820 64 savedino 0 0K - 4295 256 jblocks 2 1K - 2 128,256 softdep 1 1K - 1 512 ufs_dirhash 384 80K - 1449 16,32,64,128,256,512,1024 ufs_quota 1 2048K - 1 ufs_mount 3 41K - 5 512,8192,32768 vm_pgdata 1 1K - 1 128 UMAHash 5 22K - 13 512,1024,2048,4096,8192 memdesc 1 4K - 1 4096 pci_link 16 2K - 16 64,128 atkbddev 2 1K - 2 64 acpidev 61 4K - 61 64 CAM SIM 4 1K - 4 256 CAM XPT 46 4K - 123 16,32,64,128,256,512,1024,2048,65536 apmdev 1 1K - 1 128 madt_table 0 0K - 2 64,4096 intr 4 400K - 4 65536 io_apic 1 2K - 1 2048 local_apic 1 8K - 1 8192 MCA 14 3K - 14 32,128,256 cpus 2 1K - 2 32 msi 15 2K - 15 128 nexusdev 5 1K - 5 16 linux 32 2K - 32 64 gem_name 61 10K - 73 32,4096 drm_global 2 1K - 2 128,256 drm_dma 1 1K - 1 32 drm_sarea 1 1K - 1 16 drm_driver 70 2264K - 150 16,32,64,128,256,512,1024,2048,4096,8192,32768 drm_minor 2 1K - 2 128 drm_ctxbitmap 1 4K - 1 4096 drm_sman 18 3K - 18 128 drm_hashtab 2 4128K - 2 32768 drm_kms 92 25K - 123 16,32,64,128,256,512,2048,4096,8192 drm_vblank 7 1K - 7 32,256 ttm_pd 10 9K - 10 16,128,2048 ttm_rman 2 1K - 2 256 ttm_zone 2 1K - 2 64 ttm_poolmgr 1 1K - 1 512 nvidia 208 1183K - 209 16,32,64,128,256,512,2048,4096,8192,16384,65536 ksem 1 8K - 1 8192 nullfs_hash 1 2048K - 1 nullfs_node 3171 199K - 6018 64 nullfs_mount 17 1K - 27 32 tmpfs mount 3 1K - 5 256 tmpfs name 7868 273K - 108030 16,32,64,128 fdesc_mount 2 1K - 3 16 ------------------------------------------------------------------------ vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 224, 0, 131, 5, 135, 0, 0 UMA Zones: 936, 0, 134, 2, 138, 0, 0 UMA Slabs: 80, 0, 6032, 18, 6251, 0, 0 UMA Hash: 256, 0, 8, 7, 13, 0, 0 4 Bucket: 32, 0, 93, 1032, 11520, 0, 0 6 Bucket: 48, 0, 18, 1061, 4845, 0, 0 8 Bucket: 64, 0, 21, 1777, 56587, 21, 0 12 Bucket: 96, 0, 28, 505, 2110, 0, 0 16 Bucket: 128, 0, 77, 2217, 11169, 1, 0 32 Bucket: 256, 0, 2933, 1192, 57010, 9, 0 64 Bucket: 512, 0, 1531, 429, 17988,22077, 0 128 Bucket: 1024, 0, 578, 74, 9757, 2, 0 256 Bucket: 2048, 0, 816, 212, 24319,2571, 0 vmem: 1856, 0, 3, 1, 3, 0, 0 vmem btag: 56, 0, 18018, 3921, 49688, 155, 0 VM OBJECT: 256, 0, 32467, 29153, 1175586, 0, 0 RADIX NODE: 144, 0, 84577, 4388, 2039774, 0, 0 MAP: 240, 0, 3, 61, 3, 0, 0 KMAP ENTRY: 120, 0, 27, 501, 27, 0, 0 MAP ENTRY: 120, 0, 1505, 1003, 2871613, 0, 0 VMSPACE: 2560, 0, 40, 29, 67571, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 64 pcpu: 8, 0, 2897, 687, 2897, 0, 0 mt_stats_zone: 64, 0, 470, 554, 470, 0, 0 mt_zone: 24, 0, 470, 699, 470, 0, 0 16: 16, 0, 5740, 8065, 577599, 0, 0 32: 32, 0, 6998, 17377, 165207, 0, 0 64: 64, 0, 29768, 66146, 1457836, 0, 0 128: 128, 0, 41768, 31485, 930682, 0, 0 256: 256, 0, 4255, 52655, 1061390, 0, 0 512: 512, 0, 6471, 16649, 136736, 0, 0 1024: 1024, 0, 122, 206, 60157, 0, 0 2048: 2048, 0, 196, 60, 438829, 0, 0 4096: 4096, 0, 582, 24, 265581, 0, 0 8192: 8192, 0, 43, 2, 408, 0, 0 16384: 16384, 0, 16, 16, 491, 0, 0 32768: 32768, 0, 14, 7, 620, 0, 0 65536: 65536, 0, 26, 5, 1137, 0, 0 SLEEPQUEUE: 80, 0, 235, 292, 235, 0, 0 Files: 80, 0, 131, 619, 1005206, 0, 0 filedesc0: 1104, 0, 64, 62, 67597, 0, 0 rangeset pctrie nodes: 144, 0, 0, 0, 0, 0, 0 rl_entry: 40, 0, 93, 1007, 93, 0, 0 TURNSTILE: 136, 0, 235, 205, 235, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 umtx_shm: 88, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0 PROC: 1328, 0, 65, 52, 67596, 0, 0 THREAD: 1488, 0, 191, 43, 1021, 0, 0 cpuset: 104, 0, 16, 573, 5475, 0, 0 domainset: 40, 0, 0, 1054, 5458, 0, 0 audit_record: 1280, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 6475185, 256, 1526, 380257, 0, 0 mbuf: 256, 6475185, 1, 1528, 667250, 0, 0 mbuf_cluster: 2048, 1011746, 1771, 9, 1771, 0, 0 mbuf_jumbo_page: 4096, 505873, 0, 26, 31, 0, 0 mbuf_jumbo_9k: 9216, 449664, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 337248, 0, 0, 0, 0, 0 epoch_record pcpu: 256, 0, 4, 60, 4, 0, 0 ttyinq: 160, 0, 255, 120, 555, 0, 0 ttyoutq: 256, 0, 134, 121, 294, 0, 0 DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0 FPU_save_area: 832, 0, 0, 0, 0, 0, 0 g_bio: 376, 0, 0, 11750, 789956, 0, 0 nvme_request: 128, 0, 0, 0, 0, 0, 0 cryptop: 128, 0, 0, 0, 0, 0, 0 cryptodesc: 120, 0, 0, 0, 0, 0, 0 crypto_session: 24, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0 VNODE: 480, 0, 38597, 32747, 200711, 0, 0 VNODEPOLL: 120, 0, 0, 0, 0, 0, 0 BUF TRIE: 144, 0, 5679, 98919, 157900, 0, 0 NAMEI: 1024, 0, 0, 80, 2611442, 0, 0 rentr: 24, 0, 0, 1169, 300, 0, 0 S VFS Cache: 108, 0, 31023, 10732, 369981, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 1564, 12044, 38479, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 NCLNODE: 592, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 857, 107, 2131, 0, 0 procdesc: 136, 0, 1, 144, 2, 0, 0 AIO: 208, 0, 0, 0, 0, 0, 0 AIOP: 32, 0, 0, 0, 0, 0, 0 AIOCB: 752, 0, 0, 0, 0, 0, 0 AIOLIO: 280, 0, 0, 0, 0, 0, 0 pipe: 760, 0, 9, 86, 22164, 0, 0 Mountpoints: 2744, 0, 33, 6, 49, 0, 0 ksiginfo: 112, 0, 100, 950, 2518, 0, 0 itimer: 352, 0, 0, 121, 178, 0, 0 KNOTE: 160, 0, 0, 250, 7575, 0, 0 socket: 872, 519552, 39, 53, 8271, 0, 0 unpcb: 256, 519555, 17, 238, 4596, 0, 0 IPsec SA lft_c: 16, 0, 0, 0, 0, 0, 0 ipq: 56, 31666, 0, 284, 1, 0, 0 udp_inpcb: 488, 519552, 5, 115, 3054, 0, 0 udpcb: 32, 519625, 5, 1120, 3054, 0, 0 tcp_inpcb: 488, 519552, 12, 140, 565, 0, 0 tcpcb: 976, 519552, 12, 56, 565, 0, 0 tcptw: 88, 27810, 0, 270, 58, 0, 0 syncache: 168, 15364, 0, 69, 1, 0, 0 hostcache: 96, 15375, 4, 160, 6, 0, 0 sackhole: 32, 0, 0, 375, 3, 0, 0 tfo: 4, 0, 0, 0, 0, 0, 0 tfo_ccache_entries: 80, 0, 0, 0, 0, 0, 0 tcpreass: 48, 63246, 0, 332, 3, 0, 0 tcp_log: 400, 1000000, 0, 0, 0, 0, 0 tcp_log_bucket: 144, 0, 0, 0, 0, 0, 0 tcp_log_node: 120, 0, 0, 0, 0, 0, 0 sctp_ep: 1280, 519552, 0, 0, 0, 0, 0 sctp_asoc: 2408, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80012, 0, 581, 13, 0, 0 sctp_raddr: 736, 80000, 0, 0, 0, 0, 0 sctp_chunk: 152, 400010, 0, 0, 0, 0, 0 sctp_readq: 152, 400010, 0, 0, 0, 0, 0 sctp_stream_msg_out: 112, 400015, 0, 0, 0, 0, 0 sctp_asconf: 40, 400000, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0 udplite_inpcb: 488, 519552, 0, 0, 0, 0, 0 ripcb: 488, 519552, 3, 61, 3, 0, 0 rtentry: 208, 0, 63, 184, 66, 0, 0 selfd: 64, 0, 154, 900, 1717806, 0, 0 swpctrie: 144, 2023515, 0, 0, 0, 0, 0 swblk: 136, 2023504, 0, 0, 0, 0, 0 FFS inode: 160, 0, 27373, 7052, 86551, 0, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 27373, 7037, 86550, 0, 0 nvidia_stack_t: 12288, 0, 1, 1, 3, 0, 0 TMPFS dirent: 64, 0, 7867, 41485, 54027, 0, 0 TMPFS node: 232, 0, 7923, 41122, 54028, 0, 0 TMPFS dirent: 64, 0, 0, 0, 0, 0, 0 TMPFS node: 232, 0, 1, 67, 1, 0, 0 TMPFS dirent: 64, 0, 1, 1053, 79, 0, 0 TMPFS node: 232, 0, 41, 214, 80, 0, 0 ------------------------------------------------------------------------ vmstat -i interrupt total rate irq16: ehci0 80299 1444 irq23: ehci1 144788 2604 cpu0:timer 1152857 20732 cpu1:timer 801388 14412 cpu2:timer 715379 12865 cpu3:timer 718580 12922 irq264: hdac0 3 0 irq265: hdac1 6 0 irq266: xhci0 1407762 25316 irq267: hdac2 56 1 irq268: re0 235923 4243 irq269: ahci0 261951 4711 irq270: vgapci0 608 11 Total 5519600 99261 ------------------------------------------------------------------------ pstat -T 131/519551 files 0M/0M swap space ------------------------------------------------------------------------ pstat -s Device 512-blocks Used Avail Capacity ------------------------------------------------------------------------ iostat tty ada0 da0 da1 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 6 38.92 5 0.19 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 ------------------------------------------------------------------------ ipcs -a Message Queues: T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME Semaphores: T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME ------------------------------------------------------------------------ ipcs -T msginfo: msgmax: 16384 (max characters in a message) msgmni: 40 (# of message queues) msgmnb: 2048 (max characters in a message queue) msgtql: 40 (max # of messages in system) msgssz: 8 (size of a message segment) msgseg: 2048 (# of message segments in system) shminfo: shmmax: 536870912 (max shared memory segment size) shmmin: 1 (min shared memory segment size) shmmni: 192 (max number of shared memory identifiers) shmseg: 128 (max shared memory segments per process) shmall: 131072 (max amount of shared memory in pages) seminfo: semmni: 50 (# of semaphore identifiers) semmns: 340 (# of semaphores in system) semmnu: 150 (# of undo structures in system) semmsl: 340 (max # of semaphores per id) semopm: 100 (max # of operations per semop call) semume: 50 (max # of undo entries per process) semusz: 632 (size in bytes of undo structure) semvmx: 32767 (semaphore maximum value) semaem: 16384 (adjust on exit max value) ------------------------------------------------------------------------ netstat -s tcp: 96240 packets sent 13557 data packets (2131686 bytes) 8 data packets (8962 bytes) retransmitted 0 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 81258 ack-only packets (3366 delayed) 0 URG only packets 0 window probe packets 317 window update packets 1100 control packets 177922 packets received 13722 acks (for 2131715 bytes) 12 duplicate acks 0 acks for unsent data 164550 packets (217025714 bytes) received in-sequence 0 completely duplicate packets (0 bytes) 0 old duplicate packets 0 packets with some dup. data (0 bytes duped) 11 out-of-order packets (8895 bytes) 0 packets (0 bytes) of data after window 0 window probes 6 window update packets 433 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 discarded due to memory problems 551 connection requests 1 connection accept 0 bad connection attempts 0 listen queue overflows 0 ignored RSTs in the windows 552 connections established (including accepts) 538 times used RTT from hostcache 538 times used RTT variance from hostcache 0 times used slow-start threshold from hostcache 553 connections closed (including 2 drops) 544 connections updated cached RTT on close 544 connections updated cached RTT variance on close 0 connections updated cached ssthresh on close 0 embryonic connections dropped 13722 segments updated rtt (of 13900 attempts) 2 retransmit timeouts 0 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 0 keepalive timeouts 0 keepalive probes sent 0 connections dropped by keepalive 9307 correct ACK header predictions 163045 correct data packet header predictions 1 syncache entry added 0 retransmitted 0 dupsyn 0 dropped 1 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 1 cookie sent 0 cookies received 6 hostcache entries added 0 bucket overflow 1 SACK recovery episode 6 segment rexmits in SACK recovery episodes 8244 byte rexmits in SACK recovery episodes 11 SACK options (SACK blocks) received 11 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window 0 packets with matching signature received 0 packets with bad signature received 0 times failed to make signature due to no SA 0 times unexpected signature received 0 times no signature provided by segment 0 Path MTU discovery black hole detection activations 0 Path MTU discovery black hole detection min MSS activations 0 Path MTU discovery black hole detection failures TCP connection count by state: 0 connections in CLOSED state 9 connections in LISTEN state 0 connections in SYN_SENT state 0 connections in SYN_RCVD state 3 connections in ESTABLISHED state 0 connections in CLOSE_WAIT state 0 connections in FIN_WAIT_1 state 0 connections in CLOSING state 0 connections in LAST_ACK state 0 connections in FIN_WAIT_2 state 0 connections in TIME_WAIT state udp: 6713 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 3 with no checksum 17 dropped due to no socket 119 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 6577 delivered 9861 datagrams output 0 times multicast source filter matched ip: 17123 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 2 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 1 packet reassembled ok 17120 packets for this host 0 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 2 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 18885 packets sent from this host 7 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 823 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 17 calls to icmp_error 0 errors not generated in response to an icmp message Output histogram: destination unreachable: 17 0 messages with bad code fields 0 messages less than the minimum length 0 messages with bad checksum 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored 0 message responses generated 0 invalid return addresses 0 no return routes ipsec: 0 inbound packets violated process security policy 0 inbound packets failed due to insufficient memory 0 invalid inbound packets 0 outbound packets violated process security policy 0 outbound packets with no SA available 0 outbound packets failed due to insufficient memory 0 outbound packets with no route available 0 invalid outbound packets 0 outbound packets with bundled SAs 0 spd cache hits 0 spd cache misses 0 clusters copied during clone 0 mbufs inserted during makespace ah: 0 packets shorter than header shows 0 packets dropped; protocol family not supported 0 packets dropped; no TDB 0 packets dropped; bad KCR 0 packets dropped; queue full 0 packets dropped; no transform 0 replay counter wraps 0 packets dropped; bad authentication detected 0 packets dropped; bad authentication length 0 possible replay packets detected 0 packets in 0 packets out 0 packets dropped; invalid TDB 0 bytes in 0 bytes out 0 packets dropped; larger than IP_MAXPACKET 0 packets blocked due to policy 0 crypto processing failures 0 tunnel sanity check failures esp: 0 packets shorter than header shows 0 packets dropped; protocol family not supported 0 packets dropped; no TDB 0 packets dropped; bad KCR 0 packets dropped; queue full 0 packets dropped; no transform 0 packets dropped; bad ilen 0 replay counter wraps 0 packets dropped; bad encryption detected 0 packets dropped; bad authentication detected 0 possible replay packets detected 0 packets in 0 packets out 0 packets dropped; invalid TDB 0 bytes in 0 bytes out 0 packets dropped; larger than IP_MAXPACKET 0 packets blocked due to policy 0 crypto processing failures 0 tunnel sanity check failures ipcomp: 0 packets shorter than header shows 0 packets dropped; protocol family not supported 0 packets dropped; no TDB 0 packets dropped; bad KCR 0 packets dropped; queue full 0 packets dropped; no transform 0 replay counter wraps 0 packets in 0 packets out 0 packets dropped; invalid TDB 0 bytes in 0 bytes out 0 packets dropped; larger than IP_MAXPACKET 0 packets blocked due to policy 0 crypto processing failures 0 packets sent uncompressed; size < compr. algo. threshold 0 packets sent uncompressed; compression was useless arp: 4 ARP requests sent 1412 ARP replies sent 12674 ARP requests received 1 ARP reply received 12675 ARP packets received 0 total packets dropped due to no ARP entry 0 ARP entrys timed out 0 Duplicate IPs seen ip6: 168638 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 168089 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 88573 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 12 multicast packets which we don't join Input histogram: hop by hop: 11 TCP: 167939 ICMP6: 688 Mbuf statistics: 12 one mbuf 168626 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not contiguous 0 tunneling packets that can't find gif 0 packets discarded because of too many headers 0 failures of source address selection source addresses on an outgoing I/F 182 link-locals 1465 globals source addresses of same scope 181 link-locals 1465 globals source addresses of a different scope 1 link-local Source addresses selection rule applied: 1647 first candidate 1465 appropriate scope icmp6: 0 calls to icmp6_error 0 errors not generated in response to an icmp6 message 0 errors not generated because of rate limitation Output histogram: router solicitation: 1 neighbor solicitation: 367 neighbor advertisement: 187 MLDv2 listener report: 13 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Input histogram: router advertisement: 138 neighbor solicitation: 187 neighbor advertisement: 362 Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 0 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes ipsec6: 0 inbound packets violated process security policy 0 inbound packets failed due to insufficient memory 0 invalid inbound packets 0 outbound packets violated process security policy 0 outbound packets with no SA available 0 outbound packets failed due to insufficient memory 0 outbound packets with no route available 0 invalid outbound packets 0 outbound packets with bundled SAs 0 spd cache hits 0 spd cache misses 0 clusters copied during clone 0 mbufs inserted during makespace rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output pfkey: 0 requests sent from userland 0 bytes sent from userland 0 messages with invalid length field 0 messages with invalid version field 0 messages with invalid message type field 0 messages too short 0 messages with memory allocation failure 0 messages with duplicate extension ------------------------------------------------------------------------ netstat -m 257/3054/3311 mbufs in use (current/cache/total) 245/1535/1780/1011746 mbuf clusters in use (current/cache/total/max) 256/1526 mbuf+clusters out of packet secondary zone in use (current/cache) 0/26/26/505873 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/449664 9k jumbo clusters in use (current/cache/total/max) 0/0/0/337248 16k jumbo clusters in use (current/cache/total/max) 554K/3937K/4491K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 sendfile syscalls 0 sendfile syscalls completed without I/O request 0 requests for I/O initiated by sendfile 0 pages read by sendfile as part of a request 0 pages were valid at time of a sendfile request 0 pages were valid and substituted to bogus page 0 pages were requested for read ahead by applications 0 pages were read ahead by sendfile 0 times sendfile encountered an already busy page 0 requests for sfbufs denied 0 requests for sfbufs delayed ------------------------------------------------------------------------