Bug 243638 - [ufs SU] panic: worklist_remove: mkdir(0x29) not on list
Summary: [ufs SU] panic: worklist_remove: mkdir(0x29) not on list
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.1-RELEASE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-fs (Nobody)
URL:
Keywords: crash
Depends on:
Blocks:
 
Reported: 2020-01-27 00:20 UTC by Dirk Meyer
Modified: 2022-10-12 00:50 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dirk Meyer freebsd_committer freebsd_triage 2020-01-27 00:20:45 UTC
upgrading FreeBSD 12.0 to FreeBSD-12.1-p1
running poudriere bulk -J1

FreeBSD netcom2.dinoex.sub.de 12.1-RELEASE-p1 FreeBSD 12.1-RELEASE-p1 r357113 GENERIC  amd64

panic: worklist_remove: 0xfffff80404be0580 mkdir(0x29) not on list

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
panic: worklist_remove: 0xfffff80404be0580 mkdir(0x29) not on list
cpuid = 3
time = 1580030396
KDB: stack backtrace:
#0 0xffffffff80c1d297 at kdb_backtrace+0x67
#1 0xffffffff80bd05cd at vpanic+0x19d
#2 0xffffffff80bd0423 at panic+0x43
#3 0xffffffff80edef99 at free_newdirblk+0x209
#4 0xffffffff80ed0686 at cancel_newblk+0xc6
#5 0xffffffff80ed049e at cancel_allocdirect+0x9e
#6 0xffffffff80ed276b at softdep_setup_freeblocks+0x7eb
#7 0xffffffff80ebcb7a at ffs_truncate+0x6fa
#8 0xffffffff80ef06ac at ufs_inactive+0x22c
#9 0xffffffff8122b4ec at VOP_INACTIVE_APV+0x7c
#10 0xffffffff80ca0e63 at vinactive+0xf3
#11 0xffffffff80ca1373 at vputx+0x2c3
#12 0xffffffff80cab3cd at kern_rmdirat+0x1bd
#13 0xffffffff810a8984 at amd64_syscall+0x364
#14 0xffffffff81082330 at fast_syscall_common+0x101
Uptime: 2h3m55sDumping 1374 out of 16235 MB:..2%..11%..21%..31%..41%..51%..61%..72%..81%..91%

Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/linprocfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/linprocfs.ko
Reading symbols from /boot/kernel/linux_common.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux_common.ko.debug...done.
done.
Loaded symbols for /boot/kernel/linux_common.ko
Reading symbols from /boot/kernel/linsysfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/linsysfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/linsysfs.ko
Reading symbols from /boot/kernel/ums.ko...Reading symbols from /usr/lib/debug//boot/kernel/ums.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ums.ko
Reading symbols from /boot/kernel/uhid.ko...Reading symbols from /usr/lib/debug//boot/kernel/uhid.ko.debug...done.
done.
Loaded symbols for /boot/kernel/uhid.ko
Reading symbols from /boot/kernel/linux.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux.ko.debug...done.
done.
Loaded symbols for /boot/kernel/linux.ko
Reading symbols from /boot/kernel/linux64.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux64.ko.debug...done.
done.
Loaded symbols for /boot/kernel/linux64.ko
Reading symbols from /boot/kernel/speaker.ko...Reading symbols from /usr/lib/debug//boot/kernel/speaker.ko.debug...done.
done.
Loaded symbols for /boot/kernel/speaker.ko
Reading symbols from /boot/kernel/sem.ko...Reading symbols from /usr/lib/debug//boot/kernel/sem.ko.debug...done.
done.
Loaded symbols for /boot/kernel/sem.ko
Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/fdescfs.ko
Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/nullfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/nullfs.ko
Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/tmpfs.ko
#0  doadump () at src/sys/amd64/include/pcpu.h:234
#1  0xffffffff80bd01c8 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:451
#2  0xffffffff80bd0629 in vpanic (fmt=<value optimized out>, 
    ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:877
#3  0xffffffff80bd0423 in panic (fmt=<value optimized out>)
    at /usr/src/sys/kern/kern_shutdown.c:804
#4  0xffffffff80edef99 in free_newdirblk (newdirblk=0xfffff803e84f77c0)
    at src/sys/amd64/include/atomic.h:461
#5  0xffffffff80ed0686 in cancel_newblk (newblk=0xfffff804037fad00, 
    wk=<value optimized out>, wkhd=<value optimized out>)
    at /usr/src/sys/ufs/ffs/ffs_softdep.c:7456
#6  0xffffffff80ed049e in cancel_allocdirect (adphead=<value optimized out>, 
    adp=0xfffff804037fad00, freeblks=<value optimized out>)
    at /usr/src/sys/ufs/ffs/ffs_softdep.c:7399
#7  0xffffffff80ed276b in softdep_setup_freeblocks (ip=0xfffff802d486a000, 
    length=0, flags=2048) at /usr/src/sys/ufs/ffs/ffs_softdep.c:6988
#8  0xffffffff80ebcb7a in ffs_truncate (vp=0xfffff802d8eaed20, length=0, 
    flags=<value optimized out>, cred=<value optimized out>)
    at /usr/src/sys/ufs/ffs/ffs_inode.c:398
#9  0xffffffff80ef06ac in ufs_inactive (ap=<value optimized out>)
    at /usr/src/sys/ufs/ufs/ufs_inode.c:133
#10 0xffffffff8122b4ec in VOP_INACTIVE_APV (vop=0xffffffff81b60870, 
    a=0xfffffe0087f93788) at vnode_if.c:1955
#11 0xffffffff80ca0e63 in vinactive (vp=0xfffff802d8eaed20, 
    td=0xfffff80006a9e5e0) at vnode_if.h:807
#12 0xffffffff80ca1373 in vputx (vp=0xfffff802d8eaed20, func=2)
    at /usr/src/sys/kern/vfs_subr.c:2914
#13 0xffffffff80cab3cd in kern_rmdirat (td=0xfffff80006a9e5e0, fd=-100, 
    path=0x8006834f8 <Address 0x8006834f8 out of bounds>, 
    pathseg=UIO_USERSPACE) at /usr/src/sys/kern/vfs_syscalls.c:3762
#14 0xffffffff810a8984 in amd64_syscall (td=0xfffff80006a9e5e0, traced=0)
    at src/sys/amd64/amd64/../../kern/subr_syscall.c:135
#15 0xffffffff81082330 in fast_syscall_common ()
    at /usr/src/sys/amd64/amd64/exception.S:581
#16 0x000000080030e8da in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language:  auto; currently minimal
(kgdb) 
------------------------------------------------------------------------
ps -axlww

 UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN   STAT TT       TIME COMMAND
   0     0     0   0 -16  0     0     0 swapin   DLs   -    0:00.03 [kernel]
   0     1     0   0  26  0  9960   896 wait     DLs   -    0:00.00 [init]
   0     2     0   0 -16  0     0     0 crypto_w DL    -    0:00.00 [crypto]
   0     3     0   0 -16  0     0     0 crypto_r DL    -    0:00.00 [crypto returns 0]
   0     4     0   0 -16  0     0     0 crypto_r DL    -    0:00.00 [crypto returns 1]
   0     5     0   0 -16  0     0     0 crypto_r DL    -    0:00.00 [crypto returns 2]
   0     6     0   0 -16  0     0     0 crypto_r DL    -    0:00.00 [crypto returns 3]
   0     7     0   0 -16  0     0     0 -        RL    -    0:00.06 [cam]
   0     8     0   0 -16  0     0     0 -        DL    -    0:00.00 [soaiod1]
   0     9     0   0 -16  0     0     0 -        DL    -    0:00.00 [soaiod2]
   0    10     0   0 -16  0     0     0 audit_wo DL    -    0:00.00 [audit]
   0    11     0   0 155  0     0     0 -        RL    -  433:10.76 [idle]
   0    12     0   0 -56  0     0     0 -        WL    -    0:11.08 [intr]
   0    13     0   0  -8  0     0     0 -        DL    -    0:00.00 [geom]
   0    14     0   0 -16  0     0     0 seqstate DL    -    0:00.00 [sequencer 00]
   0    15     0   0 -68  0     0     0 -        DL    -    0:00.39 [usb]
   0    16     0   0 -16  0     0     0 -        DL    -    0:00.00 [soaiod3]
   0    17     0   0 -16  0     0     0 -        DL    -    0:00.00 [soaiod4]
   0    18     0   0 -16  0     0     0 waiting_ DL    -    0:00.00 [sctp_iterator]
   0    19     0   0 -16  0     0     0 tzpoll   DL    -    0:00.02 [acpi_thermal]
   0    20     0   0 -16  0     0     0 cooling  DL    -    0:00.00 [acpi_cooling1]
   0    21     0   0 -16  0     0     0 -        DL    -    0:00.42 [rand_harvestq]
   0    22     0   0 -16  0     0     0 psleep   DL    -    0:00.12 [pagedaemon]
   0    23     0   0 -16  0     0     0 psleep   DL    -    0:00.00 [vmdaemon]
   0    24     0   0  20  0     0     0 qsleep   DL    -    0:03.87 [bufdaemon]
   0    25     0   0  20  0     0     0 vlruwt   DL    -    0:00.06 [vnlru]
   0    26     0   0  16  0     0     0 syncer   DL    -    0:03.04 [syncer]
   0   534     1   0  20  0 11048  2388 select   Ds    -    0:00.00 [moused]
   0   563     1   0  20  0 11388  2620 select   Ds    -    0:00.00 [dhclient]
   0   566     1   0  39  0 11612  2760 select   Ds    -    0:00.00 [dhclient]
  65   612     1   0  20  0 11752  2876 select   DCs   -    0:00.00 [dhclient]
   0   613     1   0  20  0 10456  1476 select   Ds    -    0:00.34 [devd]
   0   621     1   0  20  0 16184  5456 select   Ds    -    0:00.49 [tincd]
   0   695     1   0  20  0 10908  2216 select   Ds    -    0:00.00 [rtsold]
   0   716     1   0  20  0 11400  2668 select   Ds    -    0:00.25 [syslogd]
   0   930     1   0  52  0 11084  2452 select   Ds    -    0:00.00 [lpd]
  25   966     1   0  20  0 16800  6368 pause    Ds    -    0:00.00 [sendmail]
   0   979     1   0  20  0 17156  6896 select   Ds    -    0:00.04 [openvpn]
   0   990     1   0  20  0 17104  6956 select   Ds    -    0:00.06 [openvpn]
   0  1032     1   0  20  0 19652  8372 select   Ds    -    0:00.00 [sshd]
   0  1035     1   0  20  0 11464  2676 nanslp   Ds    -    0:00.01 [cron]
   0  1048  1032   0  20  0 27344 15720 select   Ds    -    0:00.66 [sshd]
   0  1086  1048   0  20  0 14408  4652 pause    Ds    -    0:00.03 [csh]
   0  1118     1   0  52  0 11444  2600 select   Ds    -    0:00.00 [inetd]
   0  1143     1   0  52  0 10880  2288 ttyin    Ds+   -    0:00.00 [getty]
   0  1144     1   0  52  0 10880  2288 ttyin    Ds+   -    0:00.00 [getty]
   0  1145     1   0  52  0 10880  2288 ttyin    Ds+   -    0:00.00 [getty]
   0  1146     1   0  52  0 10880  2288 ttyin    Ds+   -    0:00.00 [getty]
   0  1147     1   0  52  0 10880  2288 ttyin    Ds+   -    0:00.00 [getty]
   0  1148     1   0  52  0 10880  2288 ttyin    Ds+   -    0:00.00 [getty]
   0  1149     1   0  52  0 10880  2288 ttyin    Ds+   -    0:00.00 [getty]
   0  1150     1   0  52  0 10880  2288 ttyin    Ds+   -    0:00.00 [getty]
   0  1205  1032   0  22  0 20300  9236 select   Ds    -    0:00.01 [sshd]
1001  1207  1205   0  20  0 20928  9504 select   D     -    0:00.00 [sshd]
1001  1208  1207   0  22  0 14380  4800 ttyin    Ds+   -    0:00.03 [csh]
   0  2113  1048   0  20  0 13604  4336 ttyin    Ds+   -    0:00.03 [csh]
   0  3402  3656   0  20  0 12932  3804 select   D+    -    0:00.83 [sh]
   0  3653  1086   0  21  0 13204  4052 select   D+    -    0:00.00 [tmux]
   0  3655     1   0  20  0 14468  5512 select   Ds    -    0:00.70 [tmux]
   0  3656  3655   0  20  0 13608  4344 pause    Ds    -    0:00.01 [csh]
   0  3713  3402   0  52  0 12932  3720 nanslp   D+    -    0:01.08 [sh]
   0  9037  3402   0  52  0 12932  3820 wait     D     -    0:00.00 [sh]
   0 22456  9037   0  52  0 10324  1776 wait     D     -    0:00.00 [make]
   0 22477 22456   0  52  0 10308  1780 wait     D     -    0:00.00 [make]
   0 22478 22477   0  72  0 10984  2480 -        R     -    0:00.00 [rm]
   0 24022  3402   0  26  0 12932  3796 piperd   D+    -    0:00.03 [sh]

------------------------------------------------------------------------
vmstat -s

  9627106 cpu context switches
  1816031 device interrupts
   115424 software interrupts
 39433009 traps
 62005325 system calls
       26 kernel threads created
   222615  fork() calls
    99431 vfork() calls
        2 rfork() calls
        0 swap pager pageins
        0 swap pager pages paged in
        0 swap pager pageouts
        0 swap pager pages paged out
     2251 vnode pager pageins
    14388 vnode pager pages paged in
     2029 vnode pager pageouts
    33166 vnode pager pages paged out
        0 page daemon wakeups
        0 pages examined by the page daemon
        0 clean page reclamation shortfalls
        0 pages reactivated by the page daemon
  8895388 copy-on-write faults
     6236 copy-on-write optimized faults
 28628674 zero fill pages zeroed
        0 zero fill pages prezeroed
       24 intransit blocking page faults
 39806249 total VM faults taken
     1971 page faults requiring I/O
        0 pages affected by kernel thread creation
  8267237 pages affected by  fork()
 13409617 pages affected by vfork()
      114 pages affected by rfork()
 49786058 pages freed
        0 pages freed by daemon
        0 pages freed by exiting processes
        0 pages active
        0 pages inactive
        0 pages in the laundry queue
        0 pages wired down
        0 pages free
        0 bytes per page
        0 total name lookups
          cache hits (0% pos + 0% neg) system 0% per-directory
          deletions 0%, falsehits 0%, toolong 0%

------------------------------------------------------------------------
vmstat -m

         Type InUse MemUse HighUse Requests  Size(s)
      CAM DEV    10    20K       -       12  2048
      entropy     1     1K       -    10339  32,4096
      CAM CCB     0     0K       -  1001742  2048
     CAM path    17     1K       -       65  32
   CAM periph    16     4K       -    12351  16,32,64,128,256
    acpi_perf     4     2K       -        4  512
      acpipwr    11     1K       -       11  64
         hdaa    14    57K       -       14  256,512,1024,2048,16384,32768
         hdac     3     2K       -        3  128,256,1024
        hdacc     3     1K       -        3  32
       feeder    21     2K       -       27  32,128
        mixer     4    16K       -        4  4096
CAM I/O Scheduler     6     1K       -        6  128
    CAM queue    14     6K       -       26  16,32,64,512
      scsi_cd     0     0K       -        6  16
          USB    60    92K       -       67  16,32,128,256,512,1024,2048,4096,8192,16384
       USBdev    44    10K       -       44  32,64,128,256,512,4096
CAM dev queue     4     1K       -        4  64
      scsi_da     0     0K       -       60  16,256
        vtbuf    24  4752K       -       94  4096
           vt    11     6K       -       11  512
       DEVFS2   132     5K       -      264  16,32
       DEVFS3   479   120K       -      832  256
       DEVFS1   132    66K       -      171  512
   DEVFS_RULE    65    31K       -       65  64,512
        DEVFS    53     2K       -       88  16,128
       DEVFSP     3     1K       -        3  64
NFSD V4client     1     1K       -        1  256
 NFSD lckfile     1     1K       -        1  256
 NFSD session     1     1K       -        1  1024
    pfs_nodes   388   194K       -      388  512
  pfs_vncache     8     1K       -       47  64
         GEOM   139    27K       -     1293  16,32,64,128,256,512,1024,2048,8192,16384,32768
    raid_data     0     0K       -      174  32,128,256
       isadev     8     1K       -        8  128
         cdev     5     2K       -        5  256
     filedesc     1     4K       -      349  32,64,4096,16384,32768
        sigio     0     0K       -        1  64
     filecaps     1     1K       -    55630  16,32,64
      kdtrace   204    39K       -   649877  64,256
         kenv    95    12K       -       98  16,32,64,128,8192
       kqueue    66     5K       -   329185  64,256,512
    proc-args    70     4K       -   192868  16,32,64,128,256
        hhook    13     4K       -       13  256
      ithread    95    16K       -       96  32,128,256
       prison    26    18K       -       38  16,32,256,4096
       KTRACE   100    13K       -      100  128
       linker   384  1172K       -      684  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
        lockf    37     5K       -   153749  64,128
   loginclass     3     1K       -        3  64
        cache     1     1K       -        1  32
       devbuf 18066 37897K       -    19110  16,32,64,128,256,512,1024,2048,4096,8192,65536
         temp    50    18K       -   974732  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
        evdev     5     5K       -        5  1024
       module   516    65K       -      533  128
     mtx_pool     2    72K       -        2  8192,65536
          osd     7     1K       -       42  16,32,64,128,256
     pmchooks     1     1K       -        1  128
          pmc     1     1K       -        1  64
         pgrp    34     5K       -      689  128
      session    31     4K       -      118  128
         proc     2    64K       -        2  32768
      subproc   193   328K       -   322226  512,4096
         cred   191    48K       -    37303  256
     acpiintr     1     1K       -        1  64
       plimit    25     7K       -    30650  256
      uidinfo     7     9K       -       43  128,8192
       dumper     1     1K       -        1  512
       acpica  6964   688K       -   193087  16,32,64,128,256,512,1024,2048,4096
       sysctl     0     0K       -       37  64
    sysctloid  5190   265K       -     6392  16,32,64,128
    sysctltmp     0     0K       -    54888  16,64,256,1024
     acpitask     1    64K       -        1  65536
      tidhash     1    64K       -        1  65536
      callout     5  2184K       -        5  
         umtx   504    63K       -      504  128
     p1003.1b     1     1K       -        1  16
         SWAP     1   232K       -        1  
          bus  1379   120K       -    38714  16,32,64,128,256,1024,4096
       bus-sc    84  1910K       -     3733  16,32,128,256,512,1024,2048,4096,8192,16384,32768,65536
      acpisem   114    15K       -      114  128
      devstat    10    21K       -       10  32,4096
        epoch     4     1K       -        4  128
 eventhandler   126    11K       -      127  64,128
   gtaskqueue    34    35K       -       34  16,32,256,8192
         kobj   348  1392K       -      657  4096
      Per-cpu     1     1K       -        1  32
       kbdmux     7    22K       -        7  16,512,1024,2048,16384
         rman   243    28K       -      670  32,128
         sbuf     0     0K       -     1066  16,32,64,128,256,4096,8192,16384
    toponodes    18     3K       -       18  128
    taskqueue    33     4K       -       39  16,32,256
     terminal    11     3K       -       11  256
       Unitno    32     2K       -     9539  32,64
         vmem     3   260K       -       19  2048,4096,8192,32768,65536
     ioctlops     0     0K       -      408  256,512,1024,2048
       select    81    11K       -       81  128
          iov     0     0K       -  6146374  16,32,64,128,256,512,1024,4096
          msg     4    30K       -        4  2048,4096,8192,16384
          sem     4   106K       -        4  2048,4096
          shm     1    32K       -        1  32768
          tty    16    16K       -       16  1024
          pts     4     1K       -        4  256
     mbuf_tag     0     0K       -      160  32
        shmfd     1     8K       -        1  8192
       soname     6     1K       -    75798  16,32,128
          pcb    32  2201K       -      902  16,32,64,128,1024,2048,8192
          acl     0     0K       -    26666  4096
     vfscache     4  4145K       -        4  256,16384,32768
   cl_savebuf     0     0K       -     2485  64
     vfs_hash     1  2048K       -        1  
       vnodes     3     1K       -      228  32,256
        mount   489    25K       -    14344  16,32,64,128,256,1024
       statfs     0     0K       -   307360  4096
  vnodemarker     0     0K       -     1457  512
      fadvise     0     0K       -        1  32
chacha20random     1     1K       -        1  1024
          BPF    13    18K       -       13  128,512,4096
        ifnet     6    11K       -        6  128,2048
       ifaddr    72    21K       -       72  16,32,64,128,256,512,2048,4096
  ether_multi    76     6K       -       76  16,32,64,128
        clone     7     1K       -        7  128
        ipsec     3     1K       -        3  256
      lltable    28     9K       -       28  256,512
          tun     3     1K       -        3  256
     routetbl   100     9K       -      109  32,64,128,512
         vnet     1     1K       -        1  64
    vnet_data     1   240K       -        1  
vnet_data_free     1     1K       -        1  32
         igmp     5     1K       -        5  128
     in_multi     5     2K       -        5  256
encap_export_host    12     1K       -       12  32,64
    sctp_a_it     0     0K       -       10  16
     sctp_vrf     1     1K       -        1  64
     sctp_ifa    14     2K       -       14  128
     sctp_ifn     5     1K       -        5  128
    sctp_iter     0     0K       -       10  256
   tfo_ccache     1   128K       -        1  
    hostcache     1    32K       -        1  32768
      tcpfunc     1     1K       -        1  64
     syncache     1    68K       -        1  
    in6_multi    60     8K       -       60  32,256
       ip6opt     1     1K       -        2  32,256
          mld     5     1K       -        5  128
       ip6ndp    13     2K       -       18  32,64,256
  inpcbpolicy    21     1K       -     1074  32
     secasvar     1     1K       -        1  1024
       sahead     1     1K       -        1  1024
  ipsecpolicy     2     2K       -        2  256,1024
    ipsec-saq     2     2K       -        2  1024
       crypto     2     2K       -        2  512,1024
          rpc     2     8K       -        2  4096
audit_evclass   230     8K       -      285  32
      pagedep  1391   860K       -   102399  256
     inodedep 12680  8388K       -   838338  512
    bmsafemap    14    12K       -   437652  256,8192
       newblk 10211  6649K       -   795953  256
     indirdep    49     7K       -     2983  128,32768
     freefrag     8     1K       -    20374  128
     freeblks  3691   923K       -   496629  256
     freefile   110     7K       -   495570  64
       diradd  9304  1163K       -   558934  128
        mkdir  1946   244K       -   106146  128
       dirrem  3237   405K       -   633819  128
    newdirblk  1371    86K       -    53767  64
     freework  4777   598K       -   589796  64,128
      freedep     0     0K       -     2731  64
      jaddref    12     2K       -   665080  128
      jremref  3985   499K       -   759353  128
       jmvref     0     0K       -    17939  128
      jnewblk    32     4K       -   795952  128
    jfreefrag     0     0K       -    20374  128
         jseg 25608  3201K       -   138694  128
      jsegdep 27151  1697K       -  2240759  64
        sbdep     0     0K       -      152  64
     savedino     0     0K       -    83925  256
      jblocks     2     1K       -        2  128,256
      softdep     1     1K       -        1  512
  ufs_dirhash  2649   588K       -     6252  16,32,64,128,256,512,1024,2048,4096,8192,16384
    ufs_quota     1  2048K       -        1  
    ufs_mount     3    41K       -        5  512,8192,32768
    vm_pgdata     1     1K       -        1  128
      UMAHash     5    52K       -       18  512,1024,2048,4096,8192,16384
      memdesc     1     4K       -        1  4096
     pci_link    16     2K       -       16  64,128
     atkbddev     2     1K       -        2  64
      acpidev    61     4K       -       61  64
      CAM SIM     4     1K       -        4  256
      CAM XPT    46     4K       -      123  16,32,64,128,256,512,1024,2048,65536
       apmdev     1     1K       -        1  128
   madt_table     0     0K       -        2  64,4096
         intr     4   400K       -        4  65536
      io_apic     1     2K       -        1  2048
   local_apic     1     8K       -        1  8192
          MCA    14     3K       -       14  32,128,256
         cpus     2     1K       -        2  32
          msi    15     2K       -       15  128
     nexusdev     5     1K       -        5  16
        linux    31     2K       -       32  64
         ksem     1     8K       -        1  8192
  fdesc_mount     2     1K       -        4  16
  nullfs_hash     1  2048K       -        1  
  nullfs_node  3318   208K       -     3641  64
 nullfs_mount    18     1K       -       44  32
  tmpfs mount     3     1K       -        5  256
   tmpfs name 16857   496K       -    79456  16,32,64,128

------------------------------------------------------------------------
vmstat -z

ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               224,      0,     130,       6,     135,   0,   0
UMA Zones:              936,      0,     133,       3,     138,   0,   0
UMA Slabs:               80,      0,   21291,       9,   21547,   0,   0
UMA Hash:               256,      0,       7,       8,      13,   0,   0
4 Bucket:                32,      0,      78,    1297,   12802,   0,   0
6 Bucket:                48,      0,      39,    1040,   18455,   0,   0
8 Bucket:                64,      0,      21,    1777,  115407,  21,   0
12 Bucket:               96,      0,      32,     624,    7562,   0,   0
16 Bucket:              128,      0,     122,    1831,   25193,   1,   0
32 Bucket:              256,      0,    6377,    1633,  132611,   9,   0
64 Bucket:              512,      0,    3966,    1210,  115105,15271,   0
128 Bucket:            1024,      0,    1606,     438,   49686,   1,   0
256 Bucket:            2048,      0,    1016,     220,   64476, 543,   0
vmem:                  1856,      0,       3,       1,       3,   0,   0
vmem btag:               56,      0,   21185,   23190,  108088, 313,   0
VM OBJECT:              256,      0,  270939,   13206, 4659921,   0,   0
RADIX NODE:             144,      0,  190856,    6730, 8153429,   0,   0
MAP:                    240,      0,       3,      61,       3,   0,   0
KMAP ENTRY:             120,      0,      17,     511,      27,   0,   0
MAP ENTRY:              120,      0,    1844,    1159,15661178,   0,   0
VMSPACE:               2560,      0,      40,      35,  322046,   0,   0
fakepg:                 104,      0,       0,       0,       0,   0,   0
64 pcpu:                  8,      0,    2897,     687,    2897,   0,   0
mt_stats_zone:           64,      0,     434,     334,     470,   0,   0
mt_zone:                 24,      0,     434,     401,     470,   0,   0
16:                      16,      0,   10038,    4269,  190594,   0,   0
32:                      32,      0,   11696,   12429,  183748,   0,   0
64:                      64,      0,   47278,  146038, 9719192,   0,   0
128:                    128,      0,   57393,  171015, 4786996,   0,   0
256:                    256,      0,   16820,  100060, 2457925,   0,   0
512:                    512,      0,   14466,  107198, 1056664,   0,   0
1024:                  1024,      0,     101,     215,  275010,   0,   0
2048:                  2048,      0,     164,      62, 1002130,   0,   0
4096:                  4096,      0,     557,      38,  946448,   0,   0
8192:                  8192,      0,      37,       3,     412,   0,   0
16384:                16384,      0,      17,      14,     494,   0,   0
32768:                32768,      0,      10,       9,    2993,   0,   0
65536:                65536,      0,      22,       7,     554,   0,   0
SLEEPQUEUE:              80,      0,     253,     274,     253,   0,   0
Files:                   80,      0,     137,     763, 4412052,   0,   0
filedesc0:             1104,      0,      66,      69,  322075,   0,   0
rangeset pctrie nodes:    144,      0,       0,       0,       0,   0,   0
rl_entry:                40,      0,     104,     996,     104,   0,   0
TURNSTILE:              136,      0,     253,      67,     253,   0,   0
umtx pi:                 96,      0,       0,       0,       0,   0,   0
umtx_shm:                88,      0,       0,       0,       0,   0,   0
MAC labels:              40,      0,       0,       0,       0,   0,   0
PROC:                  1328,      0,      65,      61,  322074,   0,   0
THREAD:                1488,      0,     198,      54,    5853,   0,   0
cpuset:                 104,      0,      15,     543,    3721,   0,   0
domainset:               40,      0,       0,    1054,    3702,   0,   0
audit_record:          1280,      0,       0,       0,       0,   0,   0
mbuf_packet:            256, 6475185,     258,    2274,  874918,   0,   0
mbuf:                   256, 6475185,       1,    1519,  622461,   0,   0
mbuf_cluster:          2048, 1011746,    2530,      10,    2530,   0,   0
mbuf_jumbo_page:       4096, 505873,       0,      30,     580,   0,   0
mbuf_jumbo_9k:         9216, 449664,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384, 337248,       0,       0,       0,   0,   0
epoch_record pcpu:      256,      0,       4,      60,       4,   0,   0
ttyinq:                 160,      0,     315,     135,     615,   0,   0
ttyoutq:                256,      0,     165,      90,     325,   0,   0
DMAR_MAP_ENTRY:         120,      0,       0,       0,       0,   0,   0
FPU_save_area:          832,      0,       0,       0,       0,   0,   0
g_bio:                  376,      0,       0,   14680, 2932379,   0,   0
nvme_request:           128,      0,       0,       0,       0,   0,   0
cryptop:                128,      0,       0,       0,       0,   0,   0
cryptodesc:             120,      0,       0,       0,       0,   0,   0
crypto_session:          24,      0,       0,       0,       0,   0,   0
vtnet_tx_hdr:            24,      0,       0,       0,       0,   0,   0
VNODE:                  480,      0,  335173,   13211,  983502,   0,   0
VNODEPOLL:              120,      0,       0,       0,       0,   0,   0
BUF TRIE:               144,      0,    5445,   99153,  421973,   0,   0
NAMEI:                 1024,      0,       0,      80,11451299,   0,   0
rentr:                   24,      0,       0,    1169,    1247,   0,   0
S VFS Cache:            108,      0,  136962,   20538, 1404621,   0,   0
STS VFS Cache:          148,      0,       0,       0,       0,   0,   0
L VFS Cache:            328,      0,   20017,   14555,  183532,   0,   0
LTS VFS Cache:          368,      0,       0,       0,       0,   0,   0
NCLNODE:                592,      0,       0,       0,       0,   0,   0
DIRHASH:               1024,      0,    7716,     308,   13813,   0,   0
procdesc:               136,      0,       1,     144,       2,   0,   0
AIO:                    208,      0,       0,       0,       0,   0,   0
AIOP:                    32,      0,       0,       0,       0,   0,   0
AIOCB:                  752,      0,       0,       0,       0,   0,   0
AIOLIO:                 280,      0,       0,       0,       0,   0,   0
pipe:                   760,      0,       9,      86,  141272,   0,   0
Mountpoints:           2744,      0,      34,      11,      69,   0,   0
ksiginfo:               112,      0,     109,     941,     510,   0,   0
itimer:                 352,      0,       0,     121,      24,   0,   0
KNOTE:                  160,      0,       0,     250,    4067,   0,   0
socket:                 872, 519552,      43,      61,    5058,   0,   0
unpcb:                  256, 519555,      20,     235,    3937,   0,   0
IPsec SA lft_c:          16,      0,       0,       0,       0,   0,   0
ipq:                     56,  31666,       0,       0,       0,   0,   0
udp_inpcb:              488, 519552,       5,     115,     872,   0,   0
udpcb:                   32, 519625,       5,    1120,     872,   0,   0
tcp_inpcb:              488, 519552,      13,     107,     199,   0,   0
tcpcb:                  976, 519552,      13,      55,     199,   0,   0
tcptw:                   88,  27810,       0,     135,       7,   0,   0
syncache:               168,  15364,       0,     138,       2,   0,   0
hostcache:               96,  15375,      13,     397,      16,   0,   0
sackhole:                32,      0,       0,    1125,     174,   0,   0
tfo:                      4,      0,       0,       0,       0,   0,   0
tfo_ccache_entries:      80,      0,       0,       0,       0,   0,   0
tcpreass:                48,  63246,       0,     332,      47,   0,   0
tcp_log:                400, 1000000,       0,       0,       0,   0,   0
tcp_log_bucket:         144,      0,       0,       0,       0,   0,   0
tcp_log_node:           120,      0,       0,       0,       0,   0,   0
sctp_ep:               1280, 519552,       0,       0,       0,   0,   0
sctp_asoc:             2408,  40000,       0,       0,       0,   0,   0
sctp_laddr:              48,  80012,       0,     830,      13,   0,   0
sctp_raddr:             736,  80000,       0,       0,       0,   0,   0
sctp_chunk:             152, 400010,       0,       0,       0,   0,   0
sctp_readq:             152, 400010,       0,       0,       0,   0,   0
sctp_stream_msg_out:    112, 400015,       0,       0,       0,   0,   0
sctp_asconf:             40, 400000,       0,       0,       0,   0,   0
sctp_asconf_ack:         48, 400060,       0,       0,       0,   0,   0
udplite_inpcb:          488, 519552,       0,       0,       0,   0,   0
ripcb:                  488, 519552,       3,      61,       3,   0,   0
rtentry:                208,      0,      63,     184,      66,   0,   0
selfd:                   64,      0,     141,     913, 1157477,   0,   0
swpctrie:               144, 2023515,       0,       0,       0,   0,   0
swblk:                  136, 2023504,       0,       0,       0,   0,   0
FFS inode:              160,      0,  314699,   33676,  900231,   0,   0
FFS1 dinode:            128,      0,       0,       0,       0,   0,   0
FFS2 dinode:            256,      0,  314699,   33676,  900230,   0,   0
TMPFS dirent:            64,      0,   16856,   32744,   54181,   0,   0
TMPFS node:             232,      0,   16910,   32084,   54182,   0,   0
TMPFS dirent:            64,      0,       0,       0,       0,   0,   0
TMPFS node:             232,      0,       1,      67,       1,   0,   0
TMPFS dirent:            64,      0,       1,    1053,     343,   0,   0
TMPFS node:             232,      0,     173,      82,     344,   0,   0


------------------------------------------------------------------------
vmstat -i

interrupt                          total       rate
irq16: ehci0                       11342        169
irq23: ehci1                       19689        294
cpu0:timer                        553560       8271
cpu1:timer                        528160       7891
cpu2:timer                        531123       7935
cpu3:timer                        542861       8111
irq264: hdac0                          3          0
irq265: hdac1                          6          0
irq266: xhci0                     197911       2957
irq267: hdac2                         55          1
irq268: re0                       615205       9192
irq269: ahci0                     971820      14520
Total                            3971735      59342

------------------------------------------------------------------------
pstat -T

137/519551 files
0M/0M swap space

------------------------------------------------------------------------
pstat -s

Device          512-blocks     Used    Avail Capacity

------------------------------------------------------------------------
iostat

       tty            ada0              da0              da1             cpu
 tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
   2   919 22.75 131  2.92   0.00   0  0.00   0.00   0  0.00   3  0  1  0 97

------------------------------------------------------------------------
ipcs -a

Message Queues:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME   

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME   

Semaphores:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME   


------------------------------------------------------------------------
ipcs -T

msginfo:
        msgmax:        16384    (max characters in a message)
        msgmni:           40    (# of message queues)
        msgmnb:         2048    (max characters in a message queue)
        msgtql:           40    (max # of messages in system)
        msgssz:            8    (size of a message segment)
        msgseg:         2048    (# of message segments in system)

shminfo:
        shmmax:    536870912    (max shared memory segment size)
        shmmin:            1    (min shared memory segment size)
        shmmni:          192    (max number of shared memory identifiers)
        shmseg:          128    (max shared memory segments per process)
        shmall:       131072    (max amount of shared memory in pages)

seminfo:
        semmni:           50    (# of semaphore identifiers)
        semmns:          340    (# of semaphores in system)
        semmnu:          150    (# of undo structures in system)
        semmsl:          340    (max # of semaphores per id)
        semopm:          100    (max # of operations per semop call)
        semume:           50    (max # of undo entries per process)
        semusz:          632    (size in bytes of undo structure)
        semvmx:        32767    (semaphore maximum value)
        semaem:        16384    (adjust on exit max value)


------------------------------------------------------------------------
Comment 1 Conrad Meyer freebsd_committer freebsd_triage 2020-01-27 08:50:31 UTC
Fallout from CURRENT r356714 => stable/12 r357033?  r357113 is new enough, but I have no idea if 12.1-p1 actually contains that change or not.