Bug 249258

Summary:	OpenZFS kstat.zfs.misc.dbufs extremely large
Product:	Base System	Reporter:	Eirik Oeverby <ltning-freebsd>
Component:	kern	Assignee:	Ryan Moeller <freqlabs>
Status:	Closed FIXED
Severity:	Affects Many People	CC:	freqlabs, lwhsu, mmacy
Priority:	---
Version:	CURRENT
Hardware:	amd64
OS:	Any

Description Eirik Oeverby 2020-09-11 21:26:24 UTC

Anything depending on the output of 'sysctl -a' or 'sysctl kstat.zfs.misc.dbufs' will effectively hang 'forever' on my system, and reportedly others too. The system in question is running a poudriere jail, which builds a few hundred packages every night.

System is a 8-core 256GB system; resource congestion is not an issue.

E.g. Puppet will be useless on this system, since 'sysctl -a' is run during fact collection.

Suggestion: Do not gather/output kstat.zfs.misc.dbufs unless specifically requested.

Comment 1 Eirik Oeverby 2020-09-11 21:35:39 UTC

Example output:

# time sysctl kstat.zfs.misc.dbufs | wc
       0       0       0
  186084 6885040 54442005

real	4m49.726s
user	0m0.352s
sys	4m49.402s

Comment 2 Eirik Oeverby 2020-09-17 19:01:30 UTC

Is anything happening here? This breaks various applications that rely on sysctl -a output, including Puppet (configuration management).

Comment 3 Ryan Moeller freebsd_committer

2020-09-19 13:40:00 UTC

This is going to be fixed, but it is waiting on some other things that need to land first.

Comment 4 Ryan Moeller freebsd_committer

2020-09-30 00:14:16 UTC

This kstat will be a lot faster soon, though not hidden yet.
https://github.com/openzfs/zfs/pull/10993

Comment 5 Ryan Moeller freebsd_committer

2020-09-30 00:18:19 UTC

WIP to enable skipped kstats to remain discoverable here:
https://reviews.freebsd.org/D26560

After that is finished and MFC'd I'll make it so the root kstat node is skipped unless explicitly requested.

In the meantime, the change in the pull request in #4 should make things significantly faster.

Comment 6 commit-hook freebsd_committer

2020-10-24 16:26:29 UTC

A commit references this bug:

Author: freqlabs
Date: Sat Oct 24 16:25:53 UTC 2020
New revision: 367010
URL: https://svnweb.freebsd.org/changeset/base/367010

Log:
  Skip RAW kstat sysctls by default

  hese kstats are often expensive to compute so we want to avoid them
  unless specifically requested.

  The following kstats are affected by this change:

  kstat.zfs.${pool}.multihost
  kstat.zfs.${pool}.misc.state
  kstat.zfs.${pool}.txgs
  kstat.zfs.misc.fletcher_4_bench
  kstat.zfs.misc.vdev_raidz_bench
  kstat.zfs.misc.dbufs
  kstat.zfs.misc.dbgmsg

  PR:		249258
  Reported by:	mjg
  Reviewed by:	mjg, allanjude
  Obtained from:	https://github.com/openzfs/zfs/pull/11099
  Sponsored by:	iXsystems, Inc.

Changes:
  head/sys/contrib/openzfs/module/os/freebsd/spl/spl_kstat.c