Created attachment 175531 [details] patch to display dev activities on zarc screen While watching ZFS and its memory usage activities, it is more useful to watch disk activities at same time. I took -vm disk activity display to devs.c and made them reusable components to call from -vm and -zarc. Below is a screen shot. Total MFU MRU Anon Hdr L2Hdr Other ZFS ARC 185M 30M 64M 88M 380K 0K 909K rate hits misses total hits total misses arcstats : 0% 0 0 6150 2763 arcstats.demand_data : 0% 0 0 309 8 arcstats.demand_metadata : 0% 0 0 5828 1796 arcstats.prefetch_data : 0% 0 0 0 880 arcstats.prefetch_metadata: 0% 0 0 13 79 zfetchstats : 0% 0 0 146 7806 arcstats.l2 : 0% 0 0 0 0 vdev_cache_stats : 0% 0 0 0 0 Disks md0 ada0 da0 da1 cd0 pass0 pass1 pass2 pass3 KB/t 0.00 0.00 54.87 43.92 0.00 0.00 0.00 0.00 0.00 tps 0 0 319 465 0 0 0 0 0 MB/s 0.00 0.00 17.10 19.92 0.00 0.00 0.00 0.00 0.00 %busy 0 0 86 100 0 0 0 0 0
Hi, I'm trying to review and commit you patch. At least for FreeBSD-12 the patch doesn't apply cleanly. Do you have an updated version? Thanks! --- Mike |Index: vmstat.c |=================================================================== |--- vmstat.c (revision 306437) |+++ vmstat.c (working copy) -------------------------- Patching file vmstat.c using Plan A... Hunk #1 succeeded at 66 (offset 3 lines). Hunk #2 succeeded at 124 (offset 3 lines). Hunk #3 failed at 134. Hunk #4 succeeded at 200 (offset 3 lines). Hunk #5 succeeded at 300 (offset 3 lines). Hunk #6 succeeded at 353 (offset 3 lines). Hunk #7 succeeded at 456 with fuzz 2 (offset 3 lines). Hunk #8 succeeded at 507 (offset 3 lines). Hunk #9 succeeded at 628 with fuzz 1 (offset 3 lines). Hunk #10 succeeded at 674 (offset 19 lines). Hunk #11 succeeded at 690 (offset 3 lines). Hunk #12 succeeded at 804 (offset 19 lines). Hunk #13 succeeded at 815 (offset 3 lines). 1 out of 13 hunks failed--saving rejects to vmstat.c.rej Hmm... The next patch looks like a unified diff to me... The text leading up to this was: -------------------------- |Index: zarc.c |=================================================================== |--- zarc.c (revision 306437) |+++ zarc.c (working copy) -------------------------- Patching file zarc.c using Plan A... Hunk #1 succeeded at 1. Hunk #2 failed at 32. Hunk #3 succeeded at 79 (offset 1 line). Hunk #4 succeeded at 135 (offset 1 line). Hunk #5 failed at 143. Hunk #6 succeeded at 185 (offset 1 line). 2 out of 6 hunks failed--saving rejects to zarc.c.rej
(In reply to Michael Reifenberger from comment #1) Print format has been updated in the tree. I have a working version but changed more since I posted. I don't have change set handy and will upload sometime this week.
Created attachment 200735 [details] devstat on zarc and swap pages It looks I also made code refactor and improvements on devstat. I also created https://reviews.freebsd.org/D18726 for ease of review.
Sample test cases: Humm... It looks I had %VM usage and swap#s as experiment code. %VM has been always 99% and not useful... Total MFU MRU Anon Hdr L2Hdr Other ZFS ARC 214M 1737K 208M 1070K 3039K 0K 460K rate hits misses total hits total misses arcstats : 0% 0 0 252922 27590 arcstats.demand_data : 0% 0 0 26432 17952 arcstats.demand_metadata : 0% 0 0 225950 4232 arcstats.prefetch_data : 0% 0 0 468 5225 arcstats.prefetch_metadata: 0% 0 0 72 181 zfetchstats : 0% 0 0 3382 70442 arcstats.l2 : 0% 0 0 0 0 vdev_cache_stats : 0% 0 0 0 0 Disks ada0 da0 cd0 pass0 pass1 pass2 KB/t 0.00 0.00 0.00 0.00 0.00 0.00 tps 0 0 0 0 2 0 MB/s 0.00 0.00 0.00 0.00 0.00 0.00 %busy 0 0 0 0 1 0 /0 /1 /2 /3 /4 /5 /6 /7 /8 /9 /10her Load Average || swap_maxpages: 2795520 swap_total: -32768 Disk 1K-blocks Used /0% /10 /20 /30 /40 /50 /60 /70 /80 /90 /100 ada0s4b 2097016 0 X Disks ada0 da0 cd0 pass0 pass1 pass2 KB/t 0.00 0.00 0.00 0.00 0.00 0.00 tps 0 0 0 0 2 0 MB/s 0.00 0.00 0.00 0.00 0.00 0.00 %busy 0 0 0 0 0 0 5 users Load 0.47 0.62 0.69 Jan 3 00:09 Mem usage: 68%Phy 68%Kmem 99%VM Mem: KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 689984 89364 2024616 137164 880944 count All 698672 97576 2052368 160196 pages Proc: Interrupts r p d s w Csw Trp Sys Int Sof Flt 9 ioflt 343 total 2 155 790 130 2310 58 38 32 cow atkbd0 1 zfod acpi0 9 1.5%Sys 0.0%Intr 6.0%User 0.0%Nice 92.5%Idle ozfod psm0 12 | | | | | | | | | | %ozfod 14 hdac1 ath0 =>>> daefr ehci0 ehci 28 dtbuf prcfr 41 ohci0 ohci Namei Name-cache Dir-cache 52634 desvn 40 totfr 3 ahci0 19 Calls hits % hits % 32390 numvn react 176 cpu0:timer 7 7 100 20822 frevn pdwak 109 cpu1:timer 292 pdpgs hdac0 259 Disks ada0 da0 cd0 pass0 pass1 pass2 intrn vgapci0 KB/t 0.00 0.00 0.00 0.00 0.00 0.00 444808 wire tps 0 0 0 0 2 0 728696 act MB/s 0.00 0.00 0.00 0.00 0.00 0.00 708952 inact %busy 0 0 0 0 0 0 46236 laund Showing vmstat, refresh every 1 seconds. 880944
Created attachment 200757 [details] Trimmed experiments
A commit references this bug: Author: mr Date: Tue Apr 2 14:01:04 UTC 2019 New revision: 345804 URL: https://svnweb.freebsd.org/changeset/base/345804 Log: systat -zarc to display disk activities like -vm PR: 213310 Submitted by: ota MFH: 4 weeks Differential Revision: https://reviews.freebsd.org/D18726 Changes: head/usr.bin/systat/devs.c head/usr.bin/systat/devs.h head/usr.bin/systat/iostat.c head/usr.bin/systat/swap.c head/usr.bin/systat/systat.h head/usr.bin/systat/vmstat.c head/usr.bin/systat/zarc.c
Hi, Im really sorry for the delay but finally I got around to commit your changes to FreeBSD-HEAD. If all is going well I'll MFC in 4 weeks and close this PR afterwards... Thanks for your contribution! Greetings --- mike
A commit references this bug: Author: kevans Date: Mon Apr 27 16:12:00 UTC 2020 New revision: 360383 URL: https://svnweb.freebsd.org/changeset/base/360383 Log: MFC r340361, r345804: catch up on systat in head/ [Neither of these originally mine, but the latter commit referenced fixes an -fno-common issue and the former is a bugfix] r340361: Fix printing of 64-bit counters on 32-bit ppc platforms. Several statistic counters are uint64_t values and are printed by systat using %lu. This results in displaying wrong numbers. Use PRIu64 instead. While there, print variables of size_t using %zd. r345804: systat -zarc to display disk activities like -vm PR: 213310 Changes: _U stable/12/ stable/12/usr.bin/systat/devs.c stable/12/usr.bin/systat/devs.h stable/12/usr.bin/systat/iostat.c stable/12/usr.bin/systat/swap.c stable/12/usr.bin/systat/systat.h stable/12/usr.bin/systat/vmstat.c stable/12/usr.bin/systat/zarc.c
(In reply to commit-hook from comment #8) Thanks for MFCing to 12 branch. It will be very nice if you can take a look at another systat improvement at https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237664#c3
A commit in branch stable/11 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=72b42d209ff5b37c615048fb251d7bf9eb276381 commit 72b42d209ff5b37c615048fb251d7bf9eb276381 Author: Michael Tuexen <tuexen@FreeBSD.org> AuthorDate: 2018-11-15 17:25:32 +0000 Commit: Michael Reifenberger <mr@FreeBSD.org> CommitDate: 2021-01-03 16:29:50 +0000 MFC r340361: Fix printing of 64-bit counters on 32-bit ppc platforms. Several statistic counters are uint64_t values and are printed by systat using %lu. This results in displaying wrong numbers. Use PRIu64 instead. While there, print variables of size_t using %zd. Approved by: re (gjb@) Differential Revision: https://reviews.freebsd.org/D17838 (cherry picked from commit 45e9405ea4b05926b22a743237cbfe918206db50) systat: use and correctly display 64bit counters. Following struct vmtotal changes, make systat use and correctly display 64-bit counters. Switch to humanize_number(3) to overcome homegrown arithmetics limits in pretty printing large numbers. Use 1024 as a divisor for memory fields to make it consistent with other tools and users expectations. Submitted by: Pawel Biernacki <pawel.biernacki@gmail.com> Sponsored by: Mysterious Code Ltd. PR: 2137 Differential revision: https://reviews.freebsd.org/D13105 (cherry picked from commit 90dd3e79cc4d721b17c473fdaf3ba3d6d912b138) Order declarations alphabetically. Match signess of the format and the value. Noted by: bde Sponsored by: The FreeBSD Foundation (cherry picked from commit 1b7a3d1f5ffe700203f12654cd01a08a84551b4a) MFC r340361, r345804: catch up on systat in head/ [Neither of these originally mine, but the latter commit referenced fixes an -fno-common issue and the former is a bugfix] r340361: Fix printing of 64-bit counters on 32-bit ppc platforms. Several statistic counters are uint64_t values and are printed by systat using %lu. This results in displaying wrong numbers. Use PRIu64 instead. While there, print variables of size_t using %zd. r345804: systat -zarc to display disk activities like -vm PR: 213310 (cherry picked from commit a70af2507f720c2d7f56f51a18d358123b4770a5) MFC r360919,r360929,r361084: Patch systat -zarc to display cumulative rate and round down large numbers by SI units Introduce sysputpage() to display large page size with human readable format. Using UI units allows to fit larger numbers in columns. Stop calling v_page_size - this is a value that doesn't change at runtime. Renamed WINDOW *wnd to *wd to avoid conflict with global *wnd variable. Use bit-shift to convert page size to byte. PR: 237664, 246458 Submitted by: ota@j.email.ne.jp (cherry picked from commit 65d1fd13ac2182f7ca0c223bb723fba6d780740f) MFC r363095: Handle device removal and removal+add cases to fix infinity rate. PR: 219829 Submitted by: ota@j.email.ne.jp Reported by: rezo@live.cn Differential Revision: https://reviews.freebsd.org/D25226 (cherry picked from commit 7a446028cedb13a0d2004cf6f584291a976392da) PR: 219829, 237664, 246458, 213310, 2137 usr.bin/systat/Makefile | 4 +- usr.bin/systat/devs.c | 126 ++++++++++++++++++++++-- usr.bin/systat/devs.h | 16 ++- usr.bin/systat/extern.h | 7 +- usr.bin/systat/fetch.c | 9 +- usr.bin/systat/icmp6.c | 9 +- usr.bin/systat/ifcmds.c | 2 + usr.bin/systat/ifstat.c | 118 +++++++++++++--------- usr.bin/systat/iostat.c | 46 ++++----- usr.bin/systat/ip.c | 7 +- usr.bin/systat/ip6.c | 7 +- usr.bin/systat/main.c | 13 +-- usr.bin/systat/swap.c | 17 +++- usr.bin/systat/sysput.c (new) | 118 ++++++++++++++++++++++ usr.bin/systat/systat.1 | 1 + usr.bin/systat/systat.h | 4 + usr.bin/systat/tcp.c | 3 +- usr.bin/systat/vmstat.c | 221 +++++++++++++++--------------------------- usr.bin/systat/zarc.c | 124 +++++++++++++++--------- 19 files changed, 548 insertions(+), 304 deletions(-)