Bug 104092 - [patch] iostat(8): missing blanks in iostat output
Summary: [patch] iostat(8): missing blanks in iostat output
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: Unspecified
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-doc (Nobody)
URL:
Keywords: patch
Depends on:
Blocks:
 
Reported: 2006-10-07 10:10 UTC by Dr. Markus Waldeck
Modified: 2018-05-23 10:27 UTC (History)
0 users

See Also:


Attachments
file.diff (1.90 KB, patch)
2006-10-07 10:10 UTC, Dr. Markus Waldeck
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dr. Markus Waldeck 2006-10-07 10:10:24 UTC
If some values increases over a special size the ouput gets unusable.

How-To-Repeat: find /
find / | xargs cat > /dev/null
Comment 1 Alexander Best 2010-04-28 13:40:02 UTC
i think this pr got partially fixed by r196254 (HEAD) which got mfc'ed to
stable-8 (r196255). high 'tin' and 'tout' values no longer cause a distorted
output. the -I flag however still suffers from this problem.

-- 
Alexander Best
Comment 2 Alexander Best freebsd_committer 2010-10-05 16:28:37 UTC
State Changed
From-To: open->patched

Patched in HEAD and stable/8. 


Comment 3 Alexander Best freebsd_committer 2010-10-05 16:28:37 UTC
Responsible Changed
From-To: freebsd-bugs->keramida

Assign to Comitter as MFC reminder.
Comment 4 Giorgos Keramidas freebsd_committer 2011-02-15 06:47:45 UTC
On 2006-10-07 09:06, "Dr. Markus Waldeck" <waldeck@gmx.de> wrote:
> i think this pr got partially fixed by r196254 (HEAD) which got mfc'ed to
> stable-8 (r196255). high 'tin' and 'tout' values no longer cause a distorted
> output. the -I flag however still suffers from this problem.

It's relatively 'easy' to make the tty columns overflow even now in -I
mode, e.g. by running "ls -lR /" in one terminal and watching "iostat
-I" output in another.

Here's a sample output screenshot-like log.  Note the clipping that will
almost always appear in 80-column terminals for the first sample, and
the space between tout and next column in all lines.

.--------------------------------------------------------------------------------.
|         1         2         3         4         5         6         7         8|
|12345678901234567890123456789012345678901234567890123456789012345678901234567890|
|--------------------------------------------------------------------------------|
|       tty             ad0              ad1              ad2             cpu	 |
| tin  tout  KB/t xfrs   MB   KB/t xfrs   MB   KB/t xfrs   MB  us ni sy in id    |
|   1   201  9.02 140947 1241.51   3.23  22  0.07  13.51 20178 266.28   2  0  5  2 91
|   0 109258  2.75  16  0.04   0.00   0  0.00   0.00   0  0.00   0  0 92  0  8   |
|   0 41223  2.00 119  0.23   0.00   0  0.00   0.00   0  0.00   0  0 57  0 43	 |
|   0 16353  2.00 132  0.26   0.00   0  0.00   0.00   0  0.00   0  0 39  2 59	 |
|   0 37210  2.00 130  0.25   0.00   0  0.00   0.00   0  0.00   0  0 63  1 36	 |
|   0 70689  2.00  35  0.07   0.00   0  0.00   0.00   0  0.00   0  0 77  2 22	 |
|   0 96243  2.56  18  0.04   0.00   0  0.00   0.00   0  0.00   1  0 89  0 10	 |
|   0 55505  2.05  44  0.09   0.00   0  0.00   0.00   0  0.00   1  0 68  1 30	 |
|   0 35459  2.00  57  0.11   0.00   0  0.00   0.00   0  0.00   0  0 52  1 47	 |
|   0 47856  2.00  47  0.09   0.00   0  0.00   0.00   0  0.00   1  0 55  1 43	 |
|   0 43358  2.00  56  0.11   0.00   0  0.00   0.00   0  0.00   0  0 54  2 44	 |
|   0 48254  2.00  53  0.10   0.00   0  0.00   0.00   0  0.00   0  0 60  0 40	 |
|   0 42519  2.00  74  0.14   0.00   0  0.00   0.00   0  0.00   0  0 54  2 45	 |
|   0 50765  2.04  55  0.11   0.00   0  0.00   0.00   0  0.00   0  0 61  1 38	 |
|   0 37901  2.00  46  0.09   0.00   0  0.00   0.00   0  0.00   0  0 55  2 43	 |
|   0 32938  2.00  50  0.10   0.00   0  0.00   0.00   0  0.00   0  0 48  2 50	 |
|   0 34099  2.00  56  0.11   0.00   0  0.00   0.00   0  0.00   0  0 48  0 52    |
`--------------------------------------------------------------------------------'

There should be a space between tout and the next column for all
runsnow, even when tout overflows.  We could, arguably bump the tout
column size to 6 characters though, unless that's really really bad.

Bruce, I remember you had mentioned not messing with the iostat widths
before.  Do you think it's ok to bump tout to 6 columns?
Comment 5 Bruce Evans freebsd_committer 2011-02-20 18:46:05 UTC
On Tue, 15 Feb 2011, Giorgos Keramidas wrote:

> On 2006-10-07 09:06, "Dr. Markus Waldeck" <waldeck@gmx.de> wrote:
>> i think this pr got partially fixed by r196254 (HEAD) which got mfc'ed to
>> stable-8 (r196255). high 'tin' and 'tout' values no longer cause a distorted
>> output. the -I flag however still suffers from this problem.
>
> It's relatively 'easy' to make the tty columns overflow even now in -I
> mode, e.g. by running "ls -lR /" in one terminal and watching "iostat
> -I" output in another.

In general, I think you have to subtract 1 from the field width when
forcing a space.  If more space is available, then it should have been
used already.  Allowing the fields to run together is a hack to allow
expanding the field width by 1 at no cost when the extra width is not
needed.  When the fields run together often, it is just bad.

> Here's a sample output screenshot-like log.  Note the clipping that will
> almost always appear in 80-column terminals for the first sample, and
> the space between tout and next column in all lines.
>
> .--------------------------------------------------------------------------------.
> |         1         2         3         4         5         6         7         8|
> |12345678901234567890123456789012345678901234567890123456789012345678901234567890|
> |--------------------------------------------------------------------------------|
> |       tty             ad0              ad1              ad2             cpu	 |
> | tin  tout  KB/t xfrs   MB   KB/t xfrs   MB   KB/t xfrs   MB  us ni sy in id    |
> |   1   201  9.02 140947 1241.51   3.23  22  0.07  13.51 20178 266.28   2  0  5  2 91
> |   0 109258  2.75  16  0.04   0.00   0  0.00   0.00   0  0.00   0  0 92  0  8   |
> |   0 41223  2.00 119  0.23   0.00   0  0.00   0.00   0  0.00   0  0 57  0 43	 |
> |   0 16353  2.00 132  0.26   0.00   0  0.00   0.00   0  0.00   0  0 39  2 59	 |
>
> There should be a space between tout and the next column for all
> runsnow, even when tout overflows.  We could, arguably bump the tout
> column size to 6 characters though, unless that's really really bad.
>
> Bruce, I remember you had mentioned not messing with the iostat widths
> before.  Do you think it's ok to bump tout to 6 columns?

Not in general :-).  6 is wasteful for many common cases.  OTOH, 8 or 9
is needed for both tin and tout in some cases -- I can get 3.8M for tout
for catting files to the screen on 1 1-CPU system, but that is real slow
- even disks are faster; ptys should have lower overhead and should approach
the /dev/zero throughput of 4.4G/S (but I think they don't get close).
Expand the number of CPUs and we should be about to get near 1TB/S for tout.
Similarly for tin through ptys.  Hardware ttys are real-real slow so I can
get only a couple of hundred thousand/S for them.

The tty columns are the least of the problems.  On ref9-i386 now I get:

%        tty           mfid0             cpu
%  tin  tout  KB/t tps  MB/s  us ni sy in id
%    0    51  9.13  14  0.13  -12 -13 -8 -0 133

Lots of counters seem to have overflowed.  The system has been up for 89
days, which is enough to cause problems.

%    0   135 49.50   3  0.14   0 38  1  0 61
%    0    44 112.00   2  0.22   0 36  1  0 63
%    0    45 128.00   2  0.25   0 38  0  0 62
%    0    45 68.50   4  0.27   0 39  0  0 61
%    0    44 128.00   1  0.12   0 35  0  0 64
%    0    45 83.20   5  0.41   0 38  0  0 62
%    0    44 61.00   4  0.24   0 39  0  0 61
%    0    44 32.21  19  0.60   0 38  0  0 62
%    0    44 128.00   3  0.37   0 37  0  0 63
%    0    45 46.55  11  0.50   0 36  0  0 64
%    0    44 122.67   3  0.36   0 37  0  0 63
%    0    45 128.00   3  0.37   0 37  0  0 62
%    0    45 128.00   3  0.37   0 38  0  0 62

FreeBSD cluster machine mostly have a whole 1 disk, so they don't need
much horizontal space, but the formatting is still messed up:
- the disk KB/t column is only wide enough up to 99.99 KB.  This
   is broken if either:
   - the i/o size is misreported here as the GEOM-level i/o size instead
     of the device-level i/o size.  The former is often MAXPHYS (was 128K,
     now possibly larger).  But I think tegge@ fixed this.
   - the device level i/o size exceeds 99.99 KB.  Most devices had a bogus
     limit of DFLTPHYS = 64K which fitted, but more now supportup to MAXPHYS.
- the disk MB/S column runs out at 99.99 MB/S.  This used to be adequate
   except for some RAID cases.
- the disk tps column runs out at 999.  tps is inversely proportional to
   disk latency so it has moe physical constraints than the other fields,
   but it has always been easy to exceed 999 tps on cached data only.

My local machine has many more disks than will fit, and iostat output on it
looks more like yours:

%       tty             ad0              ad1              ad2             cpu
%  tin tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
%    2-170984 16.17   1  0.01   2.75   0  0.00   7.75   0  0.00   5  0  2  0 93
%    0  232 19.00   4  0.07   0.00   0  0.00   0.00   0  0.00   0  0  0  0 100
%    0   77  0.00   0  0.00   0.00   0  0.00   0.00   0  0.00   0  0  0  0 100

The initial tout (total) has overflowed so its average is garbage and the
formatting of the garbage is no better, but everything else seems correct.

Bruce
Comment 6 Alexander Best freebsd_committer 2011-03-24 19:22:32 UTC
State Changed
From-To: patched->analyzed

I'm setting this PR into analyzed state. Although some issues mentioned in this 
PR were fixed (r196254, r196255), the tin/tout issues in connection with the -I 
flag still exist. 
Giorgos and Bruce have described the actual problem in detail (i.e. analyzed 
it). Hence the state change. Looking forward to a fix for this issue. 

Please note that a lot of the classic *stat utilities didn't catch up with the 
improved hardware. A lot of their statistics are being processed in 
inappropriate integer types, bin/30360 being one example.
Comment 7 Mark Linimon freebsd_committer freebsd_triage 2015-03-10 03:05:48 UTC
Release to wild.
Comment 8 Eitan Adler freebsd_committer freebsd_triage 2018-05-23 10:27:40 UTC
batch change of PRs untouched in 2018 marked "in progress" back to open.