Bug 243946

Summary: mdconfig -d causes iostat glitch
Product: Base System Reporter: John F. Carr <jfc>
Component: binAssignee: freebsd-bugs (Nobody) <bugs>
Status: New ---    
Severity: Affects Only Me CC: cem
Priority: ---    
Version: CURRENT   
Hardware: amd64   
OS: Any   

Description John F. Carr 2020-02-06 20:59:09 UTC
iostat glitched when I used mdconfig -d:

       tty            nvd0              da0              da1             cpu
 tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
   0     0 116.07  99 11.22  95.51 379 35.32  95.36 381 35.47   0  0  5  0 95
   2    25 114.79  91 10.23  110.73 395 42.69  111.10 390 42.28   0  0  1  0 99
   1     6 117.03 125 14.27   0.00 1841192139503780665 3460.45   0.00 1841192139503796095 14.38   0  0  1  0 99
   2     7 119.17  96 11.21   0.00   0  0.00   0.00   0  0.00   0  0  0  0 100

I was copying files off an ISO image using mdconfig to create a special device to mount as a cd9660 filesystem.  The extremely high tps in the above iostat output happened during the interval when the device was destroyed.

This particular iostat was invoked as "iostat nvd0 da0 da1 10".  da0 and da1 are mirrored drives in a ZFS pool so the true I/O counts should be very similar.

Based on another iostat invocation without an explicit device list, md0 comes before da0 in whatever order iostat uses.  My guess is destroying md0 changed indices into the device statistics table.  Iostat tried to subtract "this tick's count for index 1 = da0" from "last tick's count for index 1 = md0" or something along those lines.
Comment 1 Conrad Meyer freebsd_committer 2020-02-06 23:34:51 UTC
Yep.  The iostat interface is sort of inherecently racy, but we might be able to do a better job in this specific case.

(Even the stats collection in the kernel is racier than it has to be.  If you have a fast enough device, some starts and stop counts can be lost and you end up with similar overflow numbers.)