Bug 204901

Summary: GEOM doesn't see new disk capacity after VMDK resize without a reboot
Product: Base System Reporter: mgrooms
Component: kernAssignee: Edward Tomasz Napierala <trasz>
Status: Closed FIXED    
Severity: Affects Some People CC: ebay, feld, ken, lapo, pi, rainer, trasz
Priority: ---    
Version: CURRENT   
Hardware: amd64   
OS: Any   
Attachments:
Description Flags
patch that adds camcontrol reprobe device command none

Description mgrooms 2015-11-29 20:50:47 UTC
Created attachment 163665 [details]
patch that adds camcontrol reprobe device command

ESXi allows the user to resize a VMDK attached to a running VM. Unfortunately it doesn't return any 'Unit Attention' sense information in response to a 'Read Capacity' request. This kinda makes sense as ESXi emulates SCSI-2 disk devices and, as far as I can tell, the 0x2A/0x09 ASC/ASCQ sense code that denotes 'Capacity Data Has Changed' wasn't defined until the SCSCI-3 spec.

I am attaching a patch that allows an admin to request that a device be re-probed via camcontrol. The new command sends a CCB message that creates an async event. The event is captured by scsi_da which calls reprobe() on the peripheral, the same code that is executed when the 'Unit Attention' 'Capacity Data Has Changed' sense code is received from the device. For example ...

# camcontrol readcap da1 -h
Device Size: 32 G, Block Length: 512 bytes

# gpart show da1
=>      40  58720176  da1  GPT  (28G)
        40  58720176    1  freebsd-ufs  (28G)

Note, I resized the VMDK disk in ESXi. The camcontrol readcap output shows the size as 32G but geom still thinks its 28G.

# camcontrol devlist
<NECVMWar VMware IDE CDR10 1.00>   at scbus1 target 0 lun 0 (cd0,pass0)
<VMware Virtual disk 1.0>          at scbus2 target 0 lun 0 (pass1,da0)
<VMware Virtual disk 1.0>          at scbus2 target 1 lun 0 (pass2,da1)
<FREEBSD CTLDISK 0001>             at scbus3 target 0 lun 0 (da2,pass3)

# camcontrol reprobe 2:1:0

The kernel output looks almost identical to when the 'Unit Attention' sense data is received ...

Nov 28 17:46:13 iscsi-i kernel: (da1:mpt0:0:1:0): Re-probe requested
Nov 28 17:46:13 iscsi-i kernel: GEOM_PART: da1 was automatically resized.
Nov 28 17:46:13 iscsi-i kernel: Use `gpart commit da1` to save changes or `gpart undo da1` to revert them.

Now that geom knows about the increased disk capacity, I can resize the partition and grow the fs ...

[root@iscsi-i /home/mgrooms]# gpart show da1
=>      40  67108784  da1  GPT  (32G)
        40  58720176    1  freebsd-ufs  (28G)
  58720216   8388608       - free -  (4.0G)

# gpart resize -i 1 da1
da1p1 resized

# growfs da1p1
Device is mounted read-write; resizing will result in temporary write suspension for /var/data1.
It's strongly recommended to make a backup before growing the file system.
OK to grow filesystem on /dev/da1p1, mounted on /var/data1, from 28GB to 32GB? [Yes/No] Yes
super-block backups (for fsck_ffs -b #) at:
 58983232, 60265472, 61547712, 62829952, 64112192, 65394432, 66676672

# df -h
Filesystem    Size    Used   Avail Capacity  Mounted on
/dev/da0p3     18G    5.3G     12G    31%    /
devfs         1.0K    1.0K      0B   100%    /dev
/dev/da1p1     31G     32M     28G     0%    /var/data1
/dev/da2p1     15G     32M     14G     0%    /var/data2
Comment 2 Edward Tomasz Napierala freebsd_committer freebsd_triage 2016-05-11 08:00:52 UTC
And once again, big thanks for analysing the problem and writing the patch.
Comment 3 Mark Felder freebsd_committer freebsd_triage 2017-04-24 20:33:40 UTC
I came across this today and really wished it could have been MFH to 10-STABLE to make it into a 10.x RELEASE
Comment 4 Kurt Jaeger freebsd_committer freebsd_triage 2017-04-25 02:50:32 UTC
See r299371 for the commit to HEAD