Bug 247188 - "camcontrol devlist" does not show zvol-backed virtio devices
Summary: "camcontrol devlist" does not show zvol-backed virtio devices
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: bhyve (show other bugs)
Version: 12.1-RELEASE
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-virtualization (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-06-12 00:34 UTC by Sebastien Boisvert
Modified: 2020-06-13 18:26 UTC (History)
1 user (show)

See Also:


Attachments
Output of "vm info nova" on the host that's running the BHYVE hypervisor (4.17 KB, text/plain)
2020-06-12 00:34 UTC, Sebastien Boisvert
no flags Details
Output of dmesg in the nova guest (7.72 KB, text/plain)
2020-06-12 00:34 UTC, Sebastien Boisvert
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sebastien Boisvert 2020-06-12 00:34:14 UTC
Created attachment 215473 [details]
Output of "vm info nova" on the host that's running the BHYVE hypervisor

Hi,

I think that I found a bug. I would like to fix it myself.

My host "atlantis" is running FreeBSD 12.1-RELEASE.

I have a BHYVE guest "nova" with:
- 1 20-GB file-backed disk for the operating system (FreeBSD 12.1-RELEASE),
- 8 10-GB ZVOL-backed virtio disks, and
- 8 20-GB ZVOL-backed virtio disks.

see vm-info-nova.txt

In nova, the disks are listed in /dev:

root@nova:~ # ls -1 /dev/vtbd?
/dev/vtbd0
/dev/vtbd1
/dev/vtbd2
/dev/vtbd3
/dev/vtbd4
/dev/vtbd5
/dev/vtbd6
/dev/vtbd7
/dev/vtbd8
/dev/vtbd9


However, camcontrol returns nothing.

root@nova:~ # camcontrol devlist




gpart also does not see the disks.

root@nova:~ # gpart show /dev/vtbd1
gpart: No such geom: /dev/vtbd1.



root@nova:~ # camcontrol devlist|wc -l
       0




I think it is either a problem with CAM or with GEOM, or maybe the VIRTIO driver in the guest.

From the host point-of-view, everything seems fine.
Comment 1 Sebastien Boisvert 2020-06-12 00:34:49 UTC
Created attachment 215474 [details]
Output of dmesg in the nova guest
Comment 2 Allan Jude freebsd_committer 2020-06-12 21:30:27 UTC
camcontrol devlist only shows devices that are connected via the CAM subsystem generally, SCSI/SAS, SATA, and USB (pretends to be SCSI).

NVMe devices may not even appear under camcontrol unless you have hw.nvme.use_nvd=0 (there is a separate nvmecontrol devlist command)

NVDIMM devices are also not CAM devices.

If you want a list of disk devices, across all types, you like try:

sysctl kern.disks
Comment 3 Sebastien Boisvert 2020-06-12 22:34:30 UTC
A VirtIO device is not a NVMe device, is not a NVDIMM device.

I don't if a VirtIO device is a CAM device though.

Is it ?

root@nova:~ # dmesg | grep vtb
vtblk0: <VirtIO Block Adapter> on virtio_pci0
vtblk0: 20480MB (41943040 512 byte sectors)
Trying to mount root from ufs:/dev/vtbd0p2 [rw]...

root@nova:~ # sysctl kern.disks
kern.disks: vtbd0
Comment 4 Andriy Gapon freebsd_committer 2020-06-13 11:24:47 UTC
(In reply to Sebastien Boisvert from comment #3)
It's not.

There is virtio_scsi, though.
Comment 5 Sebastien Boisvert 2020-06-13 18:26:36 UTC
I just tested with virtio-scsi and it the device also does not show up:

[CODE]
root@atlantis:~ # grep disk1_ /tank/vm-bhyve/nova/nova.conf
disk1_name="disk1"
disk1_type="virtio-scsi"
disk1_dev="zvol"
[/CODE]


[CODE]
root@nova:~ # camcontrol devlist|wc -l
       0
[/CODE]


Allan Jude said that a SCSI device will be managed by the CAM subsystem.

So it seems that from the point of view of CAM, virtio-scsi is not a SCSI device.


I confirm that a "ahci-hd" (SATA) disk is found inside the virtual machine by CAM:

[CODE]
root@nova:~ # camcontrol devlist
<BHYVE SATA DISK 001>              at scbus0 target 0 lun 0 (ada0,pass0)
[/CODE].