Created attachment 219550 [details]
Patch to make bhyve pci passthru work inside a jail
on FreeBSD 12.2-RELEASE is possible to run bhyve inside a jail, but testing
passthru revealed that this not work inside a jail.
The problem is that the jail needs r/w access to the following devices:
/dev/pci is not a problem with a securelevel < 0. But the rest needs a patch.
I have attached a rough patch to make bhyve pci passthrough work inside a jail,I also have attached jail.conf and devfs.rules used for testing.
This was tested in jail using vnet.
Created attachment 219551 [details]
devfs.rules use for testing
Created attachment 219552 [details]
jail.conf used for testing
PRIV_IO access is not required only by /dev/io, it is also required for sysarch(I386_SET_IOPERM), which is otherwise available to jailed processes. So the patch definitely should not be committed. A better solution would be to extend pci(4) so that bhyve can use it to do everything required for PCI passthrough. Even then I'm not sure why it's useful to jail the bhyve process - what does it buy you?
(In reply to Mark Johnston from comment #3)
> Even then I'm not sure why it's useful to jail the bhyve process - what does it buy you?
It allows folks to have a production version of bhyve in the host, and to develop and test the userland components of bhyve in a jail.
Perhaps I'm missing the point, but rather than punching a giant hole in the jail security model (ie: by giving unconstrained ring 0 / kernel privileges to jailed processes), would it not be better to run these development/testing bhyve userland components in a simple chroot environment?
(In reply to Peter Wemm from comment #5)
Currently illumos is able to do pci-passthrough with bhyve running inside a zone, that gives you an extra layer of security,if there is escape from the hypervisor then the attacker will land on a jail and not the host system.
Here are relevant links on how is used on illumos :
I think it would be nice to have this feature on FreeBSD jails, as Mark stated "better solution would be to extend pci(4) so that bhyve can use it to do everything required for PCI passthrough."
I would like to explore this option any pointer on how to start would be really good.