Hi. If you install Debian 12 on bhyve from an iso image in UEFI mode, you will notice that the grub boot loader will not be found. To fix this you need to do the following: mkdir /boot/efi/EFI/BOOT/ # copy file - workaround for bhyve grub package # # Pay attention to destination file bootx64.efi # cp /boot/efi/EFI/debian/grubx64.efi /boot/efi/EFI/BOOT/bootx64.efi Taken here: https://www.cyberciti.biz/faq/how-to-install-linux-vm-on-freebsd-using-bhyve-and-zfs/ I suggest fixing bhyve so that it tries to load not only /boot/efi/EFI/BOOT/bootx64.efi, but also /boot/efi/EFI/debian/grubx64.efi.
Please try with FreeBSD 14.0 or later - have a look at the details in https://github.com/churchers/vm-bhyve/issues/336
(In reply to Ed Maste from comment #1) I'm using FreeBSD 14 and I have that problem. And nope,I don't use vm-bhyve and I don't like it. I prefer to use the raw bhyve parameters. That bug is well known and it exists since years. I never understood if it belongs to Debian or to FreeBSD...in any case I would like to see it fixed,but I don't want to be forced to use vm-bhyve to use the fix.
(In reply to mario felicioni from comment #2) You don't have to use vm-bhyve, I just linked to that thread as it has some relevant information. Are you passing a BHYVE_UEFI_VARS.fd file via bhyve's -l argument?
Usually I use this template : bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \ -s 0,hostbridge \ -s 1,ahci-hd,/bhyve/debian12-test1/Debian.img,bootindex=1 \ -s 13,virtio-net,tap4 \ -s 14,virtio-9p,sharename=/ \ -s 29,fbuf,tcp=0.0.0.0:5904,w=1600,h=950,wait \ -s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \ vm0:4 < /dev/null & sleep 2 && vncviewer 0:4
(In reply to mario felicioni from comment #4) Yes you need to pass a BHYVE_UEFI_VARS.fd file for the nvram variables to persist.
(In reply to Ed Maste from comment #1) Thx.
How can I pass a BHYVE_UEFI_VARS.fd file ? Is this correct ? bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \ -s 0,hostbridge \ -s 1,ahci-hd,/bhyve/debian12-test1/Debian.img,bootindex=1 \ -s 13,virtio-net,tap4 \ -s 14,virtio-9p,sharename=/ \ -s 29,fbuf,tcp=0.0.0.0:5904,w=1600,h=950,wait \ -s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \ -l bootrom,/usr/local/share/edk2-bhyve/BHYVE_UEFI_VARS.fd \ vm0:4 < /dev/null & sleep 2 && vncviewer 0:4
oh sorry,maybe the correct file is on /usr/local/share/uefi-firmware. So : bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \ -s 0,hostbridge \ -s 1,ahci-hd,/bhyve/debian12-test1/Debian.img,bootindex=1 \ -s 13,virtio-net,tap4 \ -s 14,virtio-9p,sharename=/ \ -s 29,fbuf,tcp=0.0.0.0:5904,w=1600,h=950,wait \ -s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_VARS.fd \ vm0:4 < /dev/null & sleep 2 && vncviewer 0:4 I doubt that it is correct...
(In reply to mario felicioni from comment #8) I found this option, but haven't tested it yet -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd,BHYVE_UEFI_VARS.fd
Somewhere I read that you can chain BHYVE_UEFI_CODE.fd and BHYVE_UEFI_VARS.fd into one only file.
Please read here : https://www.digitalfoundations.com.au/blog/macos-on-windows-with-qemu/ at a certain point,he explains how to combine OVMF_VARS.fd + OVMF_CODE.fd : copy /B OVMF_VARS.fd + OVMF_CODE.fd OVMF_combined.fd so,what about to do : copy /B BHYVE_UEFI_VARS.fd + BHYVE_UEFI_CODE.fd BHYVE_combined.fd (in Windows) and then : bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \ -s 0,hostbridge \ -s 1,ahci-hd,/bhyve/debian12-test1/Debian.img,bootindex=1 \ -s 13,virtio-net,tap4 \ -s 14,virtio-9p,sharename=/ \ -s 29,fbuf,tcp=0.0.0.0:5904,w=1600,h=950,wait \ -s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_combined.fd \ vm0:4 < /dev/null & sleep 2 && vncviewer 0:4 please try and let me know if it works.
(In reply to mario felicioni from comment #11) After connecting via VNC it crashes. # bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \ -s 0,hostbridge \ -s 1,ahci-hd,/vm/vm/debian12-test2/disk0.img,bootindex=1 \ -s 29,fbuf,tcp=0.0.0.0:5900,w=1600,h=950,wait \ -s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/vm/vm/debian12-test2/BHYVE_combined.fd \ guest fbuf frame buffer base: 0x29db7a200000 [sz 16777216] vm exit[0] reason VMX rip 0x000000000000039d inst_length 2 status 0 exit_reason 2 (Triple fault) qualification 0x0000000000000000 inst_type 0 inst_error 0 Abort trap # uname -a FreeBSD home 13.2-RELEASE-p11 FreeBSD 13.2-RELEASE-p11 releng/13.2-n254665-f5ac4e174fdd MYGENERIC amd64
cat /usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd > BHYVE_combined.fd cat /usr/local/share/uefi-firmware/BHYVE_UEFI_VARS.fd >> BHYVE_combined.fd
Based on the original report I assumed copying grubx64.efi to bootx64.efi gave you a working system - is that assumption wrong (or, did you not try connecting with VNC previously)?
That's how it can be fixed. But the problem is not fixed in that way if you don't develop something that looks like a patch.
(In reply to mario felicioni from comment #15) No, it appears there are two problems being reported here and I want to make sure I understand correctly. For the "UEFI variables don't persist" issue copying grubx64.efi to bootx64.efi is a workaround. That issue is fixed by passing appropriate parameters to the bhyve invocation. There's no patch for this issue, because there's no bug to fix.
I disagree. Debian is the only distro that does not boot automatically as soon as it is installed. You can grab any other linux distro and you will see that after the installation it can boot without any manual intervention.
(In reply to mario felicioni from comment #17) It seems Debian requires persistent variables, while other distros install a fallback efi/boot/bootx64.efi. The fix for this is to provide correct commandline parameters to persist UEFI environment variables.
(In reply to Ed Maste from comment #14) > did you not try connecting with VNC previously Yes you are right. I fixed this problem. But to reproduce the problem I need Debian12 from /boot/efi/EFI/debian/grubx64.efi not to boot. Can you please tell me how to log into UEFI to change the settings? You wrote that you can also pass parameters, but how to pass them?
(In reply to Ed Maste from comment #18) Sure,but you haven't explained how to pass both the parameters. Please give a look at comment #7 and #11. At the moment both of them don't work.
(In reply to mario felicioni from comment #20) I myself don’t know how to do this correctly. But if you look at the official man bhyve - https://man.freebsd.org/cgi/man.cgi?query=bhyve&sektion=8&format=html then the correct option is: -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd,BHYVE_UEFI_VARS.fd
(In reply to Vurefozu from comment #21) Of course you know that the 2nd file will give a not found error, no?
(In reply to Vurefozu from comment #21) I've just tried. I've used these parameters : bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \ -s 0,hostbridge \ -s 1,ahci-hd,/mnt/zroot-133/bhyve/img/Linux/Debian.img,bootindex=1 \ -s 13,virtio-net,tap1 \ -s 14,virtio-9p,sharename=/ \ -s 29,fbuf,tcp=0.0.0.0:5901,w=1600,h=950,wait \ -s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd,BHYVE_UEFI_VARS.fd \ vm0:1 < /dev/null & sleep 2 && vncviewer 0:1 I see only this message : fbuf frame buffer base: 0x2ed80000000 [sz 16777216] so,the vm tried to boot,but then it exits.
(In reply to odhiambo@gmail.com from comment #22) In man bhyve they write this: > Run a UEFI virtual machine with a VARS file to save EFI variables. > Note that bhyve will write guest modifications to the given VARS file. > Be sure to create a per-guest copy of the template VARS file from /usr. You must first copy the file /usr/local/share/uefi-firmware/BHYVE_UEFI_VARS.fd to the directory with the guest virtual machine. But I can’t figure out how to launch the UEFI menu to select the desired file and save the setting.
---> You must first copy the file /usr/local/share/uefi-firmware/BHYVE_UEFI_VARS.fd to the directory with the guest virtual machine. My BHYVE_UEFI_VARS.fd file is on /usr/local/share/uefi-firmware ; is this wrong ?
In man bhyve they write this: > Run a UEFI virtual machine with a VARS file to save EFI variables. > Note that bhyve will write guest modifications to the given VARS file. > Be sure to create a per-guest copy of the template VARS file from /usr. For me it's not clear what should I do.
(In reply to mario felicioni from comment #25) Apparently it's not correct. From man bhyve (https://man.freebsd.org/cgi/man.cgi?query=bhyve&sektion=8&format=html): > Be sure to create a per-guest copy of the template VARS file from /usr.
(In reply to mario felicioni from comment #26) > For me it's not clear what should I do. cp /usr/local/share/uefi-firmware/BHYVE_UEFI_VARS.fd .
https://ibb.co/c8cx9wZ I'm trying to boot a Debian img file that I've created with qemu and that it boots if I use qemu. I imagine that I should create a new Debian vm from scratch if I want to boot it with bhyve,right ?
(In reply to mario felicioni from comment #29) To continue the discussion, I created a topic on the forum - https://forums.freebsd.org/threads/debian-12-wont-boot-in-uefi-mode.93529/ I suggest continuing the discussion there.
You have to copy the file /usr/local/share/uefi-firmware/BHYVE_UEFI_VARS.fd to any location you want. Then you have to pass that copy to bhyve with: -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd,BHYVE_UEFI_VARS.fd Replace BHYVE_UEFI_VARS.fd with the full (or correct relative) path to your copy of the VARS file. When installing Debian, more precisely grub, it creates an EFI boot entry pointing to /EFI/debian/grubx64.efi. Boot entries are saved as EFI VARS. If you haven't passed a VARS file to bhyve when installing Debian, this information is lost after reboot. For already installed VM's you either have to reinstall grub or create a new boot entry on your own: efibootmgr --create --disk <boot-disk> --part <boot-partition> --loader /EFI/debian/grubx64.efi --label <pick-a-name> Note that your boot partition is commonly mounted at /boot/efi on Debian. Therefore, the path to the loader is just /EFI/debian/grubx64.efi. Btw. Qemu is able to use a seperate EFI CODE and EFI VARS files too. So, if you want to create your VM image with Qemu, use seperate files for CODE and VARS. Then reuse the VARS file when booting bhyve and everything should work as expected. I'm closing this bug report as it's already fixed.