Bug 251250 - bhyveload cannot find filesystem: ERROR: cannot open /boot/lua/loader.lua: no such file or directory.
Summary: bhyveload cannot find filesystem: ERROR: cannot open /boot/lua/loader.lua: no...
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: bhyve (show other bugs)
Version: 12.2-RELEASE
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-virtualization (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-11-19 07:55 UTC by Martin Birgmeier
Modified: 2020-11-25 19:53 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Birgmeier 2020-11-19 07:55:49 UTC
Scenario:
- Host system was running FreeBSD 12.1
- Using /usr/share/examples/bhyve/vmrun.sh to start a VM running FreeBSD 12.2, using a disk device attached via iSCSI and a network device via bridge/tap
- This was running without issues.
- Now, the host system has been upgraded to FreeBSD 12.2

Result:
- After entering "sh /usr/share/examples/bhyve/vmrun.sh -u -c 4 -m 3G -t tap909 -d /dev/diskid/DISK-HAL_V0909 v909", the following appears on the console:

Launching virtual machine "v909" ...
Consoles: userboot  

FreeBSD/amd64 User boot lua, Revision 1.2
ERROR: cannot open /boot/lua/loader.lua: no such file or directory.


Type '?' for a list of commands, 'help' for more detailed help.
OK 

- Entering "show" yields:

OK show
LINES=24
autoboot_delay=NO
boot_serial=1
console=userboot
currdev=disk0:
interpret=OK
loaddev=disk0:
prompt=${interpret}
script.lang=lua
smbios.bios.vendor=BHYVE
twiddle_divisor=1
OK 

- Entering "lsdev" yields:

OK lsdev
host devices:
    host0:   Host filesystem
disk devices:
    disk0:   Guest drive image
      disk0a: FreeBSD UFS
      disk0b: FreeBSD swap
      disk0d: FreeBSD UFS
OK 

- The problem seems to be that currdev and loaddev are wrong, they should point to disk0a: instead
- The filesystem is not found:

OK ls
open '/' failed: no such file or directory
OK 

- It is possible to set currdev: manually to disk0a: (loaddev cannot be changed), then load all necessary modules manually, and then issue the command "boot" in order to start the system. However, when doing it this way no console output is shown. Otherwise, the client seems to work o.k.

OK set currdev=disk0a:
OK ls
/
 d  .snap
 d  lost+found
 d  usr
 d  CVS
 d  boot
 d  root
 d  etc
 d  bin
 d  dev
 d  lib
 d  libexec
 d  media
 d  mnt
 d  proc
 d  rescue
 d  sbin
 d  tmp
 d  var
 l  sys
 d  vol
    .cshrc
    .profile
    COPYRIGHT
    entropy
 d  d
 d  a
 d  net
    patch.2020-11-01.19:15:22
    releng_12.1.r354774.tar.gz
OK 

-- Martin
Comment 1 Martin Birgmeier 2020-11-19 08:06:09 UTC
- Probably one more item of interest: The iSCSI-attached disk does not have an MBR, only a disklabel:

[0]# iscsictl -L
Target name                          Target portal    State
iqn.1995-06.xyzzy.hal:disk909        hal.xyzzy        Connected: da0 
[0]# gpart show da0
=>       0  83886080  da0  BSD  (40G)
         0       256       - free -  (128K)
       256   4194304    1  freebsd-ufs  (2.0G)
   4194560   4194304    2  freebsd-swap  (2.0G)
   8388864  75497216    4  freebsd-ufs  (36G)

[0]# bsdlabel da0
# /dev/da0:
8 partitions:
#          size     offset    fstype   [fsize bsize bps/cpg]
  a:    4194304        256    4.2BSD        0     0     0
  b:    4194304    4194560      swap                    
  c:   83886080          0    unused        0     0     # "raw" part, don't edit
  d:   75497216    8388864    4.2BSD        0     0     0
[0]# 

-- Martin
Comment 2 Martin Birgmeier 2020-11-25 19:51:07 UTC
The boot succeeds if a slice is created for holding the partitions; this is done by splitting the initial 256 sectors as follows:

[0]# gpart show
=>       1  83886079  vtbd0  MBR  (40G)
         1        15         - free -  (7.5K)
        16  83886064      4  freebsd  [active]  (40G)

=>       0  83886064  vtbd0s4  BSD  (40G)
         0       240           - free -  (120K)
       240   4194304        1  freebsd-ufs  (2.0G)
   4194544   4194304        2  freebsd-swap  (2.0G)
   8388848  75497216        4  freebsd-zfs  (36G)

[0]# bsdlabel vtbd0s4
# /dev/vtbd0s4:
8 partitions:
#          size     offset    fstype   [fsize bsize bps/cpg]
  a:    4194304        240    4.2BSD        0     0     0
  b:    4194304    4194544      swap                    
  c:   83886064          0    unused        0     0     # "raw" part, don't edit
  d:   75497216    8388848       ZFS                    
[0]# ll /dev/vtbd0*
crw-r-----  1 root  operator  0x49 Nov 25 17:56 /dev/vtbd0
crw-r-----  1 root  operator  0x4a Nov 25 17:56 /dev/vtbd0s4
crw-r-----  1 root  operator  0x4c Nov 25 17:56 /dev/vtbd0s4a
crw-r-----  1 root  operator  0x4d Nov 25 17:56 /dev/vtbd0s4b
crw-r-----  1 root  operator  0x4e Nov 25 17:56 /dev/vtbd0s4d
[0]# 

(This is from inside the running VM, therefore showing up as vtbd0 instead of da0.)

So it seems that in FreeBSD 12.2 bhyveload cannot successfully load a purely partitioned disk anymore (as opposed to sliced + partitioned). In 12.1 this wan working.

-- Martin
Comment 3 Martin Birgmeier 2020-11-25 19:53:17 UTC
I just noted that I labeled *s4d incorrectly, it should be UFS instead of ZFS. It's in any case holding the original UFS /usr filesystem.

-- Martin