Bug 201679

Summary: Upgrade of -STABLE from 20150627 to 20150717 keeps loader from loading vmm.ko
Product: Base System Reporter: hartzell
Component: kernAssignee: John Baldwin <jhb>
Status: Closed FIXED    
Severity: Affects Only Me CC: andrey, emaste, jakob, jhb, max.n.boyarov
Priority: --- Keywords: regression
Version: 10.2-STABLEFlags: jhb: mfc-stable10+
Hardware: Any   
OS: Any   

Description hartzell 2015-07-18 18:13:07 UTC 
I upgraded my 10.2-STABLE system via svn last night, did the build and install world and kernel dance.  Generally everything is as it should be.

But, I can no longer load `vmm.ko` via the loader, which used to work.

I have these lines in `/boot/loader.conf`:

```
vmm_load="YES"
pptdevs="0/23/0"
```

and/but at the machine boots, I get this on the console.

```
>> FreeBSD EFI boot block
   Loader path: /boot/loader.efi
Consoles: EFI console  
Image base: 0x78a69000
EFI version: 2.40
EFI Firmware: EDK II (rev 1.00)
 
FreeBSD/amd64 EFI loader, Revision 1.1
(root@fishy.alerce.com, Sat Jul 18 00:42:15 PDT 2015)
Loading /boot/defaults/loader.conf
/boot/kernel/kernel text=0x1396e50 data=0x132738+0x4d7f40 syms=[0x8+0x159d50+0x8
+0x17465c]
/boot/entropy size=0x1000
/boot/kernel/vmm.ko |
elf64_obj_loadimage: read failed
 
 
Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...               
Start @ 0xffffffff802f9000 ...
```

I am unable to load them by hand via the loader either:

```
OK 
OK load vmm
/boot/kernel/vmm.ko -
elf64_obj_loadimage: read failed
 
can't load file '/boot/kernel/vmm.ko': input/output error
OK 
```

Loading `/boot/kernel.old/vmm.ko` (from the old build) also fails.

I can load the module using `kldload` once the system is up and running.  

I also noticed that the `if_urtwn.ko` module loaded via the loader, so I think that loading kernel modules is not generally broken.

```
root@fishy:~ # kldload vmm
root@fishy:~ # kldstat
Id Refs Address            Size     Name
 1    9 0xffffffff80200000 1e6fa38  kernel
 2    1 0xffffffff82211000 11f42    if_urtwn.ko
 3    1 0xffffffff82223000 338ac2   vmm.ko
root@fishy:~ # 
```

I'd like to get this working.  Suggestions?
Comment 1 Jakob Alvermark 2015-07-27 20:32:24 UTC 
I get the same error on -CURRENT
Also using EFI loader.
It works fine on r285338 from July 10, updated on July 24 and now the loader refuses to load zfs.ko (which prevents me to boot since my root file system is on ZFS, only /boot is on UFS).
Unloading and loading the old kernel and zfs.ko works.
Comment 2 Jakob Alvermark 2015-08-04 11:49:03 UTC 
(In reply to Jakob Alvermark from comment #1)
I did some more tests.

Loading the old kernel lets me load the new zfs.ko.
So the problem seems to depend on the loaded kernel.
Trying to find differences I noticed that the new kernel is considerably larger in size. ~22M vs ~25M.

Removing a bunch of drivers from the kernel config got me a new kernel that's ~18M.
It works! It let's me load all the modules.

Could there be a problem with kernels larger than 24M?
Comment 3 Andrey Fesenko 2015-08-14 13:50:10 UTC 
(In reply to Jakob Alvermark from comment #1)
Confirm

I upgraded my CURRENT system to r286760

But, I can no longer load `zfs.ko` via the loader (install only new kernel)

Loading /boot/defaults/loader.conf
/boot/kernel/kernel text=0x13e0b30 data=0x1...
+0x17ac14]
/boot/entropy size=0x1000
/boot/kernel/zfs.ko |
elf64_obj_loadimage: read failed
 
.... (success load other .ko)
Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...

and system not bee start (root on ZFS)

If restore old kernel (r285339), boot success.
Comment 4 Andrey Fesenko 2015-08-14 14:37:15 UTC 
I'm build "make -C release mini-memstick"
if set # cat /boot/loader.conf 
vfs.mountroot.timeout="10"
zfs_load="YES"

module not load with start system, but success kldload after start.
Comment 5 Ed Maste freebsd_committer freebsd_triage 2015-08-14 15:08:38 UTC 
The addition of the pms driver in r285242/r286231 is responsible for the increase in kernel size noted above. This results in a failure to load some modules due to limitations in the UEFI loader. So the immediate workaround is indeed to build a kernel with unnecessary drivers removed.
Comment 6 Jakob Alvermark 2015-08-14 16:12:56 UTC 
(In reply to Ed Maste from comment #5)
Out of curiosity, what are the limits? (And why?) And is there a solution in the works?
Comment 7 John Baldwin freebsd_committer freebsd_triage 2015-08-14 17:34:49 UTC 
The current limit is 32MB.  You can change it at a compile time via 'make EFI_STAGING_SIZE=<MB>' for sys/boot/efi/loader. (or sys/boot/amd64/efi on stable/10).  Prior to r279929 (merged to stable in r281319) loading too many modules would just corrupt memory.  Now it reports an error at least. :)

It may be that we should revisit the default staging size of 32MB and bump it up to 48MB or so.  When testing the fixes mentioned above I was able to boot a 200+MB mfsroot with a sufficiently large staging size.
Comment 8 John Baldwin freebsd_committer freebsd_triage 2015-09-14 21:55:32 UTC 
Please see the candidate patch at https://reviews.freebsd.org/D3666
Comment 9 Jakob Alvermark 2015-09-17 18:24:39 UTC 
(In reply to John Baldwin from comment #8)
Tested on my Acer laptop. Seems to do the trick! Kernel and modules loads without issue and the system boots normally.
Comment 10 commit-hook freebsd_committer freebsd_triage 2015-09-17 20:37:16 UTC 
A commit references this bug:

Author: jhb
Date: Thu Sep 17 20:36:47 UTC 2015
New revision: 287934
URL: https://svnweb.freebsd.org/changeset/base/287934

Log:
  The EFI boot loader allocates a single chunk of contiguous memory to
  hold the kernel, modules, and any other loaded data.  This memory block
  is relocated to the kernel's expected location during the transfer of
  control from the loader to the kernel.

  The GENERIC kernel on amd64 has recently grown such that a kernel + zfs.ko
  no longer fits in the default staging size.  Bump the default size from
  32MB to 48MB to provide more breathing room.

  PR:		201679
  Reviewed by:	imp
  MFC after:	1 week
  Differential Revision:	https://reviews.freebsd.org/D3666

Changes:
  head/sys/boot/efi/loader/copy.c
Comment 11 commit-hook freebsd_committer freebsd_triage 2015-11-05 19:56:22 UTC 
A commit references this bug:

Author: jhb
Date: Thu Nov  5 19:55:45 UTC 2015
New revision: 290410
URL: https://svnweb.freebsd.org/changeset/base/290410

Log:
  MFC 287934:
  The EFI boot loader allocates a single chunk of contiguous memory to
  hold the kernel, modules, and any other loaded data.  This memory block
  is relocated to the kernel's expected location during the transfer of
  control from the loader to the kernel.

  The GENERIC kernel on amd64 has recently grown such that a kernel + zfs.ko
  no longer fits in the default staging size.  Bump the default size from
  32MB to 48MB to provide more breathing room.

  PR:		201679

Changes:
_U  stable/10/
  stable/10/sys/boot/amd64/efi/copy.c
Comment 12 commit-hook freebsd_committer freebsd_triage 2016-01-07 02:42:20 UTC 
A commit references this bug:

Author: emaste
Date: Thu Jan  7 02:41:58 UTC 2016
New revision: 293300
URL: https://svnweb.freebsd.org/changeset/base/293300

Log:
  MFC r287934: Increase EFI staging size from 32MB to 48MB

  The EFI boot loader allocates a single chunk of contiguous memory to
  hold the kernel, modules, and any other loaded data.  This memory block
  is relocated to the kernel's expected location during the transfer of
  control from the loader to the kernel.

  PR:		201679

Changes:
  stable/10/sys/boot/efi/loader/copy.c
Comment 13 commit-hook freebsd_committer freebsd_triage 2019-02-03 15:19:39 UTC 
A commit references this bug:

Author: arrowd
Date: Sun Feb  3 15:19:34 UTC 2019
New revision: 492054
URL: https://svnweb.freebsd.org/changeset/ports/492054

Log:
  lang/ghc862: Fix distinfo entry for aarch64 bootstrap compiler.

  PR:		201679
  Submitted by:	Mikael Urankar <mikael.urankar@gmail.com>
  Approved by:	tcberner (mentor)

Changes:
  head/lang/ghc862/distinfo