Bug 228854 - loaded pf.ko module prevents ipsec.ko from loading for VIMAGE/VNET enabled kernel
Summary: loaded pf.ko module prevents ipsec.ko from loading for VIMAGE/VNET enabled ke...
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 11.2-STABLE
Hardware: amd64 Any
: --- Affects Some People
Assignee: Bjoern A. Zeeb
URL:
Keywords: vimage
Depends on:
Blocks:
 
Reported: 2018-06-09 21:22 UTC by Marek Zarychta
Modified: 2018-12-22 01:00 UTC (History)
5 users (show)

See Also:
koobs: mfc-stable12+
koobs: mfc-stable11+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marek Zarychta 2018-06-09 21:22:20 UTC
the first introduction of dynamically loadable ipsec and tcpmd5 modules into STABLE branch. Since VIMAGE + 11-STABLE is officially unsupported I didn't report this issue before. Now I have a box running CURRNENT, so I could test this and confirm that CURRENT branch is also affected.

Loading dynamically ipsec.ko module is broken for both STABLE and CURRENT.

Quick steps for reproduction. 

KERNCONF:
include GENERIC
nooptions       IPSEC

# kldload ipsec
kldload: an error occurred while loading module ipsec. Please check dmesg(8) for more details.

# dmesg | tail -1
linker_load_file: /boot/kernel/ipsec.ko - unsupported file type

Let me also mention, that presence of VIMAGE option is and always was irrelevant to loading tcpmd5.ko module.
Comment 1 Marek Zarychta 2018-06-09 21:28:31 UTC
(In reply to Marek Zarychta from comment #0)
Please let me make a correction to the first sentence which was accidentally truncated:  

This problem is persistent from the first introduction of dynamically loadable ipsec and tcpmd5 modules into STABLE branch.
Comment 2 Eugene Grosbein freebsd_committer 2018-06-10 23:53:25 UTC
Make sure you rebuild kernel modules and not only kernel when you make changes to kernel configuration file. Some kernel options break compatibility of kernel with default kernel modules and VIMAGE is known to break it too. Rebuild of modules should solve the problem.
Comment 3 Marek Zarychta 2018-06-11 07:58:36 UTC
I am sorry for the false alert. It looks like the fresh version of CURRENT is error-free. Maybe it was connected with unclean obj dir and building option WITH_META_MODE.

# cat /etc/src-env.conf 
WITH_META_MODE=yes

STABLE anyway seems to be affected,

KERNCONF to reproduce:
include GENERIC
ident   VBSD
options         VIMAGE
options         IPSEC_SUPPORT
nooptions       IPSEC

Is this configuration supported under STABLE branch?
Comment 4 Eugene Grosbein freebsd_committer 2018-06-11 09:57:13 UTC
(In reply to Marek Zarychta from comment #3)

Yes, this is supported. So, did you rebuild kernel modules while building a kernel?
Comment 5 Marek Zarychta 2018-06-11 10:21:15 UTC
(In reply to Eugene Grosbein from comment #4)
I suppose "make buildkernel" does so in correct manner. No issue for kernels built without "option VIMAGE". I am building "WITH_META_MODE=yes", but disabling this mode doesn't make any difference.
Comment 6 Eugene Grosbein freebsd_committer 2018-06-11 10:57:57 UTC
(In reply to Marek Zarychta from comment #5)

make buildkernel does it correctly only if you have not MODULES_WITH_WORLD nor NO_MODULES, have you?

It's easy to compare modification times of installed kernel file and ipsec.ko, anyway.
Comment 7 Marek Zarychta 2018-06-11 11:16:00 UTC
Kernel (In reply to Eugene Grosbein from comment #6)
Kernel has been build and installed with modules, ipsec.ko and tcpmd5.ko are present among them. 

Aren't you able to reproduce this on 11-STABLE?

KERNCONF to reproduce:
include GENERIC
ident   VBSD
options         VIMAGE
options         IPSEC_SUPPORT
nooptions       IPSEC


KERNCONF allowing to load ipsec module:
include GENERIC
ident   VBSD
#options         VIMAGE
options         IPSEC_SUPPORT
nooptions       IPSEC
Comment 8 Andrey V. Elsukov freebsd_committer 2018-06-13 07:25:49 UTC
(In reply to Marek Zarychta from comment #0)
> # kldload ipsec
> kldload: an error occurred while loading module ipsec. Please check dmesg(8)
> for more details.
> 
> # dmesg | tail -1
> linker_load_file: /boot/kernel/ipsec.ko - unsupported file type

Usually there is one additional message, something about unresolved symbols I think.
Comment 9 Marek Zarychta 2018-06-13 11:19:51 UTC
(In reply to Andrey V. Elsukov from comment #8)
No additional information is provided, also when kldload is run in verbose mode.
Comment 10 Marek Zarychta 2018-06-13 15:56:21 UTC
(In reply to Andrey V. Elsukov from comment #8)
I attempted to load ipsec module through ktrace. 
Maybe output from kdump will shed some light on it.

  4988 ktrace   RET   ktrace 0
  4988 ktrace   CALL  execve(0x7fffffffe570,0x7fffffffeb00,0x7fffffffeb20)
  4988 ktrace   NAMI  "/sbin/kldload"
  4988 ktrace   NAMI  "/libexec/ld-elf.so.1"
  4988 kldload  RET   execve 0
  4988 kldload  CALL  mmap(0,0x8000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffff,0)
  4988 kldload  RET   mmap 34366160896/0x800620000
  4988 kldload  CALL  issetugid
  4988 kldload  RET   issetugid 0
  4988 kldload  CALL  lstat(0x800624000,0x7fffffffd2a8)
  4988 kldload  NAMI  "/etc"
  4988 kldload  STRU  struct stat {dev=3966367049, ino=34, mode=040755, nlink=27, uid=0, gid=0, rdev=4294967295, atime=1488438738, mtime=1528894917.915542000, ctime=1528894917.915542000, birthtime=1488438738.121758000, size=124, blksize=8192, blocks=17, flags=0x800 }
  4988 kldload  RET   lstat 0
  4988 kldload  CALL  lstat(0x800624000,0x7fffffffd2a8)
  4988 kldload  NAMI  "/etc/libmap.conf"
  4988 kldload  STRU  struct stat {dev=3966367049, ino=129, mode=0100644, nlink=1, uid=0, gid=0, rdev=4294967295, atime=1488438738, mtime=1486757987, ctime=1488438738.133604000, birthtime=1488438738.133580000, size=107, blksize=4096, blocks=9, flags=0x800 }
  4988 kldload  RET   lstat 0
  4988 kldload  CALL  openat(AT_FDCWD,0x800624000,0x100000<O_RDONLY|O_CLOEXEC>)
  4988 kldload  NAMI  "/etc/libmap.conf"
  4988 kldload  RET   openat 3
  4988 kldload  CALL  fstat(0x3,0x7fffffffdf68)
  4988 kldload  STRU  struct stat {dev=3966367049, ino=129, mode=0100644, nlink=1, uid=0, gid=0, rdev=4294967295, atime=1488438738, mtime=1486757987, ctime=1488438738.133604000, birthtime=1488438738.133580000, size=107, blksize=4096, blocks=9, flags=0x800 }
  4988 kldload  RET   fstat 0
  4988 kldload  CALL  mmap(0,0x6b,0x1<PROT_READ>,0x2<MAP_PRIVATE>,0x3,0)
  4988 kldload  RET   mmap 34366193664/0x800628000
  4988 kldload  CALL  close(0x3)
  4988 kldload  RET   close 0
  4988 kldload  CALL  lstat(0x800624800,0x7fffffffc648)
  4988 kldload  NAMI  "/usr"
  4988 kldload  STRU  struct stat {dev=3966367049, ino=9, mode=040755, nlink=17, uid=0, gid=0, rdev=4294967295, atime=1488438765, mtime=1491851814.556537000, ctime=1491851814.556537000, birthtime=1488438727.012480000, size=17, blksize=4096, blocks=17, flags=0x800 }
  4988 kldload  RET   lstat 0
  4988 kldload  CALL  lstat(0x800624800,0x7fffffffc648)
  4988 kldload  NAMI  "/usr/local"
  4988 kldload  STRU  struct stat {dev=3966367049, ino=481, mode=040755, nlink=18, uid=0, gid=0, rdev=4294967295, atime=1488438738, mtime=1528385864.682621000, ctime=1528385864.682621000, birthtime=1488438738.192704000, size=18, blksize=4096, blocks=17, flags=0x800 }
  4988 kldload  RET   lstat 0
  4988 kldload  CALL  lstat(0x800624800,0x7fffffffc648)
  4988 kldload  NAMI  "/usr/local/etc"
  4988 kldload  STRU  struct stat {dev=3966367049, ino=32834, mode=040755, nlink=53, uid=0, gid=0, rdev=4294967295, atime=1488441079.899548000, mtime=1528385863.622125000, ctime=1528385863.622125000, birthtime=1488441079.899548000, size=90, blksize=6144, blocks=17, flags=0x800 }
  4988 kldload  RET   lstat 0
  4988 kldload  CALL  lstat(0x800624800,0x7fffffffc648)
  4988 kldload  NAMI  "/usr/local/etc/libmap.d"
  4988 kldload  RET   lstat -1 errno 2 No such file or directory
  4988 kldload  CALL  munmap(0x800628000,0x6b)
  4988 kldload  RET   munmap 0
  4988 kldload  CALL  openat(AT_FDCWD,0x80061b429,0x100000<O_RDONLY|O_CLOEXEC>)
  4988 kldload  NAMI  "/var/run/ld-elf.so.hints"
  4988 kldload  RET   openat 3
  4988 kldload  CALL  read(0x3,0x800821628,0x80)
  4988 kldload  GIO   fd 3 read 128 bytes
       0x0000 4568 6e74 0100 0000 8000 0000 4301 0000 0000 0000 4201 0000 0000 0000 0000 0000 0000  |Ehnt........C.......B.............|
       0x0022 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000  |..................................|
       0x0044 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000  |..................................|
       0x0066 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000                      |..........................|

  4988 kldload  RET   read 128/0x80
  4988 kldload  CALL  fstat(0x3,0x7fffffffd7f0)
  4988 kldload  STRU  struct stat {dev=3966367049, ino=721374, mode=0100444, nlink=1, uid=0, gid=0, rdev=4294967295, atime=1528903774.820245000, mtime=1528903774.820294000, ctime=1528903774.820317000, birthtime=1528903774.820245000, size=451, blksize=4096, blocks=9, flags=0x800 }
  4988 kldload  RET   fstat 0
  4988 kldload  CALL  lseek(0x3,0x80,SEEK_SET)
  4988 kldload  RET   lseek 128/0x80
  4988 kldload  CALL  read(0x3,0x800625000,0x143)
  4988 kldload  GIO   fd 3 read 323 bytes
       "/lib:/usr/lib:/usr/lib/compat:/usr/local/lib:/usr/local/lib/R/lib:/usr/local/lib/compat:/usr/local/lib/gcc6:/usr/local/lib/graphvi\
	z:/usr/local/lib/nss:/usr/local/lib/opencollada:/usr/local/lib/perl5/5.26/mach/CORE:/usr/local/lib/qt5:/usr/local/lib/samba4:/usr/\
	local/lib/xrdp:/usr/local/llvm60/lib:/usr/local/share/chromium\0"
  4988 kldload  RET   read 323/0x143
  4988 kldload  CALL  close(0x3)
  4988 kldload  RET   close 0
  4988 kldload  CALL  access(0x800624800,0<F_OK>)
  4988 kldload  NAMI  "/lib/libc.so.7"
  4988 kldload  RET   access 0
  4988 kldload  CALL  openat(AT_FDCWD,0x800621060,0x300000<O_RDONLY|O_CLOEXEC|O_VERIFY>)
  4988 kldload  NAMI  "/lib/libc.so.7"
  4988 kldload  RET   openat 3
  4988 kldload  CALL  fstat(0x3,0x7fffffffdb28)
  4988 kldload  STRU  struct stat {dev=3966367049, ino=655658, mode=0100444, nlink=1, uid=0, gid=0, rdev=4294967295, atime=1528875590.643985000, mtime=1528875590.644651000, ctime=1528875590.645044000, birthtime=1528875590.643985000, size=1769376, blksize=131072, blocks=2169, flags=0x20000 }
  4988 kldload  RET   fstat 0
  4988 kldload  CALL  mmap(0,0x1000,0x1<PROT_READ>,0x40002<MAP_PRIVATE|MAP_PREFAULT_READ>,0x3,0)
  4988 kldload  RET   mmap 34366193664/0x800628000
  4988 kldload  CALL  mmap(0,0x3ba000,0<PROT_NONE>,0x2000<MAP_GUARD>,0xffffffff,0)
  4988 kldload  RET   mmap 34368266240/0x800822000
  4988 kldload  CALL  mmap(0x800822000,0x195000,0x5<PROT_READ|PROT_EXEC>,0x60012<MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ>,0x3,0)
  4988 kldload  RET   mmap 34368266240/0x800822000
  4988 kldload  CALL  mmap(0x800bb6000,0xd000,0x3<PROT_READ|PROT_WRITE>,0x40012<MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ>,0x3,0x194000)
  4988 kldload  RET   mmap 34372018176/0x800bb6000
  4988 kldload  CALL  mmap(0x800bc3000,0x19000,0x3<PROT_READ|PROT_WRITE>,0x1012<MAP_PRIVATE|MAP_FIXED|MAP_ANON>,0xffffffff,0)
  4988 kldload  RET   mmap 34372071424/0x800bc3000
  4988 kldload  CALL  munmap(0x800628000,0x1000)
  4988 kldload  RET   munmap 0
  4988 kldload  CALL  close(0x3)
  4988 kldload  RET   close 0
  4988 kldload  CALL  mmap(0,0x19000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffff,0)
  4988 kldload  RET   mmap 34366193664/0x800628000
  4988 kldload  CALL  sigprocmask(SIG_BLOCK,0x800821738,0x7fffffffdfd0)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  sigprocmask(SIG_SETMASK,0x80082174c,0)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  readlink(0x800987a18,0x7fffffffd700,0x400)
  4988 kldload  NAMI  "/etc/malloc.conf"
  4988 kldload  RET   readlink -1 errno 2 No such file or directory
  4988 kldload  CALL  issetugid
  4988 kldload  RET   issetugid 0
  4988 kldload  CALL  __sysctl(0x7fffffffd5a0,0x2,0x7fffffffd5f0,0x7fffffffd5e8,0x8009864ab,0xd)
  4988 kldload  SCTL  "sysctl.name2oid"
  4988 kldload  RET   __sysctl 0
  4988 kldload  CALL  __sysctl(0x7fffffffd5f0,0x2,0x7fffffffd6b4,0x7fffffffd6a8,0,0)
  4988 kldload  SCTL  "vm.overcommit"
  4988 kldload  RET   __sysctl 0
  4988 kldload  CALL  mmap(0,0x200000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffff,0)
  4988 kldload  RET   mmap 34372173824/0x800bdc000
  4988 kldload  CALL  munmap(0x800bdc000,0x200000)
  4988 kldload  RET   munmap 0
  4988 kldload  CALL  mmap(0,0x3ff000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffff,0)
  4988 kldload  RET   mmap 34372173824/0x800bdc000
  4988 kldload  CALL  munmap(0x800bdc000,0x24000)
  4988 kldload  RET   munmap 0
  4988 kldload  CALL  munmap(0x800e00000,0x1db000)
  4988 kldload  RET   munmap 0
  4988 kldload  CALL  sigprocmask(SIG_BLOCK,0x800821738,0x7fffffffdb50)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  sigprocmask(SIG_SETMASK,0x80082174c,0)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  sigprocmask(SIG_BLOCK,0x800821738,0x7fffffffdb50)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  sigprocmask(SIG_SETMASK,0x80082174c,0)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  kldload(0x7fffffffedab)
  4988 kldload  NAMI  "/boot/kernel/linker.hints"
  4988 kldload  NAMI  "/boot/kernel/ipsec.ko"
  4988 kldload  NAMI  "/boot/kernel/ipsec.ko"
  4988 kldload  NAMI  "/boot/kernel/ipsec.ko"
  4988 kldload  RET   kldload -1 errno 8 Exec format error
  4988 kldload  CALL  write(0x2,0x7fffffffdda0,0x9)
  4988 kldload  GIO   fd 2 wrote 9 bytes
       "kldload: "
  4988 kldload  RET   write 9
  4988 kldload  CALL  write(0x2,0x7fffffffdea0,0x53)
  4988 kldload  GIO   fd 2 wrote 83 bytes
       "an error occurred while loading the module. Please check dmesg(8) for more details."
  4988 kldload  RET   write 83/0x53
  4988 kldload  CALL  write(0x2,0x800bc1c67,0x1)
  4988 kldload  GIO   fd 2 wrote 1 byte
       "
       "
  4988 kldload  RET   write 1
  4988 kldload  CALL  sigprocmask(SIG_BLOCK,0x800821738,0x7fffffffe910)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  sigprocmask(SIG_SETMASK,0x80082174c,0)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  sigprocmask(SIG_BLOCK,0x800821738,0x7fffffffe480)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  sigprocmask(SIG_SETMASK,0x80082174c,0)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  sigprocmask(SIG_BLOCK,0x800821738,0x7fffffffe480)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  sigprocmask(SIG_SETMASK,0x80082174c,0)
  4988 kldload  RET   sigprocmask 0
  4988 kldload  CALL  exit(0x1)
Comment 11 Marek Zarychta 2018-10-28 22:37:20 UTC
Thanks to the mailing list freebsd-current it came out that the cause of failure is already loaded pf.ko module which also collides with ipfilter.ko so both can't be loaded at the same time.  

Both 11-STABLE and 12-STABLE seem to be affected.
Comment 12 Bjoern A. Zeeb freebsd_committer 2018-10-30 20:23:48 UTC
Presumably out of vnet module space.
See https://lists.freebsd.org/pipermail/freebsd-current/2018-October/071972.html thread as well.
Comment 13 commit-hook freebsd_committer 2018-10-30 20:46:18 UTC
A commit references this bug:

Author: bz
Date: Tue Oct 30 20:45:15 UTC 2018
New revision: 339930
URL: https://svnweb.freebsd.org/changeset/base/339930

Log:
  With more excessive use of modules, more kernel parts working with
  VIMAGE, and feature richness and global state increasing the 8k of
  vnet module space are no longer sufficient for people and loading
  multiple modules, e.g., pf(4) and ipl(4) or ipsec(4) will fail on
  the second module.

  Increase the module space to 8 * PAGE_SIZE which should be enough
  to hold multiple firewalls, ipsec, multicast (as in the old days was
  a problem), epair, carp, and any kind of other vnet enabled modules.

  Sadly this is a global byte array part of the vnet_set, so we cannot
  dynamically change its size;  otherwise a TUNABLE would have been
  a better solution.

  PR:			228854
  Reported by:		Ernie Luzar, Marek Zarychta
  Discussed with:		rgrimes on current
  MFC after:		3 days

Changes:
  head/sys/net/vnet.c
Comment 14 commit-hook freebsd_committer 2018-10-30 20:51:24 UTC
A commit references this bug:

Author: bz
Date: Tue Oct 30 20:51:03 UTC 2018
New revision: 339931
URL: https://svnweb.freebsd.org/changeset/base/339931

Log:
  As a follow-up to r339930 and various reports implement logging in case
  we fail during module load because the pcpu or vnet module sections are
  full.  We did return a proper error but not leaving any indication to
  the user as to what the actual problem was.

  Even worse, on 12/13 currently we are seeing an unrelated error (ENOSYS
  instead of ENOSPC, which gets skipped over in kern_linker.c) to be
  printed which made problem diagnostics even harder.

  PR:		228854
  MFC after:	3 days

Changes:
  head/sys/kern/link_elf.c
  head/sys/kern/link_elf_obj.c
Comment 15 Bjoern A. Zeeb freebsd_committer 2018-10-30 21:29:40 UTC
Broke mips;  follow-up commit to come; will try to MFC on Fri.
Comment 16 commit-hook freebsd_committer 2018-10-30 21:36:59 UTC
A commit references this bug:

Author: bz
Date: Tue Oct 30 21:35:57 UTC 2018
New revision: 339933
URL: https://svnweb.freebsd.org/changeset/base/339933

Log:
  Fix mips build after r339931.
  I erroneously thought that it was two 64bit platforms which use link_elf_obj.c.

  PR:		228854
  Reported by:	ci.f.o.
  MFC after:	3 days
  X-MFC with:	r339931
  Pointyhat to:	bz

Changes:
  head/sys/kern/link_elf_obj.c
Comment 17 commit-hook freebsd_committer 2018-10-31 12:51:31 UTC
A commit references this bug:

Author: bz
Date: Wed Oct 31 12:50:40 UTC 2018
New revision: 339951
URL: https://svnweb.freebsd.org/changeset/base/339951

Log:
  MFC r339930:

     With more excessive use of modules, more kernel parts working with
     VIMAGE, and feature richness and global state increasing the 8k of
     vnet module space are no longer sufficient for people and loading
     multiple modules, e.g., pf(4) and ipl(4) or ipsec(4) will fail on
     the second module.

     Increase the module space to 8 * PAGE_SIZE which should be enough
     to hold multiple firewalls, ipsec, multicast (as in the old days was
     a problem), epair, carp, and any kind of other vnet enabled modules.

     Sadly this is a global byte array part of the vnet_set, so we cannot
     dynamically change its size;  otherwise a TUNABLE would have been
     a better solution.

  PR:			228854
     Reported by:	Ernie Luzar, Marek Zarychta

  Approved by:	re (kib)

Changes:
_U  stable/12/
  stable/12/sys/net/vnet.c
Comment 18 commit-hook freebsd_committer 2018-10-31 14:04:34 UTC
A commit references this bug:

Author: bz
Date: Wed Oct 31 14:03:48 UTC 2018
New revision: 339953
URL: https://svnweb.freebsd.org/changeset/base/339953

Log:
  MFC r339931,r339933:

     As a follow-up to r339930 (r339951 in stable/12) and various reports
     implement logging in case we fail during module load because the pcpu
     or vnet module sections are full.
     We did return a proper error but not leaving any indication to the
     user as to what the actual problem was.

     Even worse, on 12/13 currently we are seeing an unrelated error (ENOSYS
     instead of ENOSPC, which gets skipped over in kern_linker.c) to be
     printed which made problem diagnostics even harder.

  PR:		228854

  Approved by:	re (kib)

Changes:
_U  stable/12/
  stable/12/sys/kern/link_elf.c
  stable/12/sys/kern/link_elf_obj.c
Comment 19 commit-hook freebsd_committer 2018-11-02 14:07:51 UTC
A commit references this bug:

Author: bz
Date: Fri Nov  2 14:07:07 UTC 2018
New revision: 340051
URL: https://svnweb.freebsd.org/changeset/base/340051

Log:
  MFC r339930:

    With more excessive use of modules, more kernel parts working with
    VIMAGE, and feature richness and global state increasing the 8k of
    vnet module space are no longer sufficient for people and loading
    multiple modules, e.g., pf(4) and ipl(4) or ipsec(4) will fail on
    the second module.

    Increase the module space to 8 * PAGE_SIZE which should be enough
    to hold multiple firewalls, ipsec, multicast (as in the old days was
    a problem), epair, carp, and any kind of other vnet enabled modules.

    Sadly this is a global byte array part of the vnet_set, so we cannot
    dynamically change its size;  otherwise a TUNABLE would have been
    a better solution.

  PR:			228854

Changes:
_U  stable/11/
  stable/11/sys/net/vnet.c
Comment 20 commit-hook freebsd_committer 2018-11-02 14:16:02 UTC
A commit references this bug:

Author: bz
Date: Fri Nov  2 14:15:52 UTC 2018
New revision: 340054
URL: https://svnweb.freebsd.org/changeset/base/340054

Log:
  MFC r339931,r339933

    As a follow-up to r339930 and various reports implement logging in case
    we fail during module load because the pcpu or vnet module sections are
    full.  We did return a proper error but not leaving any indication to
    the user as to what the actual problem was.

  PR:		228854

Changes:
_U  stable/11/
  stable/11/sys/kern/link_elf.c
  stable/11/sys/kern/link_elf_obj.c
Comment 21 Bjoern A. Zeeb freebsd_committer 2018-11-02 14:17:49 UTC
We think that for stable/11 and stable/12 and HEAD this should be fixed now.
Thanks a lot for reporting!