Bug 223670 - lagg panic on arm64
Summary: lagg panic on arm64
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: arm64 Any
: --- Affects Some People
Assignee: Andrew Turner
URL:
Keywords:
: 222314 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-11-14 17:21 UTC by Bjoern A. Zeeb
Modified: 2018-10-18 15:59 UTC (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Bjoern A. Zeeb freebsd_committer 2017-11-14 17:21:05 UTC
root@freebsd12-test:~ # sh /etc/rc.d/netif restart ; sleep 30 && sh /etc/rc.d/routing restart
Stopping Network: lo0 vnic0 vnic1.
lo0: flags=8048<LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vnic0: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80002b<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU>
        ether ...
        inet6 ...%vnic0 prefixlen 64 tentative scopeid 0x2
        inet6 ... prefixlen 127 tentative
        media: Ethernet autoselect <full-duplex> (10Gbase-SR <full-duppanic: vm_fault_hold: fault on nofault entry, addr: 0xffff0000ac994000
cpuid = 74
time = 1508745074
KDB: stack backtrace:
db_trace_self() at db_trace_self_wrapper+0x28
	 pc = 0xffff00000060c848  lr = 0xffff000000086b8c
	 sp = 0xffff000ba49b1db0  fp = 0xffff000ba49b1fc0

db_trace_self_wrapper() at vpanic+0x184
	 pc = 0xffff000000086b8c  lr = 0xffff000000315818
	 sp = 0xffff000ba49b1fd0  fp = 0xffff000ba49b2050

vpanic() at panic+0x44
	 pc = 0xffff000000315818  lr = 0xffff0000003158a0
	 sp = 0xffff000ba49b2060  fp = 0xffff000ba49b20e0

panic() at vm_fault_hold+0x1d90
	 pc = 0xffff0000003158a0  lr = 0xffff0000005be5f4
	 sp = 0xffff000ba49b20f0  fp = 0xffff000ba49b2250

vm_fault_hold() at vm_fault+0x70
	 pc = 0xffff0000005be5f4  lr = 0xffff0000005bc814
	 sp = 0xffff000ba49b2260  fp = 0xffff000ba49b2290

vm_fault() at data_abort+0x100
	 pc = 0xffff0000005bc814  lr = 0xffff0000006264a8
	 sp = 0xffff000ba49b22a0  fp = 0xffff000ba49b2350

data_abort() at do_el1h_sync+0xf8
	 pc = 0xffff0000006264a8  lr = 0xffff0000006262a4
	 sp = 0xffff000ba49b2360  fp = 0xffff000ba49b2390

do_el1h_sync() at handle_el1h_sync+0x74
	 pc = 0xffff0000006262a4  lr = 0xffff00000060e874
	 sp = 0xffff000ba49b23a0  fp = 0xffff000ba49b24b0

handle_el1h_sync() at _mtx_init+0x88
	 pc = 0xffff00000060e874  lr = 0xffff0000002f8484
	 sp = 0xffff000ba49b24c0  fp = 0xffff000ba49b2570

_mtx_init() at _mtx_init+0x88
	 pc = 0xffff0000002f8484  lr = 0xffff0000002f8484
	 sp = 0xffff000ba49b2580  fp = 0xffff000ba49b25b0

_mtx_init() at vnet_lagg_init+0x34
	 pc = 0xffff0000002f8484  lr = 0xffff00006bbca03c
	 sp = 0xffff000ba49b25c0  fp = 0xffff000ba49b25c0

vnet_lagg_init() at vnet_register_sysinit+0x100
	 pc = 0xffff00006bbca03c  lr = 0xffff00000042868c
	 sp = 0xffff000ba49b25d0  fp = 0xffff000ba49b25f0

vnet_register_sysinit() at linker_load_module+0xa94
	 pc = 0xffff00000042868c  lr = 0xffff0000002e8dd4
	 sp = 0xffff000ba49b2600  fp = 0xffff000ba49b2930

linker_load_module() at kern_kldload+0xec
	 pc = 0xffff0000002e8dd4  lr = 0xffff0000002ea528
	 sp = 0xffff000ba49b2940  fp = 0xffff000ba49b2970

kern_kldload() at sys_kldload+0x64
	 pc = 0xffff0000002ea528  lr = 0xffff0000002ea680
	 sp = 0xffff000ba49b2980  fp = 0xffff000ba49b29a0

sys_kldload() at do_el0_sync+0x890
	 pc = 0xffff0000002ea680  lr = 0xffff000000626e8c
	 sp = 0xffff000ba49b29b0  fp = 0xffff000ba49b2a70

do_el0_sync() at handle_el0_sync+0x74
	 pc = 0xffff000000626e8c  lr = 0xffff00000060e9f4
	 sp = 0xffff000ba49b2a80  fp = 0xffff000ba49b2b90

handle_el0_sync() at 0x21280
	 pc = 0xffff00000060e9f4  lr = 0x0000000000021280
	 sp = 0xffff000ba49b2ba0  fp = 0x0000ffffffffe350

KDB: enter: panic
[ thread pid 1208 tid 100794 ]
Stopped at      lock_init+0x2c: ldrb    w8, [x21, #10]
db>
Comment 1 Bjoern A. Zeeb freebsd_committer 2017-11-16 17:53:44 UTC
disabling KD* did not help
disabling VIMAGE on top stopped the panic for now
Comment 2 Bjoern A. Zeeb freebsd_committer 2017-11-18 15:17:31 UTC
Based on the suggestion by @emaste I compiled lagg into the kernel;

even on a GENERIC kernel (which has VIMAGE compiled in as well) this just work.

So smells like a problem with modules or modules+VIMGE
Comment 3 Andrew Turner freebsd_committer 2017-11-20 18:13:21 UTC
This seems to be an issue with static VNET_DEFINE'd variables. These are accessed via a PC-relative address, however with VIMAGE they need to be relocated to be within the vnet allocation.

As a test I've stopped making these static in the if_lagg module and found this stopped the panic.
Comment 4 commit-hook freebsd_committer 2017-11-24 19:21:44 UTC
A commit references this bug:

Author: emaste
Date: Fri Nov 24 19:21:21 UTC 2017
New revision: 326179
URL: https://svnweb.freebsd.org/changeset/base/326179

Log:
  Temporarily disable VIMAGE on arm64

  Loading a kernel module with a static VNET_DEFINE'd variable (e.g.
  if_lagg) currently results in a kernel panic.

  PR:		223670

Changes:
  head/sys/arm64/conf/GENERIC
Comment 5 commit-hook freebsd_committer 2018-07-30 15:58:29 UTC
A commit references this bug:

Author: andrew
Date: Mon Jul 30 15:57:58 UTC 2018
New revision: 336915
URL: https://svnweb.freebsd.org/changeset/base/336915

Log:
  Enable VIMAGE on arm64 again. A workaround for modules with static VNET
  variables has been committed so these should work now.

  PR:		223670
  Sponsored by:	DARPA, AFRL

Changes:
  head/sys/arm64/conf/GENERIC
Comment 6 Ed Maste freebsd_committer 2018-10-04 01:10:11 UTC
Close this now?
Comment 7 Bjoern A. Zeeb freebsd_committer 2018-10-12 11:18:22 UTC
Can't see why this should still be open.
Comment 8 Bjoern A. Zeeb freebsd_committer 2018-10-15 16:26:09 UTC
*** Bug 222314 has been marked as a duplicate of this bug. ***
Comment 9 Bjoern A. Zeeb freebsd_committer 2018-10-18 15:59:05 UTC
*** Bug 222314 has been marked as a duplicate of this bug. ***