I'm hitting a kernel panic when trying to boot recent versions of FreeBSD (both main and stable/13) on qemu-system-mips64. I've been able to establish that this is a regression caused by: 248f0cabca75 make maximum interrupt number tunable on ARM, ARM64, MIPS, and RISC-V I've been following the build instructions from (to the letter): https://wiki.freebsd.org/QemuRecipes#mips64 When booting the kernel, I got the following panic: ``` entry: platform_start() cmd line: kernel envp: memsize = 268435456 ememsize = 2147483648 modetty0 = 38400n8r memsize = 268435456 (0x10000000) ememsize = 2147483648 Cache info: icache is virtual picache_stride = 8192 picache_loopcount = 4 pdcache_stride = 4096 pdcache_loopcount = 8 max line size = 32 Physical memory chunk(s): 0x951000 - 0xfffffff, 258666496 bytes (63151 pages) 0x90000000 - 0xffffffff, 1879048192 bytes (458752 pages) Maxmem is 0x100000000 ---<<BOOT>>--- KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.0-CURRENT #3 master-bae4112-dirty: Wed Jan 27 20:24:56 UTC 2021 root@fb:/usr/obj/usr/src/mips.mips64/sys/MALTA64 mips FreeBSD clang version 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c3fe) Preloaded elf kernel "kernel" at 0xffffffff80949970. CPU: MIPS Technologies processor v160.130 MMU: Standard TLB, 48 entries (4K 16K 64K 256K 1M 16M 64M 256M pg sizes) L1 i-cache: 4 ways of 256 sets, 32 bytes per line L1 d-cache: 4 ways of 256 sets, 32 bytes per line L2 cache: disabled Config1=0xdea3519b<PerfCount,WatchRegs,EJTAG,FPU> FPU ID=0xf8200<S,D,PS,3D,W> Config2=0x80000000 real memory = 2147483648 (2097152K bytes) Physical memory chunk(s): 0x00a6e000 - 0x0fffffff, 257499136 bytes (62866 pages) 0x90000000 - 0xfcd7bfff, 1826078720 bytes (445820 pages) avail memory = 2077196288 (1980MB) panic: mips_intrcnt_setname: NULL counter time = 1 KDB: enter: panic [ thread pid 0 tid 0 ] Stopped at 0x4 ``` If I revert the following commit, the kernel boots fine: commit 248f0cabca75f421553633184b5bf2ffa459d3de Author: Oleksandr Tymoshenko <gonzo@FreeBSD.org> Date: 2020-12-30 07:59:03 +0100 make maximum interrupt number tunable on ARM, ARM64, MIPS, and RISC-V Use a machdep.nirq tunable intead of compile-time constant NIRQ as a value for maximum number of interrupts. It allows keep a system footprint small by default with an option to increase the limit for large systems like server-grade ARM64 Reviewd by: mhorne Differential Revision: https://reviews.freebsd.org/D27844 Submitted by: Klara, Inc. Sponsored by: Ampere Computing I don't have any real MIPS hardware to know if it's specific to QEMU or a generic problem.
Hi, do you have a backtrace from the panic? By code inspection, I suspect the problem comes from the fact that cpu_init_interrupts() is called before the intrcnt array is allocated.
I'm afraid not: db> bt Tracing pid 0 tid 0 td 0xffffffff80900240 --- swapped out Not even if I swapped kernel.full in place of kernel.
Created attachment 221970 [details] delay call to cpu_init_interrupts (In reply to Michał Górny from comment #2) That's unfortunate. Can you see if the attached patch fixes it? I have not compiled or tried it myself yet.
It still fails but I can get some backtrace now ;-). --- entry: platform_start() cmd line: kernel envp: memsize = 134217728 ememsize = 134217728 modetty0 = 38400n8r memsize = 134217728 (0x08000000) ememsize = 134217728 Cache info: icache is virtual picache_stride = 8192 picache_loopcount = 4 pdcache_stride = 4096 pdcache_loopcount = 8 max line size = 32 Physical memory chunk(s): 0x94e000 - 0x7ffffff, 124461056 bytes (30386 pages) Maxmem is 0x8000000 ---<<BOOT>>--- KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-ALPHA2 #8 master-bae4112-dirty: Wed Jan 27 22:47:19 UTC 2021 root@fb:/usr/obj/usr/src/mips.mips64/sys/MALTA64 mips FreeBSD clang version 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c3fe) Preloaded elf kernel "kernel" at 0xffffffff80947930. CPU: MIPS Technologies processor v160.130 MMU: Standard TLB, 48 entries (4K 16K 64K 256K 1M 16M 64M 256M pg sizes) L1 i-cache: 4 ways of 256 sets, 32 bytes per line L1 d-cache: 4 ways of 256 sets, 32 bytes per line L2 cache: disabled Config1=0xdea3519b<PerfCount,WatchRegs,EJTAG,FPU> FPU ID=0xf8200<S,D,PS,3D,W> Config2=0x80000000 real memory = 134217728 (131072K bytes) Physical memory chunk(s): 0x00a6b000 - 0x07d14fff, 120233984 bytes (29354 pages) avail memory = 119300096 (113MB) random: no preloaded entropy cache arc4random: WARNING: initial seeding bypassed the cryptographic random device because it was not yet seeded and the knob 'bypass_before_seeding' was enabled. hostuuid: using 00000000-0000-0000-0000-000000000000 panic: mips_intrcnt_setname: NULL counter time = 1 KDB: enter: panic [ thread pid 0 tid 100000 ] Stopped at 0x4 db> bt Tracing pid 0 tid 100000 td 0xffffffff80900240 kdb_enter+0x94 (?,?,?,?) ra ffffffff8046e474 sp 9800000000967b70 sz 32 vpanic+0x23c (?,?,?,?) ra ffffffff8046e0a8 sp 9800000000967b90 sz 64 panic+0x30 (?,?,?,?) ra ffffffff80772288 sp 9800000000967bd0 sz 80 cpu_init_interrupts+0x318 (?,?,?,?) ra ffffffff80755720 sp 9800000000967c20 sz 128 is_cacheable_mem+0x2d8 (?,?,?,?) ra ffffffff803d4e68 sp 9800000000967ca0 sz 16 mi_startup+0x280 (?,?,?,?) ra ffffffff80100404 sp 9800000000967cb0 sz 96 btext+0xdc (?,?,?,?) ra 0 sp 9800000000967d10 sz 0
Ooops, sorry for the breakage. Mitchell, are you looking into this or should I take this PR?
(In reply to Oleksandr Tymoshenko from comment #5) Please take it. I will not have a chance to debug any further this week.
This should fix the problem: https://reviews.freebsd.org/D28424 Let me know if the patch works for you.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=e0a0a3efcb09a10ad1de29aca622ea580b2663d2 commit e0a0a3efcb09a10ad1de29aca622ea580b2663d2 Author: Oleksandr Tymoshenko <gonzo@FreeBSD.org> AuthorDate: 2021-01-30 06:11:24 +0000 Commit: Oleksandr Tymoshenko <gonzo@FreeBSD.org> CommitDate: 2021-01-31 21:44:45 +0000 mips: fix early kernel panic when setting up interrupt counters Commit 248f0ca converted intrcnt and intrnames from u_long[] and char[] to u_long* and char* respectively, but for non-INTRNG mips these symbols were defined in .S file as a pre-allocated static arrays, so the problem wasn't cought at compile time. Conversion from an array to a pointer requires pointer initialization and it wasn't done for MIPS, so whatever happenned to be in the begginning of intcnt[] array was used as a pointer value. Move intrcnt/intrnames to C code and allocate them dynamically although with a fixed size at the moment. Reviewed by: emaste PR: 253051 Differential Revision: https://reviews.freebsd.org/D28424 MFC after: 1 day sys/mips/mips/exception.S | 36 ------------------------------------ sys/mips/mips/intr_machdep.c | 22 ++++++++++++++++++++++ 2 files changed, 22 insertions(+), 36 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=d6f9c5a6d2f87865f9714d2b8dfd1a9f3080c71e commit d6f9c5a6d2f87865f9714d2b8dfd1a9f3080c71e Author: Oleksandr Tymoshenko <gonzo@FreeBSD.org> AuthorDate: 2021-02-01 07:56:22 +0000 Commit: Oleksandr Tymoshenko <gonzo@FreeBSD.org> CommitDate: 2021-02-01 07:56:22 +0000 mips: fix NLM platforms breakage caused by e0a0a3ef NetLogic platforms have their own implementation of cpu_init_interrupts. Apply the same logic to it as to intr_machdep.c. PR: 253051 sys/mips/nlm/intr_machdep.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=10625515ad47b52f38c13f4f1d8f8c848e9df4ee commit 10625515ad47b52f38c13f4f1d8f8c848e9df4ee Author: Oleksandr Tymoshenko <gonzo@FreeBSD.org> AuthorDate: 2021-02-01 07:56:22 +0000 Commit: Oleksandr Tymoshenko <gonzo@FreeBSD.org> CommitDate: 2021-02-03 05:06:37 +0000 MFC: mips: fix early kernel panic when setting up interrupt counters Commit 248f0ca converted intrcnt and intrnames from u_long[] and char[] to u_long* and char* respectively, but for non-INTRNG mips these symbols were defined in .S file as a pre-allocated static arrays, so the problem wasn't cought at compile time. Conversion from an array to a pointer requires pointer initialization and it wasn't done for MIPS, so whatever happenned to be in the begginning of intcnt[] array was used as a pointer value. Move intrcnt/intrnames to C code and allocate them dynamically although with a fixed size at the moment. Reviewed by: emaste PR: 253051 Differential Revision: https://reviews.freebsd.org/D28424 MFC after: 1 day (cherry picked from commit e0a0a3efcb09a10ad1de29aca622ea580b2663d2) mips: fix NLM platforms breakage caused by e0a0a3ef NetLogic platforms have their own implementation of cpu_init_interrupts. Apply the same logic to it as to intr_machdep.c. PR: 253051 (cherry picked from commit d6f9c5a6d2f87865f9714d2b8dfd1a9f3080c71e) sys/mips/mips/exception.S | 36 ------------------------------------ sys/mips/mips/intr_machdep.c | 22 ++++++++++++++++++++++ sys/mips/nlm/intr_machdep.c | 18 ++++++++++++++++++ 3 files changed, 40 insertions(+), 36 deletions(-)