I have the following kernel configuration for amd64, which I try to use for arm64 on a Raspberry Pi 4b.
After some trail and error I nailed the panic down to the KERN_TLS option. I am unable to get a crash dump, because my keyboard isn't working over USB, but I have attached a screenshot of the kernel panic.
The message of the panic is "panic: Thread already setup for VFP".
(In reply to Gordon Bergling from comment #0)
I don't see any attachment. I also can't reproduce any crashes when booting an arm64 KERN_TLS-enabled kernel on a test board.
Created attachment 216709 [details]
arm64 kernel panic on RPi4b
(In reply to Mark Johnston from comment #1)
It is maybe a combination of the kernel options above. I also compile world and kernel with the following src.conf.
Ah, it does panic if I add all three of RATELIMIT, TCPHPTS and KERN_TLS:
TCP_ratelimit: Is now initialized
panic: Thread already setup for the VFP
cpuid = 1
time = 13
KDB: stack backtrace:
db_trace_self() at db_fetch_ksymtab+0x158
pc = 0xffff0000007719e8 lr = 0xffff0000001094ac
sp = 0xffff00005aa50570 fp = 0xffff00005aa50770
db_fetch_ksymtab() at vpanic+0x194
pc = 0xffff0000001094ac lr = 0xffff00000041bcb0
sp = 0xffff00005aa50780 fp = 0xffff00005aa507d0
vpanic() at panic+0x44
pc = 0xffff00000041bcb0 lr = 0xffff00000041ba58
sp = 0xffff00005aa507e0 fp = 0xffff00005aa50890
panic() at fpu_kern_thread+0x68
pc = 0xffff00000041ba58 lr = 0xffff000000792cd8
sp = 0xffff00005aa508a0 fp = 0xffff00005aa508a0
fpu_kern_thread() at ktls_enqueue+0x47c
pc = 0xffff000000792cd8 lr = 0xffff0000004aa980
sp = 0xffff00005aa508b0 fp = 0xffff00005aa508f0
ktls_enqueue() at fork_exit+0x7c
pc = 0xffff0000004aa980 lr = 0xffff0000003d8144
sp = 0xffff00005aa50900 fp = 0xffff00005aa50950
fork_exit() at fork_trampoline+0x10
pc = 0xffff0000003d8144 lr = 0xffff000000790dc4
sp = 0xffff00005aa50960 fp = 0x0000000000000000
This may be a bug in the arm64 fpu interface? Note that the kthread calls fpu_kern_thread() first thing before it's done any work at all.
Ah, I think I see the bug. cpu_copy_thread() in vm_machdep.c just blindly copies pcb_flags which means that if the first kthread runs before another one is forked, the newly forked thread will bogusly have PCB_FP_KERN set in pcb_flags.
The amd64 version is careful to do this in cpu_copy_thread:
clear_pcb_flags(pcb2, PCB_FPUINITDONE | PCB_USERFPUINITDONE |
cpu_fork() for amd64 does an fpuexit() before copying the pcb which effectively does the same thing.
On arm64, cpu_fork() calls vfp_save_state which does not clear any pcb_flags after doing vfp_store/vfp_disable. It also doesn't check pcb_flags to determine if it should store/disable which is probably wrong. cpu_copy_thread() just needs to clear the relevant flags in the new pcb I think.
Any update on this topic regarding a bugfix?