Created attachment 221415 [details] arm64 boot panic message On a recent build of -CURRENT on arm64 I get this panic during boot: "Too many early devmatch mappings".
This panic happens on GENERIC and on GENERIC-MMCCAM.
After some time bisecting commits, I identified, ------------------------------------------------------------------------------- commit e83fdf8bb391579fa422d34663cd8c1f82a00dc0 Author: Chuck Tuffli <chuck@FreeBSD.org> AuthorDate: 2021-01-08 22:36:37 +0000 Commit: Chuck Tuffli <chuck@FreeBSD.org> CommitDate: 2021-01-08 22:41:45 +0000 fix big-endian platforms after 6733401935f8 ------------------------------------------------------------------------------- as the offending commit that leads towards the kernel panic, at least on the RPi4B. The following commit boots just fine. ------------------------------------------------------------------------------- commit a82f07fc2e2456b0ddc847d611f56a3152a6f940 Author: Dimitry Andric <dim@FreeBSD.org> AuthorDate: 2021-01-08 22:38:21 +0000 Commit: Dimitry Andric <dim@FreeBSD.org> CommitDate: 2021-01-08 22:38:30 +0000 Fix 32-bit build post 6733401935f83754b4b2744bc3d33ef84b1271e0 ------------------------------------------------------------------------------- KERNCONF is the following: ------------------------------------------------------------------------------- include GENERIC # Extented TCP Congestion Control Methods options RATELIMIT options TCPHPTS # Scalable Route Multipath options ROUTE_MPATH # Modular FIB Lookup Framework options FIB_ALGO # IN Kernel TLS -> Disabled PR: 247945 #options KERN_TLS # New FENESTRASX Random Number Generator options RANDOM_FENESTRASX # Add CAMDEBUG stuff #options CAMDEBUG #options CAM_DEBUG_FLAGS=(CAM_DEBUG_INFO|CAM_DEBUG_PROBE|CAM_DEBUG_PERIPH) # pass(4) device device pass options MMCCAM nodevice mmc nodevice mmcsd options IOMMU -------------------------------------------------------------------------------
Skipping earlier steps in the investigation, the below from the lists (slightly adjusted) indicates that the failures in my testing are tied to having a monitor connected . . . NEAR QUOTE Going back to my 19cca0b9613d based debug kernel build that has the printf's reporting the values used in the test, but with no monitor attached, it boots fine and reports: pmap_mapdev early_boot: akva_devmap_vaddr: ffff007ffffff000 size: 1000 pmap_mapdev early_boot: va: ffff007fffffe000 VM_MAX_KERNEL_ADDRESS: ffff008000000000 L2_SIZE: 200000 That compares to the previously reported failure figures from having the monitor attached for that debug kernel: pmap_mapdev early_boot: akva_devmap_vaddr: ffff007fff816000 size: 1000 pmap_mapdev early_boot: va: ffff007fff815000 VM_MAX_KERNEL_ADDRESS: ffff008000000000 L2_SIZE: 200000 panic: Too many early devmap mappings where the code does: KASSERT(va >= VM_MAX_KERNEL_ADDRESS - L2_SIZE, ("Too many early devmap mappings")); Looks like akva_devmap_vaddr gets smaller to make room in high kernel memory for monitor related data and so va can end up being too small by the criteria of this test. I've no clue who would be appropriate for dealing with this. END NEAR QUOTE
(In reply to Mark Millard from comment #3) Turns out that this "too much high kernel memory in use" issue happens for a combination of 2 things being true at the same time: A) Monitor attached (sufficiently large pixel count?) B) GDB enabled, per bbfa199cbc16 . A git bisect with a 1029x1080 monitor attach ended up with: # git bisect good bbfa199cbc1698631a0e932848e62dd76559d4d7 is the first bad commit commit bbfa199cbc1698631a0e932848e62dd76559d4d7 Author: mhorne <mhorne@FreeBSD.org> Date: Wed Dec 9 16:38:42 2020 -0400 arm64: gdb(4) machine-dependent bits Everything required for remote kernel debugging over a serial connection. For FDT-based systems, a debug port can be specified by setting hw.fdt.dbgport to the desired device tree node in loader.conf. For example, hw.fdt.dbgport="uart1", or hw.fdt.dbgport="serial@ff1a0000". Looks good: emaste Tested by: rwatson MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D27727 sys/arm64/arm64/gdb_machdep.c | 112 ++++++++++++++++++++++++++++++++++++++++ sys/arm64/conf/GENERIC | 2 +- sys/arm64/include/gdb_machdep.h | 81 +++++++++++++++++++++++++++++ sys/conf/files.arm64 | 1 + 4 files changed, 195 insertions(+), 1 deletion(-) create mode 100644 sys/arm64/arm64/gdb_machdep.c create mode 100644 sys/arm64/include/gdb_machdep.h That, in turn, involves: -#options GDB # Support remote GDB. +options GDB # Support remote GDB.
Created attachment 221535 [details] fix pmap_mapdev assert Hi Mark, Thank you describing how to reproduce the issue, I was able to trigger the same issue on my rockpro64 with the HDMI cable attached and GDB enabled. Please try the attached patch and let me know if it fixes the issue for you. Indeed it appears to be an interesting combination of factors. The crux of the issue is that the KASSERT in pmap_mapdev() assumes a static devmap size of 2MB, when really it has grown to 16MB to accommodate large things like framebuffers. You'll note that pmap_mapdev_attr() does not have this issue. In my case, the mapped size of the framebuffer (1920x1080 bytes) is slightly smaller than 2MB, meaning it won't trigger the KASSERT if it is the first thing added to the early devmap. The addition of GDB means the uart gets an entry in the static as well, pushing the framebuffer va just below that assertion threshold. Of course, removing the HDMI means the framebuffer isn't mapped, so the issue isn't triggered without it.
Patch works for me on pine64 with HDMI connected too. Thanks as I probably was the one who forgot to change this when I bumped the early mapping size for efifb.
(In reply to Mitchell Horne from comment #5) The patch worked fine on the 8 GiByte RPi4B: booted like normal and the monitor is displaying the console like it should. Details for the test: # uname -apKU FreeBSD RPi4B 13.0-CURRENT FreeBSD 13.0-CURRENT #8 pure-src-c255871-g1a816c756003-dirty: Wed Jan 13 11:02:21 PST 2021 root@FBSDFHUGE:/usr/obj/aarch64dbg_default_config_clang/arm64.aarch64/usr/fbsd/pure-src/arm64.aarch64/sys/GENERIC arm64 aarch64 1300134 1300134 # ~/fbsd-based-on-what-freebsd-main.sh pure-src 1a816c75600335c4482070df055add6f96885d54 CommitDate: 2021-01-11 11:16:42 -0800 # git diff diff --git a/sys/kern/subr_devmap.c b/sys/kern/subr_devmap.c index 581e85086f0f..8e07199b7f73 100644 --- a/sys/kern/subr_devmap.c +++ b/sys/kern/subr_devmap.c @@ -275,7 +275,7 @@ pmap_mapdev(vm_offset_t pa, vm_size_t size) if (early_boot) { akva_devmap_vaddr = trunc_page(akva_devmap_vaddr - size); va = akva_devmap_vaddr; - KASSERT(va >= VM_MAX_KERNEL_ADDRESS - L2_SIZE, + KASSERT(va >= VM_MAX_KERNEL_ADDRESS - PMAP_MAPDEV_EARLY_SIZE, ("Too many early devmap mappings")); } else #endif And my "1029x1080" was a typing problem: really 1920x1080.
This should be fixed by the following commit. Unfortunately, I incorrectly tagged the PR number. Thanks everyone for your reports and testing. commit 818390ce0ca539300dd15d7a817784f1e3f7a9b8 Author: Mitchell Horne <mhorne@FreeBSD.org> AuthorDate: 2021-01-13 18:30:50 +0000 Commit: Mitchell Horne <mhorne@FreeBSD.org> CommitDate: 2021-01-13 21:27:44 +0000 arm64: fix early devmap assertion The purpose of this KASSERT is to ensure that we do not run out of space in the early devmap. However, the devmap grew beyond its initial size of 2MB in r336519, and this assertion did not grow with it. A devmap mapping of a 1080p framebuffer requires 1920x1080 bytes, or 1.977 MB, so it is just barely able to fit without triggering the assertion, provided no other devices are mapped before it. With the addition of `options GDB` in GENERIC by bbfa199cbc16, the uart is now mapped for the purposes of a debug port, before mapping the framebuffer. The presence of both these conditions pushes the selected virtual address just below the threshold, triggering the assertion. To fix this, use the correct size of the devmap, defined by PMAP_MAPDEV_EARLY_SIZE. Since this code is shared with RISC-V, define it for that platform as well (although it is a different size). PR: 25241 Reported by: gbe MFC after: 3 days Sponsored by: The FreeBSD Foundation
@mhorne, thanks for the quick bugfix!