Simple test of pristine file system causes panic: bad directory entry: rec_len is smaller than minimal offset=2040, inode=0, rec_len=8, name_len=0 panic: ext2_dirbad: /mnt: bad dir ino 32787 at offset 6136: mangled entry cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper(c119dc1c,c1000a79,c91d7000,edbc24c8,c0957e01,...) at db_trace_self_wrapper+0x2d/frame 0xedbc2498 kdb_backtrace(c135df3b,0,cb2dbd2f,edbc2554,edbc2600,...) at kdb_backtrace+0x30/frame 0xedbc2500 vpanic(edbc2554,edbc2584,cb2cedc0,cb2dbd2f,c9f5e7c0,...) at vpanic+0x11d/frame 0xedbc253c panic(cb2dbd2f,c9f5e7c0,8013,17f8,cb2dbd21,...) at panic+0x12/frame 0xedbc2548 ext2_dirbad(c8e72700,17f8,cb2dbd21,e8b3c000,edbc2664,...) at ext2_dirbad+0x70/frame 0xedbc2584 ext2_search_dirblock(c8e72700,e8b3c000,edbc2664,c8e09000,c,...) at ext2_search_dirblock+0x104/frame 0xedbc2600 ext2_htree_lookup(c8e72700,c8e09000,c,edbc2838,edbc2830,...) at ext2_htree_lookup+0x2db/frame 0xedbc26e8 ext2_lookup_ino(caf1fb40,edbc2bb8,edbc2bcc,0,0,...) at ext2_lookup_ino+0x24b/frame 0xedbc2864 ext2_lookup(edbc28e0,c1366687,c1448a30,caf1fb40,edbc28ac,...) at ext2_lookup+0x3f/frame 0xedbc2888 VOP_CACHEDLOOKUP_APV(cb2dd8b8,edbc28e0,edbc2bcc,0,0,...) at VOP_CACHEDLOOKUP_APV+0x12f/frame 0xedbc28b8 vfs_cache_lookup(edbc2970,c1366626,c11ce15d,c11aa383,c1448a30,...) at vfs_cache_lookup+0xf4/frame 0xedbc2908 VOP_LOOKUP_APV(cb2dd8b8,edbc2970,edbc2bcc,205,8,...) at VOP_LOOKUP_APV+0x12f/frame 0xedbc2938 lookup(edbc2b70,c11aa383,10c,cd,c0b70d91,...) at lookup+0x514/frame 0xedbc2998 namei(edbc2b70,c1fa0700,da,edbc2a74,c0b04ba0,...) at namei+0x4fd/frame 0xedbc2a30 vn_open_cred(edbc2b70,edbc2bfc,1b0,0,c8eed880,c8eabee0) at vn_open_cred+0xb2/frame 0xedbc2b00 vn_open(edbc2b70,edbc2bfc,1b0,c8eabee0,bfbfe798,...) at vn_open+0x3d/frame 0xedbc2b28 kern_openat(c91d7000,ffffff9c,bfbfe798,0,601,1b0) at kern_openat+0x310/frame 0xedbc2c1c sys_open(c91d7000,edbc2cc8,c1030f7f,e4734c90,0,...) at sys_open+0x39/frame 0xedbc2c40 syscall(edbc2d08) at syscall+0x31b/frame 0xedbc2cfc Details @ http://people.freebsd.org/~pho/stress/log/ext2fs-2.txt
Over to maintainers.
I think it may be related to the new htree implementation. It would be good to confirm this by disabling the directory indexing (with e2tunefs from the e2fsprogs port), and re-running the test. Thanks for the detailed report, BTW.
Ugh.. I meant to copy Zheng Liu (lz@freebsd) but bugzilla doesn't know about him :(.
(In reply to Pedro F. Giffuni from comment #2) > I think it may be related to the new htree implementation. > > It would be good to confirm this by disabling the directory indexing (with > e2tunefs from the e2fsprogs port), and re-running the test. > > Thanks for the detailed report, BTW. You are welcome. Indeed, tune2fs -O ^dir_index /dev/md... make the test run without a panic.
Hi again Peter; FWIW, Coverity found an issue that was fixed recently (r275645) and already MFC'd. It would be great if you can verify if the problem is still there.
No change: FreeBSD t2.osted.lan 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r278708: Fri Feb 13 22:06:30 CET 2015 pho@t2.osted.lan:/usr/src/sys/amd64/compile/PHO amd64 root@t2:~pho/stress2/misc # ./ext2fs.sh mke2fs 1.42.10 (18-May-2014) run: run time 0+00:10:00, incarnations 1, load 100, verbose 1 22:26:49 Loop #1 lockf2: run time 0+00:02:00, incarnations 19, load 80, verbose 1 symlink: run time 0+00:02:00, incarnations 1, load 80, verbose 1 rw: run time 0+00:02:00, incarnations 1, load 80, verbose 1 fts: run time 0+00:02:00, incarnations 10, load 80, verbose 1 link: run time 0+00:02:00, incarnations 13, load 80, verbose 1 openat: run time 0+00:02:00, incarnations 1, load 80, verbose 1 lockf: run time 0+00:02:00, incarnations 1, load 80, verbose 1 rename: run time 0+00:02:00, incarnations 17, load 80, verbose 1 mkfifo: run time 0+00:02:00, incarnations 17, load 80, verbose 1 creat: run time 0+00:02:00, incarnations 15, load 80, verbose 1 mkdir: run time 0+00:02:00, incarnations 17, load 80, verbose 1 bad directory entry: rec_len is smaller than minimal offset=2040, inode=0, rec_len=8, name_len=0 panic: ext2_dirbad: /mnt: bad dir ino 49187 at offset 6136: mangled entry cpuid = 17 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe081e1a8130 vpanic() at vpanic+0x189/frame 0xfffffe081e1a81b0 panic() at panic+0x43/frame 0xfffffe081e1a8210 ext2_search_dirblock() at ext2_search_dirblock+0x2a3/frame 0xfffffe081e1a8290 ext2_htree_lookup() at ext2_htree_lookup+0x162/frame 0xfffffe081e1a8390 ext2_lookup_ino() at ext2_lookup_ino+0x20c/frame 0xfffffe081e1a84b0 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x10f/frame 0xfffffe081e1a84e0 vfs_cache_lookup() at vfs_cache_lookup+0xd6/frame 0xfffffe081e1a8540 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x10f/frame 0xfffffe081e1a8570 lookup() at lookup+0x5d5/frame 0xfffffe081e1a8600 namei() at namei+0x536/frame 0xfffffe081e1a86c0 vn_open_cred() at vn_open_cred+0xd5/frame 0xfffffe081e1a8820 kern_openat() at kern_openat+0x257/frame 0xfffffe081e1a89a0 amd64_syscall() at amd64_syscall+0x29c/frame 0xfffffe081e1a8ab0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe081e1a8ab0 --- syscall (499, FreeBSD ELF64, sys_openat), rip = 0x800b08fba, rsp = 0x7fffffffe5b8, rbp = 0x7fffffffe690 --- KDB: enter: panic [ thread pid 970 tid 100491 ] Stopped at kdb_enter+0x3e: movq $0,kdb_why db>
Grab PR: I got tired of dir_index issues and will be removing the feature altogether.
A commit references this bug: Author: pfg Date: Fri Apr 17 22:26:03 UTC 2015 New revision: 281670 URL: https://svnweb.freebsd.org/changeset/base/281670 Log: Drop experimental dir_index support. The htree directory index is a highly desirable feature for research purposes and was meant to improve performance in our ext2/3 driver. Unfortunately our implementation has two problems: - It never really delivered any performance improvement. - It appears to corrupt the filesystem in undetermined circumstances. Strictly speaking dir_index is not required for read/write support in ext2/3 and our limited ext4 support still works fine without it. Regain stability in the ext2 driver by removing it. We may need it back (fixed) if we want to support encrypted ext4 support but thanks to the wonders of version control we can always revert this change and bring it back. PR: 191895 PR: 198731 PR: 199309 MFC after: 5 days Changes: head/sys/fs/ext2fs/ext2_dir.h head/sys/fs/ext2fs/ext2_extern.h head/sys/fs/ext2fs/ext2_hash.c head/sys/fs/ext2fs/ext2_htree.c head/sys/fs/ext2fs/ext2_lookup.c head/sys/fs/ext2fs/ext2_vfsops.c head/sys/fs/ext2fs/ext2fs.h head/sys/modules/ext2fs/Makefile
A commit references this bug: Author: pfg Date: Wed Apr 22 00:38:14 UTC 2015 New revision: 281841 URL: https://svnweb.freebsd.org/changeset/base/281841 Log: MFC r281670, r281703: Drop experimental ext2fs dir_index support. The htree directory index is a highly desirable feature for research purposes and was meant to improve performance in our ext2/3 driver. Unfortunately our implementation has two problems: - It never really delivered any performance improvement. - It appears to corrupt the filesystem in undetermined circumstances. Strictly speaking dir_index is not required for read/write support in ext2/3 and our limited ext4 support still works fine without it. Regain stability in the ext2 driver by removing it. We may need it back (fixed) if we want to support encrypted ext4 support but thanks to the wonders of version control we can always revert this change and bring it back. PR: 191895 PR: 198731 PR: 199309 Changes: _U stable/10/ stable/10/sys/conf/files stable/10/sys/fs/ext2fs/ext2_dir.h stable/10/sys/fs/ext2fs/ext2_extern.h stable/10/sys/fs/ext2fs/ext2_hash.c stable/10/sys/fs/ext2fs/ext2_htree.c stable/10/sys/fs/ext2fs/ext2_lookup.c stable/10/sys/fs/ext2fs/ext2_vfsops.c stable/10/sys/fs/ext2fs/ext2fs.h stable/10/sys/modules/ext2fs/Makefile
A commit references this bug: Author: pfg Date: Wed Apr 22 00:40:44 UTC 2015 New revision: 281842 URL: https://svnweb.freebsd.org/changeset/base/281842 Log: MFC r281670, r281703: Drop experimental ext2fs dir_index support. The htree directory index is a highly desirable feature for research purposes and was meant to improve performance in our ext2/3 driver. Unfortunately our implementation has two problems: - It never really delivered any performance improvement. - It appears to corrupt the filesystem in undetermined circumstances. Strictly speaking dir_index is not required for read/write support in ext2/3 and our limited ext4 support still works fine without it. Regain stability in the ext2 driver by removing it. We may need it back (fixed) if we want to support encrypted ext4 support but thanks to the wonders of version control we can always revert this change and bring it back. PR: 191895 PR: 198731 PR: 199309 Changes: _U stable/9/sys/ _U stable/9/sys/conf/ stable/9/sys/conf/files _U stable/9/sys/fs/ stable/9/sys/fs/ext2fs/ext2_dir.h stable/9/sys/fs/ext2fs/ext2_extern.h stable/9/sys/fs/ext2fs/ext2_hash.c stable/9/sys/fs/ext2fs/ext2_htree.c stable/9/sys/fs/ext2fs/ext2_lookup.c stable/9/sys/fs/ext2fs/ext2_vfsops.c stable/9/sys/fs/ext2fs/ext2fs.h _U stable/9/sys/modules/ stable/9/sys/modules/ext2fs/Makefile
Fixed (by brute force) but thank you!