New Panic:

Backtrace from crash tonight (after portsnap cron update):

# kgdb kernel.debug /var/crash/vmcore.1
kgdb: kvm_nlist(_stopped_cpus):
kgdb: kvm_nlist(_stoppcbs):
[GDB will not be able to debug user-mode threads:
/usr/lib/libthread_db.so: Unde                       fined symbol
"ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:
dev = ad4s1f, block = 26098960, fs = /usr
panic: ffs_blkfree: freeing free block
Uptime: 1d10h32m17s
Physical memory: 631 MB
Dumping 150 MB: 135 119 103 87 71 55 39 23 7

#0  doadump () at pcpu.h:195
195             __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) backtrace
#0  doadump () at pcpu.h:195
#1  0xc06328a3 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
#2  0xc0632aa4 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:563
#3  0xc07d4d10 in ffs_blkfree (ump=0xc323ab00, fs=0xc321c800,
    devvp=0xc328add0, bno=26098960, size=16384, inum=6500370)
    at /usr/src/sys/ufs/ffs/ffs_alloc.c:1893
#4  0xc07e8588 in indir_trunc (freeblks=0xc3e12500, dbn=103954304, level=0,
    lbn=12, countp=0xd9177c4c) at /usr/src/sys/ufs/ffs/ffs_softdep.c:2896
#5  0xc07e8850 in handle_workitem_freeblocks (freeblks=0xc3e12500, flags=0)
    at /usr/src/sys/ufs/ffs/ffs_softdep.c:2746
#6  0xc07ea0d8 in process_worklist_item (mp=0xc324b7d4, flags=Variable
"flags" is not available.
)
    at /usr/src/sys/ufs/ffs/ffs_softdep.c:963
#7  0xc07eb191 in softdep_process_worklist (mp=0xc324b7d4, full=0)
    at /usr/src/sys/ufs/ffs/ffs_softdep.c:847
#8  0xc07ed70a in softdep_flush () at /usr/src/sys/ufs/ffs/ffs_softdep.c:758
#9  0xc06121d1 in fork_exit (callout=0xc07ed290 <softdep_flush>, arg=0x0,
    frame=0xd9177d38) at /usr/src/sys/kern/kern_fork.c:796
#10 0xc0850170 in fork_trampoline () at
/usr/src/sys/i386/i386/exception.s:205
(kgdb)

# uname -a
FreeBSD peanuts.homenet.local 7.0-PRERELEASE FreeBSD 7.0-PRERELEASE
#121: Thu Oct 11 16:29:05 CEST 2007
rowi@peanuts.homenet.local:/usr/obj/usr/src/sys/PEANUTS  i386

Sources from Builddate.

I will try this patch:

<http://www.googlebit.com/freebsd/patches/ffs_softdep.c-patch>

-- 
Diese Email ist ungeeignet für seelisch labile Persönlichkeiten.

I can no longer reproduce it any more.


-- 
Diese Email ist ungeeignet für seelisch labile Persönlichkeiten.

Comment 3 Remko Lodder 2007-11-19 20:24:17 UTC

State Changed
From-To: open->closed

No longer reproducable (as per the submitter)

Hello Remko, hello Rolf,

remko@freebsd.org wrote:
 > State-Changed-From-To: open->closed
 > [...]
 > State-Changed-Why: 
 > No longer reproducable (as per the submitter)

I assume that the submitter meant that he can no longer
reproduce it after applying the patch that he mentioned
(for sys/ufs/ffs/ffs_softdep.c).

So I think this PR should stay open until that patch (or
a better one) is comitted.

By the way, I got the same panic on a RELENG_7 machine
two days ago (without the patch).

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758,  Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

"C++ is to C as Lung Cancer is to Lung."
        -- Thomas Funke

Comment 5 Remko Lodder 2007-11-28 15:11:10 UTC

State Changed
From-To: closed->open

This might not be fixed yet as per feedback from Oliver Fromme.

Oliver Fromme <olli@lurza.secnetix.de> wrote:

> I assume that the submitter meant that he can no longer
> reproduce it after applying the patch that he mentioned
> (for sys/ufs/ffs/ffs_softdep.c).
> 
> So I think this PR should stay open until that patch (or
> a better one) is comitted.

No, this patch do nothing, this was a try by Eric Anderson to get more
informations for debugging.

But after the commit to sys/vm/vm_object.c (revision 1.386
date: 2007/10/18 23:02:18) i have seen no more panics.


-- 
Diese Email ist ungeeignet für seelisch labile Persönlichkeiten.

Rolf Witt wrote:
 > Oliver Fromme wrote:
 > > I assume that the submitter meant that he can no longer
 > > reproduce it after applying the patch that he mentioned
 > > (for sys/ufs/ffs/ffs_softdep.c).
 > > 
 > > So I think this PR should stay open until that patch (or
 > > a better one) is comitted.
 >  
 > No, this patch do nothing, this was a try by Eric Anderson to get more
 > informations for debugging.

OK, thanks for clarifying.

 > But after the commit to sys/vm/vm_object.c (revision 1.386
 > date: 2007/10/18 23:02:18) i have seen no more panics.

Are you sure that patch was the culprit?  I cannot
reproduce the panic easily; it appeared only once
after 5 weeks uptime on our machine.  I notice this
machine still has rev. 1.385 (the last update was
just one day before the commit).  So maybe it's time
for the next update.

Thanks again for the information.

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758,  Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

"Documentation is like sex; when it's good, it's very, very good,
and when it's bad, it's better than nothing."
        -- Dick Brandon

I  can  confirm  that when panicking the automatic dump to swap and auto
reboot does work as advertised. ;-)

Kernel: FreeBSD 7.0-STABLE #43 r181188M: Sat Aug  2 19:31:50 CEST 2008

The  box  has  /etc  + /usr/local/etc + /home mounted through unionfs on
/mnt/flash.  Box  is an AMD LX800 with a 2 GB compact flash running soft
updates  and  noatime  to  lower write count. Nothing really special was
going  on  when  the panic occured, I tried to finish up an smbpasswd -a
for some tests which did not complete.

Regards,
Stephan

---

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:
dev = ad0s1g, block = 51945, fs = /mnt/flash
panic: ffs_blkfree: freeing free frag
cpuid = 0
KDB: stack backtrace:
db_trace_self_wrapper(c0be4f94) at db_trace_self_wrapper+0x25
kdb_backtrace(0,cd29163c,c23090d4,cae9,0,...) at kdb_backtrace+0x24
panic(c0c09d1b,c0c09cdb,c2348a78,cae9,0,...) at panic+0xf9
ffs_blkfree(c2364800,c2309000,c239878c,cae9,0,...) at ffs_blkfree+0x4e4
ffs_truncate(c2ba778c,200,0,880,c25fc200,c2590660) at ffs_truncate+0x13f2
ufs_direnter(c2ba778c,c26a5564,cd2918e0,cd291bb0,0,c298a980,cd291bb0,cd2918e0) at ufs_direnter+0x7cb
ufs_makeinode(cd291bb0,1e,cd291a34,cd291a1c,c0b38b6e,...) at ufs_makeinode+0x48a
ufs_create(cd291a34) at ufs_create+0x2a
VOP_CREATE_APV(c0d03b20,cd291a34) at VOP_CREATE_APV+0x3e
unionfs_create(cd291a9c) at unionfs_create+0x56
VOP_CREATE_APV(c0cd6900,cd291a9c,c0cd6900,cd291a9c,0,...) at VOP_CREATE_APV+0x3e
vn_open_cred(cd291b88,cd291c84,1a4,c25fc200,c237a1f8,...) at vn_open_cred+0x1c4
vn_open(cd291b88,cd291c84,1a4,c237a1f8,c2bfa600,...) at vn_open+0x1e
kern_open(c2590660,208270,0,20b,1b6,...) at kern_open+0xdb
open(c2590660,cd291cfc) at open+0x18
syscall(cd291d38) at syscall+0x2be
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (5, FreeBSD ELF32, open), eip = 0x2039ce47, esp = 0xbfbfde7c, ebp = 0xbfbfdeb8 ---
Uptime: 14h16m32s
Physical memory: 243 MB
Dumping 46 MB: 31 15

#0  doadump () at pcpu.h:195
195     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) backtrace
#0  doadump () at pcpu.h:195
#1  0xc07fcbdd in boot (howto=260) at /home/build/src/sys/kern/kern_shutdown.c:418
#2  0xc07fce8b in panic (fmt=Variable "fmt" is not available.
) at /home/build/src/sys/kern/kern_shutdown.c:572
#3  0xc0a59df4 in ffs_blkfree (ump=0xc2364800, fs=0xc2309000, devvp=0xc239878c, bno=51945, size=512, inum=3264)
    at /home/build/src/sys/ufs/ffs/ffs_alloc.c:1918
#4  0xc0a62972 in ffs_truncate (vp=0xc2ba778c, length=512, flags=Variable "flags" is not available.
) at /home/build/src/sys/ufs/ffs/ffs_inode.c:476
#5  0xc0a7f4fb in ufs_direnter (dvp=0xc2ba778c, tvp=0xc26a5564, dirp=0xcd2918e0, cnp=0xcd291bb0, newdirbp=0x0)
    at /home/build/src/sys/ufs/ufs/ufs_lookup.c:959
#6  0xc0a8491a in ufs_makeinode (mode=33188, dvp=0xc2ba778c, vpp=0xcd291a48, cnp=0xcd291bb0)
    at /home/build/src/sys/ufs/ufs/ufs_vnops.c:2403
#7  0xc0a84b8a in ufs_create (ap=0xcd291a34) at /home/build/src/sys/ufs/ufs/ufs_vnops.c:193
#8  0xc0b38b6e in VOP_CREATE_APV (vop=0xc0d03b20, a=0xcd291a34) at vnode_if.c:206
#9  0xc07a0d86 in unionfs_create (ap=0xcd291a9c) at vnode_if.h:112
#10 0xc0b38b6e in VOP_CREATE_APV (vop=0xc0cd6900, a=0xcd291a9c) at vnode_if.c:206
#11 0xc0875d74 in vn_open_cred (ndp=0xcd291b88, flagp=0xcd291c84, cmode=Variable "cmode" is not available.
) at vnode_if.h:112
#12 0xc08760de in vn_open (ndp=0xcd291b88, flagp=0xcd291c84, cmode=420, fp=0xc237a1f8) at /home/build/src/sys/kern/vfs_vnops.c:94
#13 0xc08742bb in kern_open (td=0xc2590660, path=0x208270 <Address 0x208270 out of bounds>, pathseg=UIO_USERSPACE, flags=523, 
    mode=438) at /home/build/src/sys/kern/vfs_syscalls.c:1028
#14 0xc0874758 in open (td=0xc2590660, uap=0xcd291cfc) at /home/build/src/sys/kern/vfs_syscalls.c:995
#15 0xc0b1844e in syscall (frame=0xcd291d38) at /home/build/src/sys/i386/i386/trap.c:1035
#16 0xc0b00840 in Xint0x80_syscall () at /home/build/src/sys/i386/i386/exception.s:196
#17 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb)

Comment 9 Mark Linimon 2009-05-18 04:07:17 UTC

Responsible Changed
From-To: freebsd-bugs->freebsd-fs

Over to maintainer(s).

Hello Rolf,

I just stumbled across this old PR.  If I'm not mistaken,
this particular problem was fixed long ago.  It certainly
didn't happen anymore to me after rev. 1.386 of
sys/vm/vm_object.c, almost 3 years ago.

Do you agree that the PR can be closed?

Best regards
   Oliver

http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/116913

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758,  Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

"C++ is over-complicated nonsense. And Bjorn Shoestrap's book
a danger to public health. I tried reading it once, I was in
recovery for months."
        -- Cliff Sarginson

Comment 11 Oliver Fromme 2010-08-04 16:40:10 UTC

State Changed
From-To: open->closed

Originator agrees that the problem is fixed.