Bug 160790

Summary: [fusefs] [panic] VPUTX: negative ref count with FUSE
Product: Base System Reporter: landsidel.allen
Component: kernAssignee: freebsd-fs mailing list <fs>
Status: Closed Not Enough Information    
Severity: Affects Only Me CC: asomers, cem
Priority: Normal    
Version: 8.2-STABLE   
Hardware: Any   
OS: Any   

Description landsidel.allen 2011-09-17 21:30:10 UTC
Unexpected panic from nowhere.  Had been storing files with MooseFS at 10-20 mbit/sec for several hours when all connections to machine were lost.  A visit to the console showed a panic.

System did not automatically reboot after the panic, hard reset was required.

May be related to / dupe of http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/156797 though ZFS was not involved.

Full panic is:

vputx: negative ref count
0xffffff01573bf1d8: tag fuse, type VREC
    usecount 0, writecount 0, refcount 0 mountedhere 0
    flags (VI_FREE)
  VI_LOCKed  lock type fuse: UNLOCKED
nodeid: 0, parent_nid: 38125, fh_counter: 0, nlookup: 1, flags: 0
panic: vputx: negative ref cnt
cpuid = 3
KDB: stack backtrace:
#0 0xffffffff805fdda0 at kdb_backtrace+0x60
#1 0xffffffff805cbbd4 at panic_0x1b4
#2 0xffffffff8065e211 at vputx+0x101
#3 0xffffffff8065e40e at vput+0xe
#4 0xffffffff80660c50 at kern_statat_vnhook+0x100
#5 0xffffffff80660d95 at kern_statat+0x15
#6 0xffffffff80660dbc at kern_lstat+0x1c
#7 0xffffffff80660e5d at lstat+0x2d
#8 0xffffffff8060abde at syscallenter+0x2fe
#9 0xffffffff808bd961 at syscall+0x41
#10 0xffffffff808a5ed2 at Xfast_syscall+0xe2
Uptime: 2h47m36s
Cannot dump. Device not defined or unavailable.
Automatic reboot in 15 seconds - press a key on the console to abort

How-To-Repeat: Exact steps are unknown.  The system had been up running apache and MooseFS (a FUSE based distributed filesystem) for several hours before the panic.  Apache only used to serve images via the mounted FUSE filesystem.
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2011-09-18 03:46:03 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-fs

Over to maintainer(s).
Comment 2 landsidel.allen 2011-09-18 12:40:36 UTC
The crash is repeatable though the exact steps are unknown.  Three 
identical crashes within 8-9 hours, running the same workload.
Comment 3 landsidel.allen 2012-05-31 20:30:47 UTC
This bug continues to occur on a new system, different 'hardware'.  The 
system is running MooseFS and was migrated some time ago to an VMWare 
ESX 5 VM, 64bit.

OS version is: 8.2-STABLE FreeBSD 8.2-STABLE #0: Fri Oct 14 00:11:10 UTC 
2011     
root@freebsd-82-64.master.concord.internal:/usr/obj/usr/src/sys/GENERIC  
amd64

My initial suspicion is there is some kind of race involved, as the 
error only seems to occur during times of heavy write load.

The virtualized system does reboot on the panic however, though the 
physical machine the initial bug report regards did not.

Panic:
panic: vputx: negative ref cnt
cpuid = 1
KDB: stack backtrace:
#0 0xffffffff80607050 at kdb_backtrace+0x60
#1 0xffffffff805d4e74 at panic+0x1b4
#2 0xffffffff80666e31 at vputx+0x101
#3 0xffffffff8066702e at vput+0xe
#4 0xffffffff8067206c at vn_open_cred+0x56c
#5 0xffffffff8067210c at vn_open+0x1c
#6 0xffffffff8066fc03 at kern_openat+0x163
#7 0xffffffff8066ff89 at kern_open+0x19
#8 0xffffffff8066ffa8 at open+0x18
#9 0xffffffff808c6301 at amd64_syscall+0x301
#10 0xffffffff808aee0c at Xfast_syscall+0xfc
panic: bufwrite: buffer is not busy???
Comment 4 landsidel.allen 2013-06-26 21:08:39 UTC
On RELENG_9 (9.1 stable), the crash is near instant now, and does not 
take hours to manifest.  With a MooseFS FUSE filesystem mounted, simply 
deleting a single top level directory results in an instant kernel panic.

Compiled with a debug kernel so I could catch it without having to be 
watching 24/7, I get a panic:

Fatal trap 9: general protection fault while in kernel mode
cpuid = 1; apic id = 01
...
current process = 904 (rm)
[ thread pid 904 tid 100074 ]
Stopped at M_FUSEFH: movb 0xffffffffff8182a5,%al

Screenshots of the panic and backtrace here:

http://i.imgur.com/LHwpdas.png
http://i.imgur.com/5fpNd7E.png
http://i.imgur.com/8O2yfV7.png
Comment 5 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 08:00:27 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped
Comment 6 Conrad Meyer freebsd_committer 2018-08-10 08:51:04 UTC
(In reply to landsidel.allen from comment #4)
Looks similar to bug 186645 (also on 9.x).
Comment 7 Pedro F. Giffuni freebsd_committer 2019-03-04 23:15:37 UTC
Assign to fs (again)
Comment 8 Conrad Meyer freebsd_committer 2019-03-04 23:36:43 UTC
Is this reproducible on head or recent FreeBSD?  Can you provide a list of steps to encounter the panic from a vanilla freebsd installation?