Bug 18572

Summary: /var/run as mount_mfs w/ soft-updates
Product: Base System Reporter: Hitoshi Matsunawa <matsunaw>
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: 4.0-RELEASE   
Hardware: Any   
OS: Any   

Description Hitoshi Matsunawa 2000-05-15 19:20:00 UTC
If ``halt''-ed immediately after creating some number of dirty buffer
(may required spread across several file systems), system will fail to
shutdown.

w/ ``sysctl -w debug.busyprt=1'', following messages reported on console:

---->
vflush: busy vnode: 0xc8770e00: type VREG, usecount 1, writecount 0, refcount 1, flag (VOBJBUF)
	tag VT_UFS, ino 4, on dev MFS0(253,0)

[in non-deadlock-ed case, two more busy vnode (type of VSOCK) are reported]

message to wait for ``bufdaemon''
message to wait for ``syncer''

syncing disks... 2 2 2 2 2 2 ....

1. dev:MFS0, flags: 00100000	blkno: 48, lblkno: 48
2. dev:MFS0, flags: 01020024	blkno: 64, lblkno: 64
<----

This seems to be caused by ``rundown'' race between mfs process and
other daemons who use "/var/run", and resulted in deadlock.  kernel
DDB shows named waiting on "biord" while mfs/syslogd on "inode".

If soft-updates is disabled, no problems occur (in my case).  But
soft-updates may only makes the problem easy to happen.


I tried several way to figure out what's going on and/or find out a
work-around:

(1) in mfs_start() (of src/ufs/mfs/mfs_vfops.c), ignoring signals except
    SIGKILL. 
(2) in mfs_start(), exucutes dounmount() in other execution vehicle
    (using kthread_create()).
(3) in biowait() (of src/kern/vfs_bio.c), add a time-out parameter on
    tsleep()-call and return EINTER if timed-out.

all of above seems solve the problem for me, but:

(1) user visible changes.
(2) kthread_create() seems not appropriate for this kind purpose.
(3) tried just for curiosity and to support kernel DDB's idea of "who's
    waiting on where".

There may be more choices and trade-offs.

Fix: 

I only found a few work around (described in [>Description:]).

I think fundamental problem is that once mfs process enters into
dounmount(), mfs i/o won't be served for other processes (other than
mfs process itself). I couldn't find how it's related w/
soft-updates. It's may all about timing but may not.
How-To-Repeat: 
It's depend on timing. In my case:

  % cd $INDEX_of_NAMAZU_as_WEB_SEARCH_ENGINE
  % rm *
  % sync; sync; sync
  % su
  # cd /usr/src/sys/compile/$IDENT
  # mv kernel /kernel.00
  % mknmz -a ~/src/linux-2.3.51/Documentation/
  % super halt

where, LANG=ja_JP.EUC, perl (mknmz is written in perl) used is bundled
one.
Comment 1 Poul-Henning Kamp freebsd_committer freebsd_triage 2001-05-29 19:23:19 UTC
State Changed
From-To: open->closed

MFS is deprecated