Bug 182181 - [ufs] Leakage of vnode references (race condition?)
Summary: [ufs] Leakage of vnode references (race condition?)
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 8.4-RELEASE
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-fs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-09-17 08:30 UTC by Alexey Markov
Modified: 2013-09-19 09:11 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexey Markov 2013-09-17 08:30:00 UTC
While running pgbench PostgreSQL benchmark I suddenly found a vnode leakage which leads in particular to an inability to unmount partition.

Fix: 

It seems a very similar bug was recently fixed in the HEAD and 9-STABLE: http://svnweb.freebsd.org/changeset/base/253998

I will try to apply that fix to my 8-RELEASE server and run a testing script again, but it will take a time.
How-To-Repeat: Install a releng/8.4 branch, PostgreSQL 9.1 from ports, set "wal_level = hot_standby" and "wal_keep_segments = 32" in the postgresql.conf, make a dedicated partition for testing and run a following script:

#!/bin/sh -e

i=0

while (true); do
  i=$(($i+1))
  echo Iteration: ${i}

  # Format and mount a pristine filesystem
  newfs -n /dev/da0s1f > /dev/null
  mount /test

  # Prepare an empty instance
  tar xjf /root/pgsql.tbz -C /test

  # Start the PostgreSQL
  /usr/local/etc/rc.d/postgresql start

  # Run a pgbench
  date
  su - pgsql -c "pgbench -i -s 1000 pgbench 2> /dev/null"
  date
  df -i /test
  du -xs /test

  # Stop the PostgreSQL
  /usr/local/etc/rc.d/postgresql stop

  # Try to unmount the filesystem
  umount /test
done

Result is:

ku-old:home/support# /root/2.sh
Iteration: 1
вторник, 17 сентября 2013 г. 09:33:54 (MSK)
вторник, 17 сентября 2013 г. 10:44:34 (MSK)
Filesystem  1K-blocks     Used    Avail Capacity iused   ifree %iused  Mounted on
/dev/da0s1f  56616608 15951754 36135526    31%    1053 7323617    0%   /test
15951742        /test
umount: unmount of /test failed: Device busy

ku-old:home/support# rm -rf /test/*
ku-old:home/support# df -i /test; du -xs /test
Filesystem  1K-blocks Used    Avail Capacity iused   ifree %iused  Mounted on
/dev/da0s1f  56616608   14 52087266     0%       2 7324668    0%   /test
2       /test

As you can see, df shows 12 KiB (6 fragments) more than du. After force unmount and remount both shows 2 KiB as expected.
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2013-09-18 03:26:00 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-fs

Over to maintainer(s).
Comment 2 Alexey Markov 2013-09-18 10:41:56 UTC
After applying fix from revision 253998 to the 9-RELEASE src
test script works without errors and vnode leakage is gone.

-- 
WBR, Alexey Markov.
Comment 3 devin.teske 2013-09-18 20:57:52 UTC
When you say you applied r253998 to 8-RELEASE, which branch?
I see that the equivalent one-line change introduced by r253998 has
been in stable/8 for 2 years 4 months (meaning 8.3-R and later have
the change you're talking about).

Could you clarify please?
--=20
Devin

_____________
The information contained in this message is proprietary and/or confidentia=
l. If you are not the intended recipient, please: (i) delete the message an=
d all copies; (ii) do not disclose, distribute or use the message in any ma=
nner; and (iii) notify the sender immediately. In addition, please be aware=
 that any message addressed to our domain is subject to archiving and revie=
w by persons other than the intended recipient. Thank you.
Comment 4 mckusick 2013-09-18 23:07:41 UTC
> Date: Wed, 18 Sep 2013 20:00:01 GMT
> To: freebsd-fs@freebsd.org
> From: "Teske, Devin" <Devin.Teske@fisglobal.com>
> Subject: Re: kern/182181: [ufs] Leakage of vnode references (race condition?)
> 
> The following reply was made to PR kern/182181; it has been noted by GNATS.
> 
> From: "Teske, Devin" <Devin.Teske@fisglobal.com>
> To: "bug-followup@FreeBSD.org" <bug-followup@FreeBSD.org>,
>         "redrat@mail.ru"
> 	<redrat@mail.ru>
> Cc: "Teske, Devin" <Devin.Teske@fisglobal.com>
> Subject: Re: kern/182181: [ufs] Leakage of vnode references (race condition?)
> Date: Wed, 18 Sep 2013 19:57:52 +0000
> 
>  When you say you applied r253998 to 8-RELEASE, which branch?
>  I see that the equivalent one-line change introduced by r253998 has
>  been in stable/8 for 2 years 4 months (meaning 8.3-R and later have
>  the change you're talking about).
> 
> Could you clarify please?
> --
> Devin

My fat-fingered mistake. I should have said that it was MFC'ed to
9-RELEASE. Specifically -r255104 was the MFC from current to 9-stable
and -r255231 was MFS to releng9.2 so that it will be in the 9.2 release.
Sorry for the confusion.

	Kirk McKusick
Comment 6 mckusick 2013-09-19 01:24:27 UTC
> From: "Teske, Devin" <Devin.Teske@fisglobal.com>
> To: Rick Macklem <rmacklem@uoguelph.ca>
> Subject: Re: kern/182181: [ufs] Leakage of vnode references (race condition?)
> Date: Thu, 19 Sep 2013 00:11:03 +0000
> Cc: freebsd-fs <freebsd-fs@freebsd.org>,
>         "Teske, Devin" <Devin.Teske@fisglobal.com>
> 
> On Sep 18, 2013, at 5:05 PM, Rick Macklem wrote:
> 
>> Devin Teske wrote:
>>> 
>>> 
>>> Yes, my confusion was...
>>> 
>>> 1. The PR headers say 8.4-RELEASE-p3 is affected
>>> 
>>> 2. The PR's "How-To-Repeat" starts with "Install a releng/8.4 branch"
>>> 
>>> Yet...
>>> 
>>> releng/8.4 and even releng/8.3 both use VOP_UNLOCK instead of vput
>>> (read: are patched).
>>> 
>> Did you mean "not patched"? The patched version in head has vput()
>> and the unpatched versions have VOP_UNLOCK(), if I read the coed correctly.
>> 
> 
> Well, Kirk's fat-finger made me think that VOP_UNLOCK was the patched-
> state and vput was the unpatched state. (could also be that I'm fighting
> the flu currently).
> 
> So everything is copacetic now, except the one outstanding question...
> 
> Should we not MFC r253998 to stable/8?
> 
> I'm looking to pull this into our own stable/8 kernel, but would
> like to do it by way of svn merge from the stable/8 branch.
> -- 
> Devin

Per your request, I have MFC'ed the patch to 8-stable as revision 255681.

	Kirk McKusick
Comment 7 Alexey Markov 2013-09-19 06:44:22 UTC
I am sorry for my misprint, of course it was 8-RELEASE,
not a 9-RELEASE! And I tested a fix on my 8-RELEASE-p4
server. Successfully.

Now that fix was MFCed to 8-STABLE, I think this PR may
be closed.

-- 
WBR, Alexey Markov.
Comment 8 Sergey Kandaurov freebsd_committer freebsd_triage 2013-09-19 09:09:25 UTC
State Changed
From-To: open->closed

Close per submitter request. 
The fix was MFCed to 8-STABLE.