Bug 164472 - [ufs] fsck -B panics on particular data inconsistency
Summary: [ufs] fsck -B panics on particular data inconsistency
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 8.2-RELEASE
Hardware: Any Any
: Normal Affects Only Me
Assignee: Kirk McKusick
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-01-25 09:00 UTC by Eugene M. Zheganin
Modified: 2018-11-06 14:22 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Eugene M. Zheganin 2012-01-25 09:00:29 UTC
fsck -B panics on particular inconsistencies.
I got one machine that sometimes locks up (probably die to some other bug), and after reset when it runs fsck -B it leads to panic.

Since this happens quite often last time I created an image on a partition that makes FreeBSD panic.

Unfortunately this machine doesn't run a debug kernel, and, due to a bug when FreeBSD reboots immidiately upon a key press from a screen where it says 'Automatic reboot in 15 seconds, press any key to abort' (which is still unreported for some reason, while lots of people confirm its existance) I was (and I am) unable to capture a panic screen.

But, since I have an image, everyone can easily reproduce this panic.
This is 100% reproduceable, at least I've done it 5 times on a test machine and got a panic each time. Unfortunately, this machine is too old to build debug kernel in some reasonable amount of time (I really think anyone will download this image faster than I will build a debug kernel).

So... here comes the image in case someone is interested.

Attention, FreeBSD panics only when fsck is run with -B. Ordinary fsck run doesn't panic and is able to successfully resolve all the filesystem errors.

Fix: 

Run fsck without -B.
How-To-Repeat: Get an image from http://tech.norma.perm.ru/files/var.dsk (sorry, this link is about a couple of megabits, my really broadband link is served by a server from my previous report with a buggy pf route-to/reply-to, so I'm using this old server). Mount it read-write (I didn't test it on an unmounted or read-only image). Like this:

mdconfig -a -t vnode -f var.dsk
mount /dev/md0 /mnt/panic

Run an fsck (since it's a partition image you need to manually specify the fsck of the type needed):

fsck_4.2bsd -B /dev/md0
<here it panics>
Comment 1 Eugene M. Zheganin 2012-01-25 09:21:48 UTC
P.S . I forgot to mention that this image size in 2048 megs, so I'm 
sorry if you cannot afford its downloading.
Comment 2 Eugene M. Zheganin 2012-01-25 09:48:16 UTC
Yeah, my bad, I compressed it with xz and redirected web-server to it. 
Now it's about 400 megs. Old URL still works.
Comment 3 Mikhail Timofeev 2012-01-25 12:56:57 UTC
Confirm, bug is reproduced on FreeBSD 9.0-STABLE #2 r230483: Tue Jan
24 00:48:28 NOVT 2012     root@bsd.loc:/usr/obj/usr/src/sys/GENERIC
i386
Comment 4 Mark Linimon freebsd_committer freebsd_triage 2012-01-30 04:13:00 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-fs

reclassify.
Comment 5 Kostik Belousov 2012-01-30 05:30:04 UTC
You failed to mention which panic you got. Was it 'dup alloc' ? A
backtace would be also useful.

If it was indeed 'dup alloc', then there is nothing fsck or snapshots
can be accused for. Your filesystem is in inconsistent state, which
requires full fsck to recover. It must be not mounted while not
repaired.

Somewhat more interesting is how the fs got into this state.
Comment 6 emz 2012-01-30 07:50:35 UTC
This state can be achieved (and sometimes is achieved) after a server 
hangup and/or reset.
I cannot agree that using fsck -B should lead to panic, because there is 
no way to distinguish filesystem between the state where it can be cured 
with fsck -B and where it can not. After all, this is what the bgfsck is 
for.
Comment 7 mckusick 2012-02-01 06:26:43 UTC
> From: Kostik Belousov <kostikbel@gmail.com>
> To: bug-followup@FreeBSD.org, eugene@zhegan.in
> Cc:  
> Subject: Re: kern/164472: [ufs] fsck -B panics on particular data inconsistency
> Date: Mon, 30 Jan 2012 07:30:04 +0200
> 
>  You failed to mention which panic you got. Was it 'dup alloc' ? A
>  backtace would be also useful.
> 
>  If it was indeed 'dup alloc', then there is nothing fsck or snapshots
>  can be accused for. Your filesystem is in inconsistent state, which
>  requires full fsck to recover. It must be not mounted while not
>  repaired.
> 
>  Somewhat more interesting is how the fs got into this state.

Thanks for your report and in particular a small file image that
demonstrates the problem. I have been able to reproduce your panic
reliably on my test machine.

Running a normal fsck on the image does indeed show that the filesystem
has corruption that is unexpected on a filesystem running with soft
updates. So, in the end, if the background fsck were able to run, it
would fail and notify the system that it needed to be checked by a
full fsck. But as you have aptly demonstrated, the background fsck
crashes the system as it tries to take a snapshot of the filesystem
on which to run its check.

The cause of the crash is because in taking a snapshot, the filesystem
needs to allocate an inode for the snapshot. As it turns out, the
inode that it tries to allocate is marked free in the inode map, but
is in fact already allocated which leads to the panic.

I am still mulling over how to resolve this problem, but have not
yet come up with one. I am looking for a solution that effectively
will let the snapshot fail rather than crashing the system so that
the fsck -B can then gracefully fail and lead to the full fsck as
is needed in this case.

	Kirk McKusick
Comment 8 Eugene Grosbein freebsd_committer 2017-08-17 10:24:04 UTC
It seems Kirk had some thoughts on the problem.
Comment 9 emz 2017-08-18 09:27:18 UTC
The original image made available again on old URL (old host was retired and disassembled, but I managed to save the file), sorry it was inaccessible for some serious time.

http://tech.norma.perm.ru/files/var.dsk.xz 386M, xz-compressed, original URL still working too.
Comment 10 Eitan Adler freebsd_committer freebsd_triage 2018-05-28 19:50:02 UTC
batch change:

For bugs that match the following
-  Status Is In progress 
AND
- Untouched since 2018-01-01.
AND
- Affects Base System OR Documentation

DO:

Reset to open status.


Note:
I did a quick pass but if you are getting this email it might be worthwhile to double check to see if this bug ought to be closed.