Bug 120858

Summary: [patch] [cam] panic: ufs_dirbad with CLARiiON CX3-40
Product: Base System Reporter: Oleg Sharoiko <os>
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed Overcome By Events    
Severity: Affects Only Me CC: os, rew
Priority: Normal    
Version: Unspecified   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
file.diff
none
clariion.diff none

Description Oleg Sharoiko 2008-02-19 22:00:04 UTC
	Had a chance to play with FreeBSD on IBM BladeCenter H and
EMC CLARiiON CX3-40. Moderate I/O activity on ufs filesystem gave
me a reproducable ufs_dirbad panic and corrupted root directory of
filesystem.

Fix: Lowering queue depth down to 63 (with camcontrol tags da0 -N 63)
fixes this issue. I did several successfull make -9 buildworld and 
make -j buildkerel. With queue depth 64 I still got panics. Here is a patch
which sets maxtags to 63 for volumes on CLARiiON.

	The LUNs from CLARiON are identified by camcontrol devlist
as follows:

<DGC RAID 5 0324>                  at scbus0 target 0 lun 0 (pass0,da0)
<DGC RAID 5 0324>                  at scbus0 target 1 lun 0 (pass1,da1)
<DGC RAID 5 0324>                  at scbus1 target 0 lun 0 (pass2,da2)
<DGC RAID 5 0324>                  at scbus1 target 1 lun 0 (pass3,da3)

	I'm not sure if DGC is a sufficently narrow pattern for
xpt_quirk_table, but can't suggest anything better.
How-To-Repeat: 	Assuming da0 is a LUN on CX3:

	newfs -U /dev/da0s1a
	mount /dev/da0s1a /mnt
	tar cf - -C / --one-file-system . usr var tmp | tar xvf -C /mnt
Comment 1 Remko Lodder freebsd_committer freebsd_triage 2008-02-20 06:36:17 UTC
Responsible Changed
From-To: freebsd-bugs->scottl

Hi scott this might be something for you (or thomas), can you give 
it a look please?
Comment 2 Scott Long 2008-02-20 15:10:07 UTC
Can you post the inquiry information from this device, either from the 
dmesg output for from running camcontrol?
Comment 3 Oleg Sharoiko 2008-02-20 20:52:10 UTC
Here are lines from dmesg:

da0 at isp0 bus 0 target 0 lun 0
da0: <DGC RAID 5 0324> Fixed Direct Access SCSI-4 device 
da0: Serial Number CK200073700517
da0: 200.000MB/s transfers
da0: Command Queueing Enabled
da0: 102400MB (209715200 512 byte sectors: 255H 63S/T 13054C)

--
Oleg Sharoiko
Comment 4 Oleg Sharoyko 2010-11-10 19:56:07 UTC
Hi Scott,

would you please, if is't possible, bring kern/120858
(http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/120858) back to your
attention. I have an updated patch against the recent current
(attached to this mail) and it also looks like the actual limit is
only 32 tags but not 63 as I have thought earlier. Please let me know
if you need more information on this issue.

-- 
Oleg Sharoyko
Comment 5 Panagiotis Christias 2011-04-05 08:49:35 UTC
A kind request/reminder regarding this patch. 32 tag openings is right 
number for CLARiiON devices as reported in EMC's documentation (I can 
provide more details).

Regards,
Panagiotis

-- 
Panagiotis J. Christias    Network Management Center
p.christias@noc.ntua.gr    National Technical Univ. of Athens, GREECE
Comment 6 Pavel Timofeev 2011-10-04 12:27:36 UTC
Excuse me, I'm currently testing FreeBSD 9 BETA-3 amd64 with EMC CLARiiON
CX3-40 via iSCSI.

And I have problems! It works very unstable and sometimes gives "panic:
ffs_alloccg: map corrupted".

Have you any advices?



Panagiotis Christias, where can I find such documentation?
Comment 7 Oleg Sharoyko 2011-10-11 21:12:25 UTC
Hello Pavel,

On 4 October 2011 12:27, Pavel Timofeev <timp87@gmail.com> wrote:

> Excuse me, I'm currently testing FreeBSD 9 BETA-3 amd64 with EMC CLARiiON
> CX3-40 via iSCSI.
> And I have problems! It works very unstable and sometimes gives "panic:
> ffs_alloccg: map corrupted".
> Have you any advices?

I'm afraid I can't offer much help. I don't have access to the
equipment any more and have never played with iSCSI. You could try
debugging it, but you should be ready to dig the code. Another thing
you could try is to limit the number of tags to a very low value using
`camcontrol tags' and check if that helps. I'd recommend recreating a
file system with newfs after changing tags and before running any
tests. If setting a low value helps you can try binary search to find
the maximum number of tags that works for you. I would expect a
significant drop in performance when running with a low (like 2 or
near that) number of tags and better results with more tags.

Hope that helps.
Regards
-- 
Oleg
Comment 8 Pavel Timofeev 2011-10-12 01:49:09 UTC
Thank you, Oleg!
But bug was found.
http://www.freebsd.org/cgi/query-pr.cgi?pr=160943
It covers many devices from different vendors.
I've tested various opening tags value. Without this patch system crashes
always.
But if it patched increasing opening tag gets more performance and doesn't
affect stability.
12.10.2011 0:12 полÑзоваÑÐµÐ»Ñ "Oleg Sharoyko" <osharoiko@gmail.com> напиÑал:

> Hello Pavel,
>
> On 4 October 2011 12:27, Pavel Timofeev <timp87@gmail.com> wrote:
>
> > Excuse me, I'm currently testing FreeBSD 9 BETA-3 amd64 with EMC CLARiiON
> > CX3-40 via iSCSI.
> > And I have problems! It works very unstable and sometimes gives "panic:
> > ffs_alloccg: map corrupted".
> > Have you any advices?
>
> I'm afraid I can't offer much help. I don't have access to the
> equipment any more and have never played with iSCSI. You could try
> debugging it, but you should be ready to dig the code. Another thing
> you could try is to limit the number of tags to a very low value using
> `camcontrol tags' and check if that helps. I'd recommend recreating a
> file system with newfs after changing tags and before running any
> tests. If setting a low value helps you can try binary search to find
> the maximum number of tags that works for you. I would expect a
> significant drop in performance when running with a low (like 2 or
> near that) number of tags and better results with more tags.
>
> Hope that helps.
> Regards
> --
> Oleg

>
Comment 9 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 08:00:55 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped