Bug 200993 - camcontrol commands results in Periph destroyed on mrsas
Summary: camcontrol commands results in Periph destroyed on mrsas
Status: Closed Unable to Reproduce
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 10.1-STABLE
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-06-20 16:26 UTC by Phil Kulin
Modified: 2024-06-22 07:50 UTC (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Phil Kulin 2015-06-20 16:26:58 UTC
FreeBSD 10.1 amd64 on Dell R530 server

camcontrol identify ...
smartctl -a  ...
detach disk with message:
da0 at mrsas0 bus 1 scbus1 target 0 lun 0
da0: <ATA EDGE Boost Pro P BBF0> s/n WZ141217AS1648620    detached
(da0:mrsas0:1:0:0): Periph destroyed

But camcontrol rescan sometimes return disk

# smartctl -a /dev/da0
# camcontrol rescan 1:0:0
# camcontrol identify /dev/da0
# MegaCli -FwTermLog -Dsply -a0
�06/20/15 12:29:19: C0:EVT#02773-06/20/15 12:29:19: 113=Unexpected sense: PD 00(e0x20/s0) Path 4433221104000000, CDB: 00 00 00 00 00 00, Sense: 1/00/1d
�06/20/15 12:32:56: C0:SysDma: localAddrPlb 50e0241070, localAddr e0241070 00 00 1d 00 00 00 00
06/20/15 16:07:36: C0:EVT#02781-06/20/15 16:07:36: 113=Unexpected sense: PD 07(e0x20/s7) Path 4433221103000000, CDB: 1b 00 00 00 02 00, Sense: 5/24/00

# uname -a
FreeBSD ********** 10.1-RELEASE-p10 FreeBSD 10.1-RELEASE-p10 #0: Wed May 13 06:54:13 UTC 2015     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64

# sysctl -a | grep mrsas
device	mrsas
da0 at mrsas0 bus 1 scbus1 target 0 lun 0
(da0:mrsas0:1:0:0): Periph destroyed
da0 at mrsas0 bus 1 scbus1 target 0 lun 0
da0 at mrsas0 bus 1 scbus1 target 0 lun 0
(da0:mrsas0:1:0:0): Periph destroyed
da0 at mrsas0 bus 1 scbus1 target 0 lun 0
da0 at mrsas0 bus 1 scbus1 target 0 lun 0
(da0:mrsas0:1:0:0): Periph destroyed
da0 at mrsas0 bus 1 scbus1 target 0 lun 0
da7 at mrsas0 bus 1 scbus1 target 7 lun 0
hw.mfi.mrsas_enable: 1
dev.mrsas.%parent: 
dev.mrsas.0.%desc: LSI Invader SAS Controller
dev.mrsas.0.%driver: mrsas
dev.mrsas.0.%location: slot=0 function=0
dev.mrsas.0.%pnpinfo: vendor=0x1000 device=0x005d subvendor=0x1028 subdevice=0x1f49 class=0x010400
dev.mrsas.0.%parent: pci1
dev.mrsas.0.disable_ocr: 0
dev.mrsas.0.driver_version: 06.704.01.01-fbsd
dev.mrsas.0.reset_count: 0
dev.mrsas.0.fw_outstanding: 0
dev.mrsas.0.io_cmds_highwater: 84
dev.mrsas.0.mrsas_debug: 9
dev.mrsas.0.mrsas_io_timeout: 180000
dev.mrsas.0.mrsas_fw_fault_check_delay: 1
dev.mrsas.0.reset_in_progress: 0


# MegaCli -AdpAllInfo -aAll
                                     
Adapter #0

==============================================================================
                    Versions
                ================
Product Name    : PERC H730 Mini
Serial No       : 4CF01GX
FW Package Build: 25.2.1.0037

                    Mfg. Data
                ================
Mfg. Date       : 12/18/14
Rework Date     : 12/18/14
Revision No     : A00
Battery FRU     : N/A

                Image Versions in Flash:
                ================
BIOS Version       : 6.18.02.0_4.16.07.00_0x06070400
Ctrl-R Version     : 5.03-0010
FW Version         : 4.240.00-3615
NVDATA Version     : 3.1310.00-0081
Boot Block Version : 3.02.00.00-0000

                Pending Images in Flash
                ================
None

                PCI Info
                ================
Controller Id	: 0000
Vendor Id       : 1000
Device Id       : 005d
SubVendorId     : 1028
SubDeviceId     : 1f49

Host Interface  : PCIE

ChipRevision    : C0

Link Speed 	     : 3 
Number of Frontend Port: 0 
Device Interface  : PCIE

Number of Backend Port: 8 
Port  :  Address
0        4433221100000000 
1        4433221101000000 
2        4433221102000000 
3        4433221103000000 
4        4433221104000000 
5        4433221105000000 
6        4433221106000000 
7        4433221107000000 

                HW Configuration
                ================
SAS Address      : 544a84200655a100
BBU              : Present
Alarm            : Absent
NVRAM            : Present
Serial Debugger  : Present
Memory           : Present
Flash            : Present
Memory Size      : 1024MB
TPM              : Absent
On board Expander: Absent
Upgrade Key      : Absent
Temperature sensor for ROC    : Present
Temperature sensor for controller    : Present

ROC temperature : 33  degree Celsius
Controller temperature : 33  degree Celcius

                Settings
                ================
Current Time                     : 16:24:37 6/20, 2015
Predictive Fail Poll Interval    : 300sec
Interrupt Throttle Active Count  : 16
Interrupt Throttle Completion    : 50us
Rebuild Rate                     : 30%
PR Rate                          : 30%
BGI Rate                         : 30%
Check Consistency Rate           : 30%
Reconstruction Rate              : 30%
Cache Flush Interval             : 4s
Max Drives to Spinup at One Time : 4
Delay Among Spinup Groups        : 12s
Physical Drive Coercion Mode     : 128MB
Cluster Mode                     : Disabled
Alarm                            : Disabled
Auto Rebuild                     : Enabled
Battery Warning                  : Enabled
Ecc Bucket Size                  : 255
Ecc Bucket Leak Rate             : 240 Minutes
Restore HotSpare on Insertion    : Disabled
Expose Enclosure Devices         : Disabled
Maintain PD Fail History         : Disabled
Host Request Reordering          : Enabled
Auto Detect BackPlane Enabled    : SGPIO/i2c SEP
Load Balance Mode                : Auto
Use FDE Only                     : Yes
Security Key Assigned            : No
Security Key Failed              : No
Security Key Not Backedup        : No
Default LD PowerSave Policy      : Controller Defined
Maximum number of direct attached drives to spin up in 1 min : 0 
Auto Enhanced Import             : No
Any Offline VD Cache Preserved   : No
Allow Boot with Preserved Cache  : No
Disable Online Controller Reset  : No
PFK in NVRAM                     : No
Use disk activity for locate     : No
POST delay			 : 90 seconds
BIOS Error Handling          	 : Pause on Errors
Current Boot Mode 		  :Normal
                Capabilities
                ================
RAID Level Supported             : RAID0, RAID1, RAID5, RAID6, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning, PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span
Supported Drives                 : SAS, SATA

Allowed Mixing:

Mix in Enclosure Allowed

                Status
                ================
ECC Bucket Count                 : 0

                Limitations
                ================
Max Arms Per VD          : 32 
Max Spans Per VD         : 8 
Max Arrays               : 128 
Max Number of VDs        : 64 
Max Parallel Commands    : 928 
Max SGE Count            : 60 
Max Data Transfer Size   : 8192 sectors 
Max Strips PerIO         : 42 
Max LD per array         : 16 
Min Strip Size           : 64 KB
Max Strip Size           : 1.0 MB
Max Configurable CacheCade Size: 0 GB
Current Size of CacheCade      : 0 GB
Current Size of FW Cache       : 0 MB

                Device Present
                ================
Virtual Drives    : 0 
  Degraded        : 0 
  Offline         : 0 
Physical Devices  : 9 
  Disks           : 8 
  Critical Disks  : 0 
  Failed Disks    : 0 

                Supported Adapter Operations
                ================
Rebuild Rate                    : Yes
CC Rate                         : Yes
BGI Rate                        : Yes
Reconstruct Rate                : Yes
Patrol Read Rate                : Yes
Alarm Control                   : Yes
Cluster Support                 : No
BBU                             : Yes
Spanning                        : Yes
Dedicated Hot Spare             : Yes
Revertible Hot Spares           : Yes
Foreign Config Import           : Yes
Self Diagnostic                 : Yes
Allow Mixed Redundancy on Array : No
Global Hot Spares               : Yes
Deny SCSI Passthrough           : No
Deny SMP Passthrough            : No
Deny STP Passthrough            : No
Support Security                : Yes
Snapshot Enabled                : No
Support the OCE without adding drives : Yes
Support PFK                     : No
Support PI                      : Yes
Support Boot Time PFK Change    : No
Disable Online PFK Change       : No
Support LDPI Type1                      : No
Support LDPI Type2                      : Yes
Support LDPI Type3                      : No
Support Shield State            : Yes
Block SSD Write Disk Cache Change: No
Support Online FW Update	: Yes

                Supported VD Operations
                ================
Read Policy          : Yes
Write Policy         : Yes
IO Policy            : Yes
Access Policy        : Yes
Disk Cache Policy    : Yes
Reconstruction       : Yes
Deny Locate          : No
Deny CC              : No
Allow Ctrl Encryption: No
Enable LDBBM         : Yes
Support Breakmirror  : Yes
Power Savings        : Yes

                Supported PD Operations
                ================
Force Online                            : Yes
Force Offline                           : Yes
Force Rebuild                           : Yes
Deny Force Failed                       : No
Deny Force Good/Bad                     : No
Deny Missing Replace                    : No
Deny Clear                              : No
Deny Locate                             : No
Support Temperature                     : Yes
NCQ                                     : No
Disable Copyback                        : No
Enable JBOD                             : Yes
Enable Copyback on SMART                : Yes
Enable Copyback to SSD on SMART Error   : Yes
Enable SSD Patrol Read                  : No
PR Correct Unconfigured Areas           : Yes
Enable Spin Down of UnConfigured Drives : No
Disable Spin Down of hot spares         : Yes
Spin Down time                          : 30 
T10 Power State                         : Yes
                Error Counters
                ================
Memory Correctable Errors   : 0 
Memory Uncorrectable Errors : 0 

                Cluster Information
                ================
Cluster Permitted     : No
Cluster Active        : No

                Default Settings
                ================
Phy Polarity                     : 0 
Phy PolaritySplit                : 0 
Background Rate                  : 30 
Strip Size                       : 64kB
Flush Time                       : 4 seconds
Write Policy                     : WB
Read Policy                      : Adaptive
Cache When BBU Bad               : Disabled
Cached IO                        : No
SMART Mode                       : Mode 6
Alarm Disable                    : No
Coercion Mode                    : 128MB
ZCR Config                       : Unknown
Dirty LED Shows Drive Activity   : No
BIOS Continue on Error           : 1 
Spin Down Mode                   : None
Allowed Device Type              : SAS/SATA Mix
Allow Mix in Enclosure           : Yes
Allow HDD SAS/SATA Mix in VD     : No
Allow SSD SAS/SATA Mix in VD     : No
Allow HDD/SSD Mix in VD          : No
Allow SATA in Cluster            : No
Max Chained Enclosures           : 4 
Disable Ctrl-R                   : No
Enable Web BIOS                  : No
Direct PD Mapping                : Yes
BIOS Enumerate VDs               : Yes
Restore Hot Spare on Insertion   : No
Expose Enclosure Devices         : No
Maintain PD Fail History         : No
Disable Puncturing               : No
Zero Based Enclosure Enumeration : Yes
PreBoot CLI Enabled              : No
LED Show Drive Activity          : Yes
Cluster Disable                  : Yes
SAS Disable                      : No
Auto Detect BackPlane Enable     : SGPIO/i2c SEP
Use FDE Only                     : Yes
Enable Led Header                : No
Delay during POST                : 0 
EnableCrashDump                  : No
Disable Online Controller Reset  : No
EnableLDBBM                      : Yes
Un-Certified Hard Disk Drives    : Allow
Treat Single span R1E as R10     : Yes
Max LD per array                 : 16
Power Saving option              : Don't spin down unconfigured drives
Don't spin down Hot spares
Don't Auto spin down Configured Drives
Power settings apply to all drives - individual PD/LD power settings cannot be set
Max power savings option is  not allowed for LDs. Only T10 power conditions are to be used.
Cached writes are not used for spun down VDs
Can schedule disable power savings at controller level
Default spin down time in minutes: 30 
Enable JBOD                      : Yes
TTY Log In Flash                 : Yes
Auto Enhanced Import             : No
BreakMirror RAID Support         : Yes
Disable Join Mirror              : Yes
Enable Shield State              : No
Time taken to detect CME         : 60s

Exit Code: 0x00
Comment 1 Phil Kulin 2015-06-20 16:29:40 UTC
# camcontrol devlist -v
scbus0 on mrsas0 bus 0:
<>                                 at scbus0 target -1 lun ffffffff ()
scbus1 on mrsas0 bus 1:
<ATA EDGE Boost Pro P BBF0>        at scbus1 target 0 lun 0 (da0,pass0)
<ATA EDGE Boost Pro P BBF0>        at scbus1 target 1 lun 0 (pass1,da1)
<ATA EDGE Boost Pro P BBF0>        at scbus1 target 2 lun 0 (pass2,da2)
<ATA EDGE Boost Pro P BBF0>        at scbus1 target 3 lun 0 (pass3,da3)
<ATA EDGE Boost Pro P BBF0>        at scbus1 target 4 lun 0 (pass4,da4)
<ATA EDGE Boost Pro P BBF0>        at scbus1 target 5 lun 0 (pass5,da5)
<ATA ST2000NM0033-9ZM GA67>        at scbus1 target 6 lun 0 (pass6,da6)
<ATA ST2000NM0033-9ZM GA67>        at scbus1 target 7 lun 0 (pass7,da7)
<>                                 at scbus1 target -1 lun ffffffff ()
scbus2 on ahcich0 bus 0:
<>                                 at scbus2 target -1 lun ffffffff ()
scbus3 on ahcich1 bus 0:
<>                                 at scbus3 target -1 lun ffffffff ()
scbus4 on ahcich2 bus 0:
<>                                 at scbus4 target -1 lun ffffffff ()
scbus5 on ahcich3 bus 0:
<>                                 at scbus5 target -1 lun ffffffff ()
scbus6 on ahciem0 bus 0:
<AHCI SGPIO Enclosure 1.00 0001>   at scbus6 target 0 lun 0 (pass8,ses0)
<>                                 at scbus6 target -1 lun ffffffff ()
scbus7 on ahcich4 bus 0:
<>                                 at scbus7 target -1 lun ffffffff ()
scbus8 on ahcich5 bus 0:
<>                                 at scbus8 target -1 lun ffffffff ()
scbus9 on ahcich6 bus 0:
<>                                 at scbus9 target -1 lun ffffffff ()
scbus10 on ahcich7 bus 0:
<>                                 at scbus10 target -1 lun ffffffff ()
scbus11 on ahcich8 bus 0:
<>                                 at scbus11 target -1 lun ffffffff ()
scbus12 on ahcich9 bus 0:
<>                                 at scbus12 target -1 lun ffffffff ()
scbus13 on ahciem1 bus 0:
<AHCI SGPIO Enclosure 1.00 0001>   at scbus13 target 0 lun 0 (pass9,ses1)
<>                                 at scbus13 target -1 lun ffffffff ()
scbus-1 on xpt0 bus 0:
<>                                 at scbus-1 target -1 lun ffffffff (xpt0)
Comment 2 Sibananda Sahu 2015-06-22 08:03:18 UTC
Hi Phil,

The driver version that you are using is :
dev.mrsas.0.driver_version: 06.704.01.01-fbsd

Which is quite old.

You can take the latest mrsas(4) driver from http://svn.freebsd.org/base/head/sys/dev/mrsas
Driver version : 06.707.04.03-fbsd
Revision number is: r284697

Please let me know if you see the same issue on the latest driver.


Thanks.
Comment 3 Alex P 2015-06-23 21:50:04 UTC
Hello,

I have the same issue happening on a Supermicro X9SAE-V system with LSI 9341-8i card.

Issue is present with Phil's mrsas version (from 10.1-RELEASE iso media) and with latest mrsas SVN version r284739.


Thanks
Alex


smartctl -a /dev/da0
# causes da0 to disapper with log:
da0 at mrsas0 bus 1 scbus1 target 8 lun 0
da0: <ATA TOSHIBA DT01ACA3 ABB0> s/n            Z4602W6GS detached
(da0:mrsas0:1:8:0): Periph destroyed
# followed by:
smartctl -a /dev/da2
# causes:
da2 at mrsas0 bus 1 scbus1 target 10 lun 0
da2: <ATA TOSHIBA DT01ACA3 ABB0> s/n            Z46Z2NUGS detached
(da2:mrsas0:1:10:0): Periph destroyed


# uname -a
FreeBSD host 10.1-RELEASE FreeBSD 10.1-RELEASE #1: Tue Jun 23 22:05:59 PDT 2015


# sysctl dev.mrsas
dev.mrsas.%parent: 
dev.mrsas.0.%desc: AVAGO Fury SAS Controller
dev.mrsas.0.%driver: mrsas
dev.mrsas.0.%location: slot=0 function=0
dev.mrsas.0.%pnpinfo: vendor=0x1000 device=0x005f subvendor=0x1000 subdevice=0x9341 class=0x010400
dev.mrsas.0.%parent: pci2
dev.mrsas.0.disable_ocr: 0
dev.mrsas.0.driver_version: 06.707.04.03-fbsd
dev.mrsas.0.reset_count: 0
dev.mrsas.0.fw_outstanding: 0
dev.mrsas.0.io_cmds_highwater: 8
dev.mrsas.0.mrsas_debug: 4
dev.mrsas.0.mrsas_io_timeout: 180000
dev.mrsas.0.mrsas_fw_fault_check_delay: 1
dev.mrsas.0.reset_in_progress: 0
Comment 4 Phil Kulin 2015-06-23 21:55:08 UTC
Alex, try
camcontrol identify /dev/da0
Comment 5 Alex P 2015-06-23 21:58:46 UTC
Also forgot to mention that camcontrol rescan brings back any detached drives

(In reply to Phil Kulin from comment #4)
# camcontrol identify /dev/da0
pass1: <TOSHIBA DT01ACA300 MX6OABB0> ATA-8 SATA 3.x device
camcontrol: get_cpi: error sending Path Inquiry CCB: Device not configured

protocol              ATA/ATAPI-8 SATA 3.x
device model          TOSHIBA DT01ACA300
firmware revision     MX6OABB0
serial number         Z46Z2NUGS
WWN                   5000039ff4e9dc2b
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 4096, offset 0
LBA supported         268435455 sectors
LBA48 supported       5860533168 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6 
media RPM             7200

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes      yes
write cache                    yes      yes
flush cache                    yes      yes
overlap                        no
Tagged Command Queuing (TCQ)   no       no
Native Command Queuing (NCQ)   yes              32 tags
NCQ Queue Management           no
NCQ Streaming                  no
Receive & Send FPDMA Queued    no
SMART                          yes      yes
microcode download             yes      yes
security                       yes      no
power management               yes      yes
advanced power management      yes      no      0/0x00
automatic acoustic management  no       no
media status notification      no       no
power-up in Standby            yes      no
write-read-verify              no       no
unload                         no       no
general purpose logging        yes      yes
free-fall                      no       no
Data Set Management (DSM/TRIM) no
Host Protected Area (HPA)      yes      no      5860533168/1
HPA - Security                 no
Comment 6 Alex P 2015-06-24 00:07:19 UTC
UPDATE: upgrading the card firmware to the latest version (package 24.8.0-0020) has resolved this issue with mrsas SVN r284739.

Thanks
Alex
Comment 7 Phil Kulin 2015-08-10 13:02:52 UTC
I am sorry, but I can not quickly produce the tests on production client's server. I have updated the firmware to the latest version for Dell PowerEdge 530. I have recompile the FreeBSD mrsas driver to the latest SVN revision. Problem still exists.


(CYPRUS === GENERIC with RACCT enabled and  mrsas from HEAD)
# uname -a
FreeBSD kf7.openstar.ru 10.1-RELEASE-p17 FreeBSD 10.1-RELEASE-p17 #3: Mon Aug 10 06:21:02 MSK 2015     root@**********:/usr/obj/usr/src/sys/CYPRUS  amd64

# sysctl dev.mrsas
dev.mrsas.%parent: 
dev.mrsas.0.%desc: AVAGO Invader SAS Controller
dev.mrsas.0.%driver: mrsas
dev.mrsas.0.%location: slot=0 function=0
dev.mrsas.0.%pnpinfo: vendor=0x1000 device=0x005d subvendor=0x1028 subdevice=0x1f49 class=0x010400
dev.mrsas.0.%parent: pci1
dev.mrsas.0.disable_ocr: 0
dev.mrsas.0.driver_version: 06.707.05.00-fbsd
dev.mrsas.0.reset_count: 0
dev.mrsas.0.fw_outstanding: 0
dev.mrsas.0.io_cmds_highwater: 81
dev.mrsas.0.mrsas_debug: 4
dev.mrsas.0.mrsas_io_timeout: 180000
dev.mrsas.0.mrsas_fw_fault_check_delay: 1
dev.mrsas.0.reset_in_progress: 0

(Last 2 lines - problem)
# MegaCli -AdpAllInfo -aAll
08/10/15  3:33:35: C0:EVT#03026-08/10/15  3:33:35:  44=Time established as 08/10/15  3:33:35; (19 seconds since power on)
08/10/15  3:33:51: C0:FlashFind: Searching for HIIM :Found @:0x80200000(App:any)
08/10/15  3:33:51: C0:FlashFind: Searching for HIIM :Found @:0x80200000(App:any)
08/10/15  3:33:51: C0:LOAD section: src=fc0401ce, size=37ecb, dst=0, mode=1...done
08/10/15  3:35:49: C0:MFI INIT: IOC Frame Received : timestamp=99
08/10/15  3:35:49: C0:SystemRequestFrameSize=40 ReplyDescriptorPostQueueDepth=740 ReplyFreeQueueDepth=0 
08/10/15  3:35:49: C0:ReplyDescriptorPostQueueAddress=017903000 SystemRequestFrameBaseAddress=017733000 
08/10/15  3:35:49: C0:Driver Version: 06.707.05.00-fbsd

08/10/15  3:35:49: C0:SRIOV is disabled, Combined Reply Q is enabled, setting maxQueues to 60
08/10/15  3:35:49: C0:MSI-X Enabled: MSIX offset = 0000e000
08/10/15  3:35:49: C0:MSIX[0]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[1]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[2]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[3]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[4]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[5]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[6]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[7]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[8]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[9]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[a]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[b]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[c]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[d]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[e]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:MSIX[f]: 00000000:00000000 - 0
08/10/15  3:35:49: C0:findMsiState: Setting MSI Mode to 2 (0=Legacy, 1=MSI, 2=MSI-X)
08/10/15  3:35:49: C0:MFI INIT 2: Changing FW state to Operational 
08/10/15  3:35:49: C0:EVT#03027-08/10/15  3:35:49: 389=Host driver is loaded and operational
08/10/15  3:35:49: C0:initFreeDDBs=xorPool c0036400-c003fc00 genPool c0000000-c0010000 runTime 1 numXorDdb 40 numCipherDdb =200
08/10/15  3:35:49: C0:initBladeSGL c0010000-c0036400 runTime 1 numSglFrame 4c8
08/10/15  3:35:49: C0:initBladeSgl: FreeHead:c0010000 FreeTail:c0036380

08/10/15  3:35:49: C0:C0333494: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 c20d0400
08/10/15  3:35:49: C0:C03334B4: c003fc00 00000001 ffffffff ffffffff 00000000 00000000

08/10/15  3:35:49: C0:C03334CC: 00000001 00000000 00000000 00000000 00000000 00000000 00000000 c20d0800
08/10/15  3:35:49: C0:C03334EC: c0081800 00000001 ffffffff ffffffff 00000000 00000000
08/10/15  3:35:49: C1:initFreeDDBs=xorPool c0078000-c0081800 genPool c0041c00-c0051c00 runTime 1 numXorDdb 40 numCipherDdb =200
08/10/15  3:35:49: C1:initBladeSGL c0051c00-c0078000 runTime 1 numSglFrame 4c8
08/10/15  3:35:49: C1:initBladeSgl: FreeHead:c0051c00 FreeTail:c0077f80
08/10/15  3:35:49: C0:LdDcmdRaidMapCompleteExt: Completing FW_RAID_MAP cmd
08/10/15  3:35:49: C0:ld sync: all LDs sync'd
08/10/15  3:35:49: C0:LdDcmdRaidMapCompleteExt: Completing FW_RAID_MAP cmd
08/10/15  3:35:52: C0:ld sync: all LDs sync'd
08/10/15  3:38:21: C0:FP_GetSenseData could not get SCSI error as Host had no sense buffer :c00ade00
08/10/15  3:38:42: C0:FP_GetSenseData could not get SCSI error as Host had no sense buffer :c00a5900

# MegaCli -AdpAllInfo -aAll
                                     
Adapter #0

==============================================================================
                    Versions
                ================
Product Name    : PERC H730 Mini
Serial No       : 4CF01GX
FW Package Build: 25.3.0.0016

                    Mfg. Data
                ================
Mfg. Date       : 12/18/14
Rework Date     : 12/18/14
Revision No     : A00
Battery FRU     : N/A

                Image Versions in Flash:
                ================
BIOS Version       : 6.23.03.0_4.16.07.00_0x060C0200
Ctrl-R Version     : 5.04-0012
FW Version         : 4.250.00-4402
NVDATA Version     : 3.1411.00-0019
Boot Block Version : 3.02.00.00-0001

                Pending Images in Flash
                ================
None

                PCI Info
                ================
Controller Id	: 0000
Vendor Id       : 1000
Device Id       : 005d
SubVendorId     : 1028
SubDeviceId     : 1f49

Host Interface  : PCIE

ChipRevision    : C0

Link Speed 	     : 3 
Number of Frontend Port: 0 
Device Interface  : PCIE

Number of Backend Port: 8 
Port  :  Address
0        4433221100000000 
1        4433221101000000 
2        4433221102000000 
3        4433221103000000 
4        4433221104000000 
5        4433221105000000 
6        4433221106000000 
7        4433221107000000 

                HW Configuration
                ================
SAS Address      : 544a84200655a100
BBU              : Present
Alarm            : Absent
NVRAM            : Present
Serial Debugger  : Present
Memory           : Present
Flash            : Present
Memory Size      : 1024MB
TPM              : Absent
On board Expander: Absent
Upgrade Key      : Absent
Temperature sensor for ROC    : Present
Temperature sensor for controller    : Present

ROC temperature : 35  degree Celsius
Controller temperature : 35  degree Celcius

                Settings
                ================
Current Time                     : 13:1:0 8/10, 2015
Predictive Fail Poll Interval    : 300sec
Interrupt Throttle Active Count  : 16
Interrupt Throttle Completion    : 50us
Rebuild Rate                     : 30%
PR Rate                          : 30%
BGI Rate                         : 30%
Check Consistency Rate           : 30%
Reconstruction Rate              : 30%
Cache Flush Interval             : 4s
Max Drives to Spinup at One Time : 4
Delay Among Spinup Groups        : 12s
Physical Drive Coercion Mode     : 128MB
Cluster Mode                     : Disabled
Alarm                            : Disabled
Auto Rebuild                     : Enabled
Battery Warning                  : Enabled
Ecc Bucket Size                  : 255
Ecc Bucket Leak Rate             : 240 Minutes
Restore HotSpare on Insertion    : Disabled
Expose Enclosure Devices         : Disabled
Maintain PD Fail History         : Disabled
Host Request Reordering          : Enabled
Auto Detect BackPlane Enabled    : SGPIO/i2c SEP
Load Balance Mode                : Auto
Use FDE Only                     : Yes
Security Key Assigned            : No
Security Key Failed              : No
Security Key Not Backedup        : No
Default LD PowerSave Policy      : Controller Defined
Maximum number of direct attached drives to spin up in 1 min : 0 
Auto Enhanced Import             : No
Any Offline VD Cache Preserved   : No
Allow Boot with Preserved Cache  : No
Disable Online Controller Reset  : No
PFK in NVRAM                     : No
Use disk activity for locate     : No
POST delay			 : 90 seconds
BIOS Error Handling          	 : Pause on Errors
Current Boot Mode 		  :Normal
                Capabilities
                ================
RAID Level Supported             : RAID0, RAID1, RAID5, RAID6, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning, PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span
Supported Drives                 : SAS, SATA

Allowed Mixing:

Mix in Enclosure Allowed

                Status
                ================
ECC Bucket Count                 : 0

                Limitations
                ================
Max Arms Per VD          : 32 
Max Spans Per VD         : 8 
Max Arrays               : 128 
Max Number of VDs        : 64 
Max Parallel Commands    : 928 
Max SGE Count            : 60 
Max Data Transfer Size   : 8192 sectors 
Max Strips PerIO         : 42 
Max LD per array         : 16 
Min Strip Size           : 64 KB
Max Strip Size           : 1.0 MB
Max Configurable CacheCade Size: 0 GB
Current Size of CacheCade      : 0 GB
Current Size of FW Cache       : 0 MB

                Device Present
                ================
Virtual Drives    : 0 
  Degraded        : 0 
  Offline         : 0 
Physical Devices  : 9 
  Disks           : 8 
  Critical Disks  : 0 
  Failed Disks    : 0 

                Supported Adapter Operations
                ================
Rebuild Rate                    : Yes
CC Rate                         : Yes
BGI Rate                        : Yes
Reconstruct Rate                : Yes
Patrol Read Rate                : Yes
Alarm Control                   : Yes
Cluster Support                 : No
BBU                             : Yes
Spanning                        : Yes
Dedicated Hot Spare             : Yes
Revertible Hot Spares           : Yes
Foreign Config Import           : Yes
Self Diagnostic                 : Yes
Allow Mixed Redundancy on Array : No
Global Hot Spares               : Yes
Deny SCSI Passthrough           : No
Deny SMP Passthrough            : No
Deny STP Passthrough            : No
Support Security                : Yes
Snapshot Enabled                : No
Support the OCE without adding drives : Yes
Support PFK                     : No
Support PI                      : Yes
Support Boot Time PFK Change    : No
Disable Online PFK Change       : No
Support LDPI Type1                      : No
Support LDPI Type2                      : Yes
Support LDPI Type3                      : No
Support Shield State            : Yes
Block SSD Write Disk Cache Change: No
Support Online FW Update	: Yes

                Supported VD Operations
                ================
Read Policy          : Yes
Write Policy         : Yes
IO Policy            : Yes
Access Policy        : Yes
Disk Cache Policy    : Yes
Reconstruction       : Yes
Deny Locate          : No
Deny CC              : No
Allow Ctrl Encryption: No
Enable LDBBM         : Yes
Support Breakmirror  : Yes
Power Savings        : Yes

                Supported PD Operations
                ================
Force Online                            : Yes
Force Offline                           : Yes
Force Rebuild                           : Yes
Deny Force Failed                       : No
Deny Force Good/Bad                     : No
Deny Missing Replace                    : No
Deny Clear                              : No
Deny Locate                             : No
Support Temperature                     : Yes
NCQ                                     : No
Disable Copyback                        : No
Enable JBOD                             : Yes
Enable Copyback on SMART                : Yes
Enable Copyback to SSD on SMART Error   : Yes
Enable SSD Patrol Read                  : No
PR Correct Unconfigured Areas           : Yes
Enable Spin Down of UnConfigured Drives : No
Disable Spin Down of hot spares         : Yes
Spin Down time                          : 30 
T10 Power State                         : Yes
                Error Counters
                ================
Memory Correctable Errors   : 0 
Memory Uncorrectable Errors : 0 

                Cluster Information
                ================
Cluster Permitted     : No
Cluster Active        : No

                Default Settings
                ================
Phy Polarity                     : 0 
Phy PolaritySplit                : 0 
Background Rate                  : 30 
Strip Size                       : 64kB
Flush Time                       : 4 seconds
Write Policy                     : WB
Read Policy                      : Adaptive
Cache When BBU Bad               : Disabled
Cached IO                        : No
SMART Mode                       : Mode 6
Alarm Disable                    : No
Coercion Mode                    : 128MB
ZCR Config                       : Unknown
Dirty LED Shows Drive Activity   : No
BIOS Continue on Error           : 1 
Spin Down Mode                   : None
Allowed Device Type              : SAS/SATA Mix
Allow Mix in Enclosure           : Yes
Allow HDD SAS/SATA Mix in VD     : No
Allow SSD SAS/SATA Mix in VD     : No
Allow HDD/SSD Mix in VD          : No
Allow SATA in Cluster            : No
Max Chained Enclosures           : 4 
Disable Ctrl-R                   : No
Enable Web BIOS                  : No
Direct PD Mapping                : Yes
BIOS Enumerate VDs               : Yes
Restore Hot Spare on Insertion   : No
Expose Enclosure Devices         : No
Maintain PD Fail History         : No
Disable Puncturing               : No
Zero Based Enclosure Enumeration : Yes
PreBoot CLI Enabled              : No
LED Show Drive Activity          : Yes
Cluster Disable                  : Yes
SAS Disable                      : No
Auto Detect BackPlane Enable     : SGPIO/i2c SEP
Use FDE Only                     : Yes
Enable Led Header                : No
Delay during POST                : 0 
EnableCrashDump                  : No
Disable Online Controller Reset  : No
EnableLDBBM                      : Yes
Un-Certified Hard Disk Drives    : Allow
Treat Single span R1E as R10     : Yes
Max LD per array                 : 16
Power Saving option              : Don't spin down unconfigured drives
Don't spin down Hot spares
Don't Auto spin down Configured Drives
Power settings apply to all drives - individual PD/LD power settings cannot be set
Max power savings option is  not allowed for LDs. Only T10 power conditions are to be used.
Cached writes are not used for spun down VDs
Can schedule disable power savings at controller level
Default spin down time in minutes: 30 
Enable JBOD                      : Yes
TTY Log In Flash                 : Yes
Auto Enhanced Import             : No
BreakMirror RAID Support         : Yes
Disable Join Mirror              : Yes
Enable Shield State              : No
Time taken to detect CME         : 60s

Exit Code: 0x00
Comment 8 informatique.src 2015-10-12 14:28:44 UTC
Hi,
I'm having the same issue on a Dell r730xd machine with a PERC H330 Mini controller (It Mode) and FreeBSD 10.2p5

Is there any fix or workaround available please ? 
Thanks ! 

root@localhost:~ # MegaCli -AdpAllInfo -aAll
Adapter #0

==============================================================================
                    Versions
                ================
Product Name    : PERC H330 Mini
Serial No       : 58Q004F
FW Package Build: 25.3.0.0016

                    Mfg. Data
                ================
Mfg. Date       : 08/28/15
Rework Date     : 08/28/15
Revision No     : A01
Battery FRU     : N/A

                Image Versions in Flash:
                ================
BIOS Version       : 6.23.03.0_4.16.07.00_0x060C0200
Ctrl-R Version     : 5.04-0012
FW Version         : 4.250.01-4405
NVDATA Version     : 3.1411.01-0015
Boot Block Version : 3.02.00.00-0001

                Pending Images in Flash
                ================
None

                PCI Info
                ================
Controller Id   : 0000
Vendor Id       : 1000
Device Id       : 005f
SubVendorId     : 1028
SubDeviceId     : 1f4b

Host Interface  : PCIE

ChipRevision    : C0

Link Speed           : 3 
Number of Frontend Port: 0 
Device Interface  : PCIE

Number of Backend Port: 8 
Port  :  Address
0        500056b3cc9fe2ff 
1        0000000000000000 
2        0000000000000000 
3        0000000000000000 
4        0000000000000000 
5        0000000000000000 
6        0000000000000000 
7        0000000000000000 

                HW Configuration
                ================
SAS Address      : 514187702f772200
BBU              : Absent
Alarm            : Absent
NVRAM            : Present
Serial Debugger  : Present
Memory           : Absent
Flash            : Present
Memory Size      : 0MB
TPM              : Absent
On board Expander: Absent
Upgrade Key      : Absent
Temperature sensor for ROC    : Present
Temperature sensor for controller    : Present

ROC temperature : 51  degree Celsius
Controller temperature : 51  degree Celcius

                Settings
                ================
Current Time                     : 16:21:39 10/12, 2015
Predictive Fail Poll Interval    : 300sec
Interrupt Throttle Active Count  : 16
Interrupt Throttle Completion    : 50us
Rebuild Rate                     : 30%
PR Rate                          : 30%
BGI Rate                         : 30%
Check Consistency Rate           : 30%
Reconstruction Rate              : 30%
Cache Flush Interval             : 4s
Max Drives to Spinup at One Time : 4
Delay Among Spinup Groups        : 12s
Physical Drive Coercion Mode     : 128MB
Cluster Mode                     : Disabled
Alarm                            : Disabled
Auto Rebuild                     : Enabled
Battery Warning                  : Disabled
Ecc Bucket Size                  : 255
Ecc Bucket Leak Rate             : 240 Minutes
Restore HotSpare on Insertion    : Disabled
Expose Enclosure Devices         : Enabled
Maintain PD Fail History         : Disabled
Host Request Reordering          : Enabled
Auto Detect BackPlane Enabled    : SGPIO/i2c SEP
Load Balance Mode                : Auto
Use FDE Only                     : Yes
Security Key Assigned            : No
Security Key Failed              : No
Security Key Not Backedup        : No
Default LD PowerSave Policy      : Controller Defined
Maximum number of direct attached drives to spin up in 1 min : 0 
Auto Enhanced Import             : No
Any Offline VD Cache Preserved   : No
Allow Boot with Preserved Cache  : No
Disable Online Controller Reset  : No
PFK in NVRAM                     : No
Use disk activity for locate     : No
POST delay                       : 90 seconds
BIOS Error Handling              : Pause on Errors
Current Boot Mode                 :Normal
                Capabilities
                ================
RAID Level Supported             : RAID00
Supported Drives                 : SAS, SATA

Allowed Mixing:

Mix in Enclosure Allowed

                Status
                ================
ECC Bucket Count                 : 0

                Limitations
                ================
Max Arms Per VD          : 0 
Max Spans Per VD         : 0 
Max Arrays               : 0 
Max Number of VDs        : 0 
Max Parallel Commands    : 256 
Max SGE Count            : 60 
Max Data Transfer Size   : 8192 sectors 
Max Strips PerIO         : 20 
Max LD per array         : 16 
Min Strip Size           : 64 KB
Max Strip Size           : 64 KB
Max Configurable CacheCade Size: 0 GB
Current Size of CacheCade      : 0 GB
Current Size of FW Cache       : 0 MB

                Device Present
                ================
Virtual Drives    : 0 
  Degraded        : 0 
  Offline         : 0 
Physical Devices  : 13 
  Disks           : 12 
  Critical Disks  : 0 
  Failed Disks    : 0 

                Supported Adapter Operations
                ================
Rebuild Rate                    : Yes
CC Rate                         : Yes
BGI Rate                        : Yes
Reconstruct Rate                : Yes
Patrol Read Rate                : Yes
Alarm Control                   : Yes
Cluster Support                 : No
BBU                             : No
Spanning                        : No
Dedicated Hot Spare             : No
Revertible Hot Spares           : Yes
Foreign Config Import           : No
Self Diagnostic                 : Yes
Allow Mixed Redundancy on Array : No
Global Hot Spares               : No
Deny SCSI Passthrough           : No
Deny SMP Passthrough            : No
Deny STP Passthrough            : No
Support Security                : No
Snapshot Enabled                : No
Support the OCE without adding drives : Yes
Support PFK                     : No
Support PI                      : Yes
Support Boot Time PFK Change    : No
Disable Online PFK Change       : No
Support LDPI Type1                      : No
Support LDPI Type2                      : Yes
Support LDPI Type3                      : No
Support Shield State            : Yes
Block SSD Write Disk Cache Change: No
Support Online FW Update        : Yes

                Supported VD Operations
                ================
Read Policy          : No
Write Policy         : No
IO Policy            : No
Access Policy        : Yes
Disk Cache Policy    : Yes
Reconstruction       : Yes
Deny Locate          : No
Deny CC              : No
Allow Ctrl Encryption: No
Enable LDBBM         : Yes
Support Breakmirror  : Yes
Power Savings        : Yes
                Supported PD Operations
                ================
Force Online                            : No
Force Offline                           : Yes
Force Rebuild                           : No
Deny Force Failed                       : No
Deny Force Good/Bad                     : No
Deny Missing Replace                    : No
Deny Clear                              : No
Deny Locate                             : No
Support Temperature                     : Yes
NCQ                                     : No
Disable Copyback                        : No
Enable JBOD                             : Yes
Enable Copyback on SMART                : No
Enable Copyback to SSD on SMART Error   : No
Enable SSD Patrol Read                  : No
PR Correct Unconfigured Areas           : Yes
Enable Spin Down of UnConfigured Drives : No
Disable Spin Down of hot spares         : Yes
Spin Down time                          : 30 
T10 Power State                         : Yes
                Error Counters
                ================
Memory Correctable Errors   : 0 
Memory Uncorrectable Errors : 0 

                Cluster Information
                ================
Cluster Permitted     : No
Cluster Active        : No

                Default Settings
                ================
Phy Polarity                     : 0 
Phy PolaritySplit                : 0 
Background Rate                  : 30 
Strip Size                       : 64kB
Flush Time                       : 4 seconds
Write Policy                     : WT
Read Policy                      : None
Cache When BBU Bad               : Disabled
Cached IO                        : No
SMART Mode                       : Mode 6
Alarm Disable                    : No
Coercion Mode                    : 128MB
ZCR Config                       : Unknown
Dirty LED Shows Drive Activity   : No
BIOS Continue on Error           : 1 
Spin Down Mode                   : None
Allowed Device Type              : SAS/SATA Mix
Allow Mix in Enclosure           : Yes
Allow HDD SAS/SATA Mix in VD     : No
Allow SSD SAS/SATA Mix in VD     : No
Allow HDD/SSD Mix in VD          : No
Allow SATA in Cluster            : No
Max Chained Enclosures           : 2 
Disable Ctrl-R                   : No
Enable Web BIOS                  : No
Direct PD Mapping                : Yes
BIOS Enumerate VDs               : Yes
Restore Hot Spare on Insertion   : No
Expose Enclosure Devices         : Yes
Maintain PD Fail History         : No
Disable Puncturing               : No
Zero Based Enclosure Enumeration : Yes
PreBoot CLI Enabled              : No
LED Show Drive Activity          : Yes
Cluster Disable                  : Yes
SAS Disable                      : No
Auto Detect BackPlane Enable     : SGPIO/i2c SEP
Use FDE Only                     : Yes
Enable Led Header                : No
Delay during POST                : 0 
EnableCrashDump                  : No
Disable Online Controller Reset  : No
EnableLDBBM                      : Yes
Un-Certified Hard Disk Drives    : Allow
Treat Single span R1E as R10     : Yes
Max LD per array                 : 16
Power Saving option              : Don't spin down unconfigured drives
Don't spin down Hot spares
Don't Auto spin down Configured Drives
Power settings apply to all drives - individual PD/LD power settings cannot be set
Max power savings option is  not allowed for LDs. Only T10 power conditions are to be used.
Cached writes are not used for spun down VDs
Can schedule disable power savings at controller level
Default spin down time in minutes: 30 
Enable JBOD                      : Yes
TTY Log In Flash                 : Yes
Auto Enhanced Import             : No
BreakMirror RAID Support         : Yes
Disable Join Mirror              : Yes
Enable Shield State              : No
Time taken to detect CME         : 60s

Exit Code: 0x00
Comment 9 marek 2016-04-12 14:18:40 UTC
Hi,

I can confirm, that the new version of firmware (25.4.0.0015), released by Dell on 01 Apr 2016, solves this problem. You can download the new firmware here: 
http://www.dell.com/support/home/us/en/19/Drivers/DriversDetails?driverId=TMKHJ

Tested on PowerEdge R530 with PERC H730P Mini. The hard drives no longer disappear after 'camcontrol identify'.
Comment 10 Garrett Wollman freebsd_committer freebsd_triage 2016-04-21 00:15:47 UTC
I see this on the Dell H330 as well, and the latest firmware from Dell for that adapter (25.3.0.0016,A02) was on the system at manufacturing in December.  This system also gives the annoying once-every-minute "mrsas0: 3515 (514512868s/0x0002/info) - Unexpected sense: Encl PD 20 Path 5241807048b40a00, CDB: 1c 01 02 80 00 00, Sense: 5/24/00" messages as well, which seem to be both well documented and a WONTFIX on Dell's part.
Comment 11 Miroslav Lachman 2016-06-23 15:49:12 UTC
(In reply to Garrett Wollman from comment #10)

I can confirm this on brand new Dell PowerEdge T130 with PERC H330

It works with mfi but once i switched to mrsas I can use camcontrol devlist because it dettaches first 2 of 4 connected disks. So I switched back to mfi.
mrsas and mfi both have problems with smartctl but it is unrelated to this annoying bug.

I am on FreeBSD 10.3 amd64, 
details provided by mfiutil once I am back on mfi driver.

# mfiutil show adapter
mfi0 Adapter:
    Product Name: PERC H330 Adapter
   Serial Number: 62J0915
        Firmware: 25.3.0.0016
     RAID Levels: JBOD, RAID0, RAID1, RAID5, RAID10, RAID50
  Battery Backup: not present
           NVRAM: 32K
  Onboard Memory: 0M
  Minimum Stripe: 64K
  Maximum Stripe: 64K


# mfiutil show firmware
mfi0 Firmware Package Version: 25.3.0.0016
mfi0 Firmware Images:
Name  Version                          Date         Time         Status
BIOS  6.23.03.0_4.16.07.00_0x060C0200  04/23/2015
  04/23/2015
  active
CTLR  5.04-0012                        Apr 23 2015  11:03:48     active
CPLD  0002                             01/22/15     15:28:00     active
APP   4.250.01-4405                    May 01 2015  12:19:12     active
NVDT  3.1411.01-0015                   Mar 31 2015  11:30:09     active
SGFW  1.00.02                          Aug 24 2013  05:06:18     active
BTBL  3.02.00.00-0001                  May 24 2014  13:04:16     active


# mfiutil show drives
mfi0 Physical Drives:
 0 (  932G) JBOD <TOSHIBA MG03ACA1 FL2H serial=4684KQ8DF> SATA S0
 1 (  932G) JBOD <TOSHIBA MG03ACA1 FL2H serial=4683K23KF> SATA S1
 2 (  932G) JBOD <TOSHIBA MG03ACA1 FL2H serial=4684KQ8KF> SATA S2
 3 (  932G) JBOD <TOSHIBA MG03ACA1 FL2H serial=4684KQ8IF> SATA S3
Comment 12 emz 2016-07-06 19:17:14 UTC
Same stuff here. camcontrol identify doesn't lead to topology change, but smartctl -a definitely does. This is reproducible on a fresh 11.[ and 10.3.

Controller:
LSI MegaRAID SAS 9341-4i

Firmware:
Product Name    : LSI MegaRAID SAS 9341-4i
Serial No       : SV62122829
FW Package Build: 24.2.1-0021

Console log:
[root@san1:/etc]# zpool status
  pool: esx
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 5h40m with 0 errors on Tue Jun 28 03:27:20 2016
config:

        NAME        STATE     READ WRITE CKSUM
        esx         ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            da3     ONLINE       0     0     0
            da4     ONLINE       0     0     0
          mirror-1  ONLINE       0     0     0
            da6     ONLINE       0     0     0
            da7     ONLINE       0     0     0
          mirror-2  ONLINE       0     0     0
            da9     ONLINE       0     0     0
            da10    ONLINE       0     0     0
          mirror-3  ONLINE       0     0     0
            da12    ONLINE       0     0     0
            da13    ONLINE       0     0     0
          mirror-4  ONLINE       0     0     0
            da5     ONLINE       0     0     0
            da8     ONLINE       0     0     0
          mirror-5  ONLINE       0     0     0
            da11    ONLINE       0     0     0
            da14    ONLINE       0     0     0
          mirror-6  ONLINE       0     0     0
            da15    ONLINE       0     0     0
            da16    ONLINE       0     0     0

errors: No known data errors

  pool: gamestop
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: resilvered 729M in 0h0m with 0 errors on Thu Jul  7 04:02:53 2016
config:

        NAME        STATE     READ WRITE CKSUM
        gamestop    ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            da0     ONLINE       0     0     0
            da17    ONLINE       0     0     0
          mirror-1  ONLINE       0     0     0
            da18    ONLINE       0     0     0
            da19    ONLINE       0     0     0
          mirror-2  ONLINE       0     0     0
            da20    ONLINE       0     0     0
            da21    ONLINE       0     0     0
          mirror-3  ONLINE       0     0     0
            da22    ONLINE       0     0     0
            da23    ONLINE       0     0     0

errors: No known data errors

  pool: zroot
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 0h1m with 0 errors on Mon Jun 27 21:21:40 2016
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            da1p3   ONLINE       0     0     0
            da2p3   ONLINE       0     0     0

errors: No known data errors
[root@san1:/etc]# smartctl -a /dev/da2
smartctl 6.5 2016-05-07 r4318 [FreeBSD 11.0-ALPHA5 amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     ST2000NX0253
Serial Number:    S460ED5C
LU WWN Device Id: 5 000c50 08ca8b9aa
Firmware Version: SN02
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      2.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-3 (minor revision not indicated)
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Thu Jul  7 03:04:27 2016 MSK
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART Status command failed: Input/output error
SMART overall-health self-assessment test result: PASSED
Warning: This result is based on an Attribute check.

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  609) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        ( 330) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x50bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   081   063   044    Pre-fail  Always       -       134895728
  3 Spin_Up_Time            0x0003   092   092   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       22
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       8
  7 Seek_Error_Rate         0x000f   072   060   030    Pre-fail  Always       -       19270762
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       624
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       22
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   068   067   040    Old_age   Always       -       32 (Min/Max 20/33)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       7
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       0
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       1280
194 Temperature_Celsius     0x0022   032   040   000    Old_age   Always       -       32 (0 12 0 0 0)
195 Hardware_ECC_Recovered  0x001a   005   004   000    Old_age   Always       -       134895728
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0

Read SMART Log Directory failed: Input/output error

Read SMART Error Log failed: Input/output error

Read SMART Self-test Log failed: Input/output error

Read SMART Selective Self-test Log failed: Input/output error

[root@san1:/etc]# dmesg
Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-ALPHA5 #1: Thu Jun 30 22:56:33 MSK 2016
    emz@san1:/usr/obj/usr/src/sys/SAN amd64
[...]
da2 at mrsas0 bus 1 scbus1 target 7 lun 0
da2: <ATA ST2000NX0253 SN02> s/n S460ED5C detached
GEOM_MIRROR: Device swap: provider da2p2 disconnected.
[root@san1:/etc]# camcontrol rescan 1:7:0
Re-scan of 1:7:0 was successful
[root@san1:/etc]# zpool status
  pool: esx
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 5h40m with 0 errors on Tue Jun 28 03:27:20 2016
config:

        NAME        STATE     READ WRITE CKSUM
        esx         ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            da3     ONLINE       0     0     0
            da4     ONLINE       0     0     0
          mirror-1  ONLINE       0     0     0
            da6     ONLINE       0     0     0
            da7     ONLINE       0     0     0
          mirror-2  ONLINE       0     0     0
            da9     ONLINE       0     0     0
            da10    ONLINE       0     0     0
          mirror-3  ONLINE       0     0     0
            da12    ONLINE       0     0     0
            da13    ONLINE       0     0     0
          mirror-4  ONLINE       0     0     0
            da5     ONLINE       0     0     0
            da8     ONLINE       0     0     0
          mirror-5  ONLINE       0     0     0
            da11    ONLINE       0     0     0
            da14    ONLINE       0     0     0
          mirror-6  ONLINE       0     0     0
            da15    ONLINE       0     0     0
            da16    ONLINE       0     0     0

errors: No known data errors

  pool: gamestop
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: resilvered 729M in 0h0m with 0 errors on Thu Jul  7 04:02:53 2016
config:

        NAME        STATE     READ WRITE CKSUM
        gamestop    ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            da0     ONLINE       0     0     0
            da17    ONLINE       0     0     0
          mirror-1  ONLINE       0     0     0
            da18    ONLINE       0     0     0
            da19    ONLINE       0     0     0
          mirror-2  ONLINE       0     0     0
            da20    ONLINE       0     0     0
            da21    ONLINE       0     0     0
          mirror-3  ONLINE       0     0     0
            da22    ONLINE       0     0     0
            da23    ONLINE       0     0     0

errors: No known data errors

  pool: zroot
 state: DEGRADED
status: One or more devices has been removed by the administrator.
        Sufficient replicas exist for the pool to continue functioning in a
        degraded state.
action: Online the device using 'zpool online' or replace the device with
        'zpool replace'.
  scan: scrub repaired 0 in 0h1m with 0 errors on Mon Jun 27 21:21:40 2016
config:

        NAME                      STATE     READ WRITE CKSUM
        zroot                     DEGRADED     0     0     0
          mirror-0                DEGRADED     0     0     0
            da1p3                 ONLINE       0     0     0
            17040910893353730136  REMOVED      0     0     0  was /dev/da2p3

errors: No known data errors
[root@san1:/etc]# zpool online zroot da2p3
[root@san1:/etc]# zpool status
  pool: esx
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 5h40m with 0 errors on Tue Jun 28 03:27:20 2016
config:

        NAME        STATE     READ WRITE CKSUM
        esx         ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            da3     ONLINE       0     0     0
            da4     ONLINE       0     0     0
          mirror-1  ONLINE       0     0     0
            da6     ONLINE       0     0     0
            da7     ONLINE       0     0     0
          mirror-2  ONLINE       0     0     0
            da9     ONLINE       0     0     0
            da10    ONLINE       0     0     0
          mirror-3  ONLINE       0     0     0
            da12    ONLINE       0     0     0
            da13    ONLINE       0     0     0
          mirror-4  ONLINE       0     0     0
            da5     ONLINE       0     0     0
            da8     ONLINE       0     0     0
          mirror-5  ONLINE       0     0     0
            da11    ONLINE       0     0     0
            da14    ONLINE       0     0     0
          mirror-6  ONLINE       0     0     0
            da15    ONLINE       0     0     0
            da16    ONLINE       0     0     0

errors: No known data errors

  pool: gamestop
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: resilvered 729M in 0h0m with 0 errors on Thu Jul  7 04:02:53 2016
config:

        NAME        STATE     READ WRITE CKSUM
        gamestop    ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            da0     ONLINE       0     0     0
            da17    ONLINE       0     0     0
          mirror-1  ONLINE       0     0     0
            da18    ONLINE       0     0     0
            da19    ONLINE       0     0     0
          mirror-2  ONLINE       0     0     0
            da20    ONLINE       0     0     0
            da21    ONLINE       0     0     0
          mirror-3  ONLINE       0     0     0
            da22    ONLINE       0     0     0
            da23    ONLINE       0     0     0

errors: No known data errors

  pool: zroot
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: resilvered 16K in 0h0m with 0 errors on Thu Jul  7 03:05:18 2016
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            da1p3   ONLINE       0     0     0
            da2p3   ONLINE       0     0     0

errors: No known data errors
[root@san1:/etc]#
Comment 13 emz 2016-07-06 19:20:44 UTC
And I also have an error in the LSI log:

07/07/16  3:04:26: C0:EVT#04219-07/07/16  3:04:26: 113=Unexpected sense: PD 07(e0x04/s11) Path 50030480091b514b, CDB: 00 00 00 00 00 00, Sense: 1/00/1d
Ъ7/07/16  3:04:26: C0:Raw Sense for PD 7: 70 00 01 00 00 00 00 0a 00 00 00 00 00 1d 00 00 00 00
Exit Code: 0x00

does it mean it's a controller/firmware issue ?
Comment 14 emz 2016-07-07 03:49:04 UTC
Can't reproduse anymore after flashing the last available firmware, 24.15.0-0016 for my MegaRAID SAS 9341-4i.

Looks like this isn't a FreeeBSD driver bug.