FreeBSD 10.1 amd64 on Dell R530 server camcontrol identify ... smartctl -a ... detach disk with message: da0 at mrsas0 bus 1 scbus1 target 0 lun 0 da0: <ATA EDGE Boost Pro P BBF0> s/n WZ141217AS1648620 detached (da0:mrsas0:1:0:0): Periph destroyed But camcontrol rescan sometimes return disk # smartctl -a /dev/da0 # camcontrol rescan 1:0:0 # camcontrol identify /dev/da0 # MegaCli -FwTermLog -Dsply -a0 �06/20/15 12:29:19: C0:EVT#02773-06/20/15 12:29:19: 113=Unexpected sense: PD 00(e0x20/s0) Path 4433221104000000, CDB: 00 00 00 00 00 00, Sense: 1/00/1d �06/20/15 12:32:56: C0:SysDma: localAddrPlb 50e0241070, localAddr e0241070 00 00 1d 00 00 00 00 06/20/15 16:07:36: C0:EVT#02781-06/20/15 16:07:36: 113=Unexpected sense: PD 07(e0x20/s7) Path 4433221103000000, CDB: 1b 00 00 00 02 00, Sense: 5/24/00 # uname -a FreeBSD ********** 10.1-RELEASE-p10 FreeBSD 10.1-RELEASE-p10 #0: Wed May 13 06:54:13 UTC 2015 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 # sysctl -a | grep mrsas device mrsas da0 at mrsas0 bus 1 scbus1 target 0 lun 0 (da0:mrsas0:1:0:0): Periph destroyed da0 at mrsas0 bus 1 scbus1 target 0 lun 0 da0 at mrsas0 bus 1 scbus1 target 0 lun 0 (da0:mrsas0:1:0:0): Periph destroyed da0 at mrsas0 bus 1 scbus1 target 0 lun 0 da0 at mrsas0 bus 1 scbus1 target 0 lun 0 (da0:mrsas0:1:0:0): Periph destroyed da0 at mrsas0 bus 1 scbus1 target 0 lun 0 da7 at mrsas0 bus 1 scbus1 target 7 lun 0 hw.mfi.mrsas_enable: 1 dev.mrsas.%parent: dev.mrsas.0.%desc: LSI Invader SAS Controller dev.mrsas.0.%driver: mrsas dev.mrsas.0.%location: slot=0 function=0 dev.mrsas.0.%pnpinfo: vendor=0x1000 device=0x005d subvendor=0x1028 subdevice=0x1f49 class=0x010400 dev.mrsas.0.%parent: pci1 dev.mrsas.0.disable_ocr: 0 dev.mrsas.0.driver_version: 06.704.01.01-fbsd dev.mrsas.0.reset_count: 0 dev.mrsas.0.fw_outstanding: 0 dev.mrsas.0.io_cmds_highwater: 84 dev.mrsas.0.mrsas_debug: 9 dev.mrsas.0.mrsas_io_timeout: 180000 dev.mrsas.0.mrsas_fw_fault_check_delay: 1 dev.mrsas.0.reset_in_progress: 0 # MegaCli -AdpAllInfo -aAll Adapter #0 ============================================================================== Versions ================ Product Name : PERC H730 Mini Serial No : 4CF01GX FW Package Build: 25.2.1.0037 Mfg. Data ================ Mfg. Date : 12/18/14 Rework Date : 12/18/14 Revision No : A00 Battery FRU : N/A Image Versions in Flash: ================ BIOS Version : 6.18.02.0_4.16.07.00_0x06070400 Ctrl-R Version : 5.03-0010 FW Version : 4.240.00-3615 NVDATA Version : 3.1310.00-0081 Boot Block Version : 3.02.00.00-0000 Pending Images in Flash ================ None PCI Info ================ Controller Id : 0000 Vendor Id : 1000 Device Id : 005d SubVendorId : 1028 SubDeviceId : 1f49 Host Interface : PCIE ChipRevision : C0 Link Speed : 3 Number of Frontend Port: 0 Device Interface : PCIE Number of Backend Port: 8 Port : Address 0 4433221100000000 1 4433221101000000 2 4433221102000000 3 4433221103000000 4 4433221104000000 5 4433221105000000 6 4433221106000000 7 4433221107000000 HW Configuration ================ SAS Address : 544a84200655a100 BBU : Present Alarm : Absent NVRAM : Present Serial Debugger : Present Memory : Present Flash : Present Memory Size : 1024MB TPM : Absent On board Expander: Absent Upgrade Key : Absent Temperature sensor for ROC : Present Temperature sensor for controller : Present ROC temperature : 33 degree Celsius Controller temperature : 33 degree Celcius Settings ================ Current Time : 16:24:37 6/20, 2015 Predictive Fail Poll Interval : 300sec Interrupt Throttle Active Count : 16 Interrupt Throttle Completion : 50us Rebuild Rate : 30% PR Rate : 30% BGI Rate : 30% Check Consistency Rate : 30% Reconstruction Rate : 30% Cache Flush Interval : 4s Max Drives to Spinup at One Time : 4 Delay Among Spinup Groups : 12s Physical Drive Coercion Mode : 128MB Cluster Mode : Disabled Alarm : Disabled Auto Rebuild : Enabled Battery Warning : Enabled Ecc Bucket Size : 255 Ecc Bucket Leak Rate : 240 Minutes Restore HotSpare on Insertion : Disabled Expose Enclosure Devices : Disabled Maintain PD Fail History : Disabled Host Request Reordering : Enabled Auto Detect BackPlane Enabled : SGPIO/i2c SEP Load Balance Mode : Auto Use FDE Only : Yes Security Key Assigned : No Security Key Failed : No Security Key Not Backedup : No Default LD PowerSave Policy : Controller Defined Maximum number of direct attached drives to spin up in 1 min : 0 Auto Enhanced Import : No Any Offline VD Cache Preserved : No Allow Boot with Preserved Cache : No Disable Online Controller Reset : No PFK in NVRAM : No Use disk activity for locate : No POST delay : 90 seconds BIOS Error Handling : Pause on Errors Current Boot Mode :Normal Capabilities ================ RAID Level Supported : RAID0, RAID1, RAID5, RAID6, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning, PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span Supported Drives : SAS, SATA Allowed Mixing: Mix in Enclosure Allowed Status ================ ECC Bucket Count : 0 Limitations ================ Max Arms Per VD : 32 Max Spans Per VD : 8 Max Arrays : 128 Max Number of VDs : 64 Max Parallel Commands : 928 Max SGE Count : 60 Max Data Transfer Size : 8192 sectors Max Strips PerIO : 42 Max LD per array : 16 Min Strip Size : 64 KB Max Strip Size : 1.0 MB Max Configurable CacheCade Size: 0 GB Current Size of CacheCade : 0 GB Current Size of FW Cache : 0 MB Device Present ================ Virtual Drives : 0 Degraded : 0 Offline : 0 Physical Devices : 9 Disks : 8 Critical Disks : 0 Failed Disks : 0 Supported Adapter Operations ================ Rebuild Rate : Yes CC Rate : Yes BGI Rate : Yes Reconstruct Rate : Yes Patrol Read Rate : Yes Alarm Control : Yes Cluster Support : No BBU : Yes Spanning : Yes Dedicated Hot Spare : Yes Revertible Hot Spares : Yes Foreign Config Import : Yes Self Diagnostic : Yes Allow Mixed Redundancy on Array : No Global Hot Spares : Yes Deny SCSI Passthrough : No Deny SMP Passthrough : No Deny STP Passthrough : No Support Security : Yes Snapshot Enabled : No Support the OCE without adding drives : Yes Support PFK : No Support PI : Yes Support Boot Time PFK Change : No Disable Online PFK Change : No Support LDPI Type1 : No Support LDPI Type2 : Yes Support LDPI Type3 : No Support Shield State : Yes Block SSD Write Disk Cache Change: No Support Online FW Update : Yes Supported VD Operations ================ Read Policy : Yes Write Policy : Yes IO Policy : Yes Access Policy : Yes Disk Cache Policy : Yes Reconstruction : Yes Deny Locate : No Deny CC : No Allow Ctrl Encryption: No Enable LDBBM : Yes Support Breakmirror : Yes Power Savings : Yes Supported PD Operations ================ Force Online : Yes Force Offline : Yes Force Rebuild : Yes Deny Force Failed : No Deny Force Good/Bad : No Deny Missing Replace : No Deny Clear : No Deny Locate : No Support Temperature : Yes NCQ : No Disable Copyback : No Enable JBOD : Yes Enable Copyback on SMART : Yes Enable Copyback to SSD on SMART Error : Yes Enable SSD Patrol Read : No PR Correct Unconfigured Areas : Yes Enable Spin Down of UnConfigured Drives : No Disable Spin Down of hot spares : Yes Spin Down time : 30 T10 Power State : Yes Error Counters ================ Memory Correctable Errors : 0 Memory Uncorrectable Errors : 0 Cluster Information ================ Cluster Permitted : No Cluster Active : No Default Settings ================ Phy Polarity : 0 Phy PolaritySplit : 0 Background Rate : 30 Strip Size : 64kB Flush Time : 4 seconds Write Policy : WB Read Policy : Adaptive Cache When BBU Bad : Disabled Cached IO : No SMART Mode : Mode 6 Alarm Disable : No Coercion Mode : 128MB ZCR Config : Unknown Dirty LED Shows Drive Activity : No BIOS Continue on Error : 1 Spin Down Mode : None Allowed Device Type : SAS/SATA Mix Allow Mix in Enclosure : Yes Allow HDD SAS/SATA Mix in VD : No Allow SSD SAS/SATA Mix in VD : No Allow HDD/SSD Mix in VD : No Allow SATA in Cluster : No Max Chained Enclosures : 4 Disable Ctrl-R : No Enable Web BIOS : No Direct PD Mapping : Yes BIOS Enumerate VDs : Yes Restore Hot Spare on Insertion : No Expose Enclosure Devices : No Maintain PD Fail History : No Disable Puncturing : No Zero Based Enclosure Enumeration : Yes PreBoot CLI Enabled : No LED Show Drive Activity : Yes Cluster Disable : Yes SAS Disable : No Auto Detect BackPlane Enable : SGPIO/i2c SEP Use FDE Only : Yes Enable Led Header : No Delay during POST : 0 EnableCrashDump : No Disable Online Controller Reset : No EnableLDBBM : Yes Un-Certified Hard Disk Drives : Allow Treat Single span R1E as R10 : Yes Max LD per array : 16 Power Saving option : Don't spin down unconfigured drives Don't spin down Hot spares Don't Auto spin down Configured Drives Power settings apply to all drives - individual PD/LD power settings cannot be set Max power savings option is not allowed for LDs. Only T10 power conditions are to be used. Cached writes are not used for spun down VDs Can schedule disable power savings at controller level Default spin down time in minutes: 30 Enable JBOD : Yes TTY Log In Flash : Yes Auto Enhanced Import : No BreakMirror RAID Support : Yes Disable Join Mirror : Yes Enable Shield State : No Time taken to detect CME : 60s Exit Code: 0x00
# camcontrol devlist -v scbus0 on mrsas0 bus 0: <> at scbus0 target -1 lun ffffffff () scbus1 on mrsas0 bus 1: <ATA EDGE Boost Pro P BBF0> at scbus1 target 0 lun 0 (da0,pass0) <ATA EDGE Boost Pro P BBF0> at scbus1 target 1 lun 0 (pass1,da1) <ATA EDGE Boost Pro P BBF0> at scbus1 target 2 lun 0 (pass2,da2) <ATA EDGE Boost Pro P BBF0> at scbus1 target 3 lun 0 (pass3,da3) <ATA EDGE Boost Pro P BBF0> at scbus1 target 4 lun 0 (pass4,da4) <ATA EDGE Boost Pro P BBF0> at scbus1 target 5 lun 0 (pass5,da5) <ATA ST2000NM0033-9ZM GA67> at scbus1 target 6 lun 0 (pass6,da6) <ATA ST2000NM0033-9ZM GA67> at scbus1 target 7 lun 0 (pass7,da7) <> at scbus1 target -1 lun ffffffff () scbus2 on ahcich0 bus 0: <> at scbus2 target -1 lun ffffffff () scbus3 on ahcich1 bus 0: <> at scbus3 target -1 lun ffffffff () scbus4 on ahcich2 bus 0: <> at scbus4 target -1 lun ffffffff () scbus5 on ahcich3 bus 0: <> at scbus5 target -1 lun ffffffff () scbus6 on ahciem0 bus 0: <AHCI SGPIO Enclosure 1.00 0001> at scbus6 target 0 lun 0 (pass8,ses0) <> at scbus6 target -1 lun ffffffff () scbus7 on ahcich4 bus 0: <> at scbus7 target -1 lun ffffffff () scbus8 on ahcich5 bus 0: <> at scbus8 target -1 lun ffffffff () scbus9 on ahcich6 bus 0: <> at scbus9 target -1 lun ffffffff () scbus10 on ahcich7 bus 0: <> at scbus10 target -1 lun ffffffff () scbus11 on ahcich8 bus 0: <> at scbus11 target -1 lun ffffffff () scbus12 on ahcich9 bus 0: <> at scbus12 target -1 lun ffffffff () scbus13 on ahciem1 bus 0: <AHCI SGPIO Enclosure 1.00 0001> at scbus13 target 0 lun 0 (pass9,ses1) <> at scbus13 target -1 lun ffffffff () scbus-1 on xpt0 bus 0: <> at scbus-1 target -1 lun ffffffff (xpt0)
Hi Phil, The driver version that you are using is : dev.mrsas.0.driver_version: 06.704.01.01-fbsd Which is quite old. You can take the latest mrsas(4) driver from http://svn.freebsd.org/base/head/sys/dev/mrsas Driver version : 06.707.04.03-fbsd Revision number is: r284697 Please let me know if you see the same issue on the latest driver. Thanks.
Hello, I have the same issue happening on a Supermicro X9SAE-V system with LSI 9341-8i card. Issue is present with Phil's mrsas version (from 10.1-RELEASE iso media) and with latest mrsas SVN version r284739. Thanks Alex smartctl -a /dev/da0 # causes da0 to disapper with log: da0 at mrsas0 bus 1 scbus1 target 8 lun 0 da0: <ATA TOSHIBA DT01ACA3 ABB0> s/n Z4602W6GS detached (da0:mrsas0:1:8:0): Periph destroyed # followed by: smartctl -a /dev/da2 # causes: da2 at mrsas0 bus 1 scbus1 target 10 lun 0 da2: <ATA TOSHIBA DT01ACA3 ABB0> s/n Z46Z2NUGS detached (da2:mrsas0:1:10:0): Periph destroyed # uname -a FreeBSD host 10.1-RELEASE FreeBSD 10.1-RELEASE #1: Tue Jun 23 22:05:59 PDT 2015 # sysctl dev.mrsas dev.mrsas.%parent: dev.mrsas.0.%desc: AVAGO Fury SAS Controller dev.mrsas.0.%driver: mrsas dev.mrsas.0.%location: slot=0 function=0 dev.mrsas.0.%pnpinfo: vendor=0x1000 device=0x005f subvendor=0x1000 subdevice=0x9341 class=0x010400 dev.mrsas.0.%parent: pci2 dev.mrsas.0.disable_ocr: 0 dev.mrsas.0.driver_version: 06.707.04.03-fbsd dev.mrsas.0.reset_count: 0 dev.mrsas.0.fw_outstanding: 0 dev.mrsas.0.io_cmds_highwater: 8 dev.mrsas.0.mrsas_debug: 4 dev.mrsas.0.mrsas_io_timeout: 180000 dev.mrsas.0.mrsas_fw_fault_check_delay: 1 dev.mrsas.0.reset_in_progress: 0
Alex, try camcontrol identify /dev/da0
Also forgot to mention that camcontrol rescan brings back any detached drives (In reply to Phil Kulin from comment #4) # camcontrol identify /dev/da0 pass1: <TOSHIBA DT01ACA300 MX6OABB0> ATA-8 SATA 3.x device camcontrol: get_cpi: error sending Path Inquiry CCB: Device not configured protocol ATA/ATAPI-8 SATA 3.x device model TOSHIBA DT01ACA300 firmware revision MX6OABB0 serial number Z46Z2NUGS WWN 5000039ff4e9dc2b cylinders 16383 heads 16 sectors/track 63 sector size logical 512, physical 4096, offset 0 LBA supported 268435455 sectors LBA48 supported 5860533168 sectors PIO supported PIO4 DMA supported WDMA2 UDMA6 media RPM 7200 Feature Support Enabled Value Vendor read ahead yes yes write cache yes yes flush cache yes yes overlap no Tagged Command Queuing (TCQ) no no Native Command Queuing (NCQ) yes 32 tags NCQ Queue Management no NCQ Streaming no Receive & Send FPDMA Queued no SMART yes yes microcode download yes yes security yes no power management yes yes advanced power management yes no 0/0x00 automatic acoustic management no no media status notification no no power-up in Standby yes no write-read-verify no no unload no no general purpose logging yes yes free-fall no no Data Set Management (DSM/TRIM) no Host Protected Area (HPA) yes no 5860533168/1 HPA - Security no
UPDATE: upgrading the card firmware to the latest version (package 24.8.0-0020) has resolved this issue with mrsas SVN r284739. Thanks Alex
I am sorry, but I can not quickly produce the tests on production client's server. I have updated the firmware to the latest version for Dell PowerEdge 530. I have recompile the FreeBSD mrsas driver to the latest SVN revision. Problem still exists. (CYPRUS === GENERIC with RACCT enabled and mrsas from HEAD) # uname -a FreeBSD kf7.openstar.ru 10.1-RELEASE-p17 FreeBSD 10.1-RELEASE-p17 #3: Mon Aug 10 06:21:02 MSK 2015 root@**********:/usr/obj/usr/src/sys/CYPRUS amd64 # sysctl dev.mrsas dev.mrsas.%parent: dev.mrsas.0.%desc: AVAGO Invader SAS Controller dev.mrsas.0.%driver: mrsas dev.mrsas.0.%location: slot=0 function=0 dev.mrsas.0.%pnpinfo: vendor=0x1000 device=0x005d subvendor=0x1028 subdevice=0x1f49 class=0x010400 dev.mrsas.0.%parent: pci1 dev.mrsas.0.disable_ocr: 0 dev.mrsas.0.driver_version: 06.707.05.00-fbsd dev.mrsas.0.reset_count: 0 dev.mrsas.0.fw_outstanding: 0 dev.mrsas.0.io_cmds_highwater: 81 dev.mrsas.0.mrsas_debug: 4 dev.mrsas.0.mrsas_io_timeout: 180000 dev.mrsas.0.mrsas_fw_fault_check_delay: 1 dev.mrsas.0.reset_in_progress: 0 (Last 2 lines - problem) # MegaCli -AdpAllInfo -aAll 08/10/15 3:33:35: C0:EVT#03026-08/10/15 3:33:35: 44=Time established as 08/10/15 3:33:35; (19 seconds since power on) 08/10/15 3:33:51: C0:FlashFind: Searching for HIIM :Found @:0x80200000(App:any) 08/10/15 3:33:51: C0:FlashFind: Searching for HIIM :Found @:0x80200000(App:any) 08/10/15 3:33:51: C0:LOAD section: src=fc0401ce, size=37ecb, dst=0, mode=1...done 08/10/15 3:35:49: C0:MFI INIT: IOC Frame Received : timestamp=99 08/10/15 3:35:49: C0:SystemRequestFrameSize=40 ReplyDescriptorPostQueueDepth=740 ReplyFreeQueueDepth=0 08/10/15 3:35:49: C0:ReplyDescriptorPostQueueAddress=017903000 SystemRequestFrameBaseAddress=017733000 08/10/15 3:35:49: C0:Driver Version: 06.707.05.00-fbsd 08/10/15 3:35:49: C0:SRIOV is disabled, Combined Reply Q is enabled, setting maxQueues to 60 08/10/15 3:35:49: C0:MSI-X Enabled: MSIX offset = 0000e000 08/10/15 3:35:49: C0:MSIX[0]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[1]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[2]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[3]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[4]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[5]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[6]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[7]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[8]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[9]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[a]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[b]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[c]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[d]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[e]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:MSIX[f]: 00000000:00000000 - 0 08/10/15 3:35:49: C0:findMsiState: Setting MSI Mode to 2 (0=Legacy, 1=MSI, 2=MSI-X) 08/10/15 3:35:49: C0:MFI INIT 2: Changing FW state to Operational 08/10/15 3:35:49: C0:EVT#03027-08/10/15 3:35:49: 389=Host driver is loaded and operational 08/10/15 3:35:49: C0:initFreeDDBs=xorPool c0036400-c003fc00 genPool c0000000-c0010000 runTime 1 numXorDdb 40 numCipherDdb =200 08/10/15 3:35:49: C0:initBladeSGL c0010000-c0036400 runTime 1 numSglFrame 4c8 08/10/15 3:35:49: C0:initBladeSgl: FreeHead:c0010000 FreeTail:c0036380 08/10/15 3:35:49: C0:C0333494: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 c20d0400 08/10/15 3:35:49: C0:C03334B4: c003fc00 00000001 ffffffff ffffffff 00000000 00000000 08/10/15 3:35:49: C0:C03334CC: 00000001 00000000 00000000 00000000 00000000 00000000 00000000 c20d0800 08/10/15 3:35:49: C0:C03334EC: c0081800 00000001 ffffffff ffffffff 00000000 00000000 08/10/15 3:35:49: C1:initFreeDDBs=xorPool c0078000-c0081800 genPool c0041c00-c0051c00 runTime 1 numXorDdb 40 numCipherDdb =200 08/10/15 3:35:49: C1:initBladeSGL c0051c00-c0078000 runTime 1 numSglFrame 4c8 08/10/15 3:35:49: C1:initBladeSgl: FreeHead:c0051c00 FreeTail:c0077f80 08/10/15 3:35:49: C0:LdDcmdRaidMapCompleteExt: Completing FW_RAID_MAP cmd 08/10/15 3:35:49: C0:ld sync: all LDs sync'd 08/10/15 3:35:49: C0:LdDcmdRaidMapCompleteExt: Completing FW_RAID_MAP cmd 08/10/15 3:35:52: C0:ld sync: all LDs sync'd 08/10/15 3:38:21: C0:FP_GetSenseData could not get SCSI error as Host had no sense buffer :c00ade00 08/10/15 3:38:42: C0:FP_GetSenseData could not get SCSI error as Host had no sense buffer :c00a5900 # MegaCli -AdpAllInfo -aAll Adapter #0 ============================================================================== Versions ================ Product Name : PERC H730 Mini Serial No : 4CF01GX FW Package Build: 25.3.0.0016 Mfg. Data ================ Mfg. Date : 12/18/14 Rework Date : 12/18/14 Revision No : A00 Battery FRU : N/A Image Versions in Flash: ================ BIOS Version : 6.23.03.0_4.16.07.00_0x060C0200 Ctrl-R Version : 5.04-0012 FW Version : 4.250.00-4402 NVDATA Version : 3.1411.00-0019 Boot Block Version : 3.02.00.00-0001 Pending Images in Flash ================ None PCI Info ================ Controller Id : 0000 Vendor Id : 1000 Device Id : 005d SubVendorId : 1028 SubDeviceId : 1f49 Host Interface : PCIE ChipRevision : C0 Link Speed : 3 Number of Frontend Port: 0 Device Interface : PCIE Number of Backend Port: 8 Port : Address 0 4433221100000000 1 4433221101000000 2 4433221102000000 3 4433221103000000 4 4433221104000000 5 4433221105000000 6 4433221106000000 7 4433221107000000 HW Configuration ================ SAS Address : 544a84200655a100 BBU : Present Alarm : Absent NVRAM : Present Serial Debugger : Present Memory : Present Flash : Present Memory Size : 1024MB TPM : Absent On board Expander: Absent Upgrade Key : Absent Temperature sensor for ROC : Present Temperature sensor for controller : Present ROC temperature : 35 degree Celsius Controller temperature : 35 degree Celcius Settings ================ Current Time : 13:1:0 8/10, 2015 Predictive Fail Poll Interval : 300sec Interrupt Throttle Active Count : 16 Interrupt Throttle Completion : 50us Rebuild Rate : 30% PR Rate : 30% BGI Rate : 30% Check Consistency Rate : 30% Reconstruction Rate : 30% Cache Flush Interval : 4s Max Drives to Spinup at One Time : 4 Delay Among Spinup Groups : 12s Physical Drive Coercion Mode : 128MB Cluster Mode : Disabled Alarm : Disabled Auto Rebuild : Enabled Battery Warning : Enabled Ecc Bucket Size : 255 Ecc Bucket Leak Rate : 240 Minutes Restore HotSpare on Insertion : Disabled Expose Enclosure Devices : Disabled Maintain PD Fail History : Disabled Host Request Reordering : Enabled Auto Detect BackPlane Enabled : SGPIO/i2c SEP Load Balance Mode : Auto Use FDE Only : Yes Security Key Assigned : No Security Key Failed : No Security Key Not Backedup : No Default LD PowerSave Policy : Controller Defined Maximum number of direct attached drives to spin up in 1 min : 0 Auto Enhanced Import : No Any Offline VD Cache Preserved : No Allow Boot with Preserved Cache : No Disable Online Controller Reset : No PFK in NVRAM : No Use disk activity for locate : No POST delay : 90 seconds BIOS Error Handling : Pause on Errors Current Boot Mode :Normal Capabilities ================ RAID Level Supported : RAID0, RAID1, RAID5, RAID6, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning, PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span Supported Drives : SAS, SATA Allowed Mixing: Mix in Enclosure Allowed Status ================ ECC Bucket Count : 0 Limitations ================ Max Arms Per VD : 32 Max Spans Per VD : 8 Max Arrays : 128 Max Number of VDs : 64 Max Parallel Commands : 928 Max SGE Count : 60 Max Data Transfer Size : 8192 sectors Max Strips PerIO : 42 Max LD per array : 16 Min Strip Size : 64 KB Max Strip Size : 1.0 MB Max Configurable CacheCade Size: 0 GB Current Size of CacheCade : 0 GB Current Size of FW Cache : 0 MB Device Present ================ Virtual Drives : 0 Degraded : 0 Offline : 0 Physical Devices : 9 Disks : 8 Critical Disks : 0 Failed Disks : 0 Supported Adapter Operations ================ Rebuild Rate : Yes CC Rate : Yes BGI Rate : Yes Reconstruct Rate : Yes Patrol Read Rate : Yes Alarm Control : Yes Cluster Support : No BBU : Yes Spanning : Yes Dedicated Hot Spare : Yes Revertible Hot Spares : Yes Foreign Config Import : Yes Self Diagnostic : Yes Allow Mixed Redundancy on Array : No Global Hot Spares : Yes Deny SCSI Passthrough : No Deny SMP Passthrough : No Deny STP Passthrough : No Support Security : Yes Snapshot Enabled : No Support the OCE without adding drives : Yes Support PFK : No Support PI : Yes Support Boot Time PFK Change : No Disable Online PFK Change : No Support LDPI Type1 : No Support LDPI Type2 : Yes Support LDPI Type3 : No Support Shield State : Yes Block SSD Write Disk Cache Change: No Support Online FW Update : Yes Supported VD Operations ================ Read Policy : Yes Write Policy : Yes IO Policy : Yes Access Policy : Yes Disk Cache Policy : Yes Reconstruction : Yes Deny Locate : No Deny CC : No Allow Ctrl Encryption: No Enable LDBBM : Yes Support Breakmirror : Yes Power Savings : Yes Supported PD Operations ================ Force Online : Yes Force Offline : Yes Force Rebuild : Yes Deny Force Failed : No Deny Force Good/Bad : No Deny Missing Replace : No Deny Clear : No Deny Locate : No Support Temperature : Yes NCQ : No Disable Copyback : No Enable JBOD : Yes Enable Copyback on SMART : Yes Enable Copyback to SSD on SMART Error : Yes Enable SSD Patrol Read : No PR Correct Unconfigured Areas : Yes Enable Spin Down of UnConfigured Drives : No Disable Spin Down of hot spares : Yes Spin Down time : 30 T10 Power State : Yes Error Counters ================ Memory Correctable Errors : 0 Memory Uncorrectable Errors : 0 Cluster Information ================ Cluster Permitted : No Cluster Active : No Default Settings ================ Phy Polarity : 0 Phy PolaritySplit : 0 Background Rate : 30 Strip Size : 64kB Flush Time : 4 seconds Write Policy : WB Read Policy : Adaptive Cache When BBU Bad : Disabled Cached IO : No SMART Mode : Mode 6 Alarm Disable : No Coercion Mode : 128MB ZCR Config : Unknown Dirty LED Shows Drive Activity : No BIOS Continue on Error : 1 Spin Down Mode : None Allowed Device Type : SAS/SATA Mix Allow Mix in Enclosure : Yes Allow HDD SAS/SATA Mix in VD : No Allow SSD SAS/SATA Mix in VD : No Allow HDD/SSD Mix in VD : No Allow SATA in Cluster : No Max Chained Enclosures : 4 Disable Ctrl-R : No Enable Web BIOS : No Direct PD Mapping : Yes BIOS Enumerate VDs : Yes Restore Hot Spare on Insertion : No Expose Enclosure Devices : No Maintain PD Fail History : No Disable Puncturing : No Zero Based Enclosure Enumeration : Yes PreBoot CLI Enabled : No LED Show Drive Activity : Yes Cluster Disable : Yes SAS Disable : No Auto Detect BackPlane Enable : SGPIO/i2c SEP Use FDE Only : Yes Enable Led Header : No Delay during POST : 0 EnableCrashDump : No Disable Online Controller Reset : No EnableLDBBM : Yes Un-Certified Hard Disk Drives : Allow Treat Single span R1E as R10 : Yes Max LD per array : 16 Power Saving option : Don't spin down unconfigured drives Don't spin down Hot spares Don't Auto spin down Configured Drives Power settings apply to all drives - individual PD/LD power settings cannot be set Max power savings option is not allowed for LDs. Only T10 power conditions are to be used. Cached writes are not used for spun down VDs Can schedule disable power savings at controller level Default spin down time in minutes: 30 Enable JBOD : Yes TTY Log In Flash : Yes Auto Enhanced Import : No BreakMirror RAID Support : Yes Disable Join Mirror : Yes Enable Shield State : No Time taken to detect CME : 60s Exit Code: 0x00
Hi, I'm having the same issue on a Dell r730xd machine with a PERC H330 Mini controller (It Mode) and FreeBSD 10.2p5 Is there any fix or workaround available please ? Thanks ! root@localhost:~ # MegaCli -AdpAllInfo -aAll Adapter #0 ============================================================================== Versions ================ Product Name : PERC H330 Mini Serial No : 58Q004F FW Package Build: 25.3.0.0016 Mfg. Data ================ Mfg. Date : 08/28/15 Rework Date : 08/28/15 Revision No : A01 Battery FRU : N/A Image Versions in Flash: ================ BIOS Version : 6.23.03.0_4.16.07.00_0x060C0200 Ctrl-R Version : 5.04-0012 FW Version : 4.250.01-4405 NVDATA Version : 3.1411.01-0015 Boot Block Version : 3.02.00.00-0001 Pending Images in Flash ================ None PCI Info ================ Controller Id : 0000 Vendor Id : 1000 Device Id : 005f SubVendorId : 1028 SubDeviceId : 1f4b Host Interface : PCIE ChipRevision : C0 Link Speed : 3 Number of Frontend Port: 0 Device Interface : PCIE Number of Backend Port: 8 Port : Address 0 500056b3cc9fe2ff 1 0000000000000000 2 0000000000000000 3 0000000000000000 4 0000000000000000 5 0000000000000000 6 0000000000000000 7 0000000000000000 HW Configuration ================ SAS Address : 514187702f772200 BBU : Absent Alarm : Absent NVRAM : Present Serial Debugger : Present Memory : Absent Flash : Present Memory Size : 0MB TPM : Absent On board Expander: Absent Upgrade Key : Absent Temperature sensor for ROC : Present Temperature sensor for controller : Present ROC temperature : 51 degree Celsius Controller temperature : 51 degree Celcius Settings ================ Current Time : 16:21:39 10/12, 2015 Predictive Fail Poll Interval : 300sec Interrupt Throttle Active Count : 16 Interrupt Throttle Completion : 50us Rebuild Rate : 30% PR Rate : 30% BGI Rate : 30% Check Consistency Rate : 30% Reconstruction Rate : 30% Cache Flush Interval : 4s Max Drives to Spinup at One Time : 4 Delay Among Spinup Groups : 12s Physical Drive Coercion Mode : 128MB Cluster Mode : Disabled Alarm : Disabled Auto Rebuild : Enabled Battery Warning : Disabled Ecc Bucket Size : 255 Ecc Bucket Leak Rate : 240 Minutes Restore HotSpare on Insertion : Disabled Expose Enclosure Devices : Enabled Maintain PD Fail History : Disabled Host Request Reordering : Enabled Auto Detect BackPlane Enabled : SGPIO/i2c SEP Load Balance Mode : Auto Use FDE Only : Yes Security Key Assigned : No Security Key Failed : No Security Key Not Backedup : No Default LD PowerSave Policy : Controller Defined Maximum number of direct attached drives to spin up in 1 min : 0 Auto Enhanced Import : No Any Offline VD Cache Preserved : No Allow Boot with Preserved Cache : No Disable Online Controller Reset : No PFK in NVRAM : No Use disk activity for locate : No POST delay : 90 seconds BIOS Error Handling : Pause on Errors Current Boot Mode :Normal Capabilities ================ RAID Level Supported : RAID00 Supported Drives : SAS, SATA Allowed Mixing: Mix in Enclosure Allowed Status ================ ECC Bucket Count : 0 Limitations ================ Max Arms Per VD : 0 Max Spans Per VD : 0 Max Arrays : 0 Max Number of VDs : 0 Max Parallel Commands : 256 Max SGE Count : 60 Max Data Transfer Size : 8192 sectors Max Strips PerIO : 20 Max LD per array : 16 Min Strip Size : 64 KB Max Strip Size : 64 KB Max Configurable CacheCade Size: 0 GB Current Size of CacheCade : 0 GB Current Size of FW Cache : 0 MB Device Present ================ Virtual Drives : 0 Degraded : 0 Offline : 0 Physical Devices : 13 Disks : 12 Critical Disks : 0 Failed Disks : 0 Supported Adapter Operations ================ Rebuild Rate : Yes CC Rate : Yes BGI Rate : Yes Reconstruct Rate : Yes Patrol Read Rate : Yes Alarm Control : Yes Cluster Support : No BBU : No Spanning : No Dedicated Hot Spare : No Revertible Hot Spares : Yes Foreign Config Import : No Self Diagnostic : Yes Allow Mixed Redundancy on Array : No Global Hot Spares : No Deny SCSI Passthrough : No Deny SMP Passthrough : No Deny STP Passthrough : No Support Security : No Snapshot Enabled : No Support the OCE without adding drives : Yes Support PFK : No Support PI : Yes Support Boot Time PFK Change : No Disable Online PFK Change : No Support LDPI Type1 : No Support LDPI Type2 : Yes Support LDPI Type3 : No Support Shield State : Yes Block SSD Write Disk Cache Change: No Support Online FW Update : Yes Supported VD Operations ================ Read Policy : No Write Policy : No IO Policy : No Access Policy : Yes Disk Cache Policy : Yes Reconstruction : Yes Deny Locate : No Deny CC : No Allow Ctrl Encryption: No Enable LDBBM : Yes Support Breakmirror : Yes Power Savings : Yes Supported PD Operations ================ Force Online : No Force Offline : Yes Force Rebuild : No Deny Force Failed : No Deny Force Good/Bad : No Deny Missing Replace : No Deny Clear : No Deny Locate : No Support Temperature : Yes NCQ : No Disable Copyback : No Enable JBOD : Yes Enable Copyback on SMART : No Enable Copyback to SSD on SMART Error : No Enable SSD Patrol Read : No PR Correct Unconfigured Areas : Yes Enable Spin Down of UnConfigured Drives : No Disable Spin Down of hot spares : Yes Spin Down time : 30 T10 Power State : Yes Error Counters ================ Memory Correctable Errors : 0 Memory Uncorrectable Errors : 0 Cluster Information ================ Cluster Permitted : No Cluster Active : No Default Settings ================ Phy Polarity : 0 Phy PolaritySplit : 0 Background Rate : 30 Strip Size : 64kB Flush Time : 4 seconds Write Policy : WT Read Policy : None Cache When BBU Bad : Disabled Cached IO : No SMART Mode : Mode 6 Alarm Disable : No Coercion Mode : 128MB ZCR Config : Unknown Dirty LED Shows Drive Activity : No BIOS Continue on Error : 1 Spin Down Mode : None Allowed Device Type : SAS/SATA Mix Allow Mix in Enclosure : Yes Allow HDD SAS/SATA Mix in VD : No Allow SSD SAS/SATA Mix in VD : No Allow HDD/SSD Mix in VD : No Allow SATA in Cluster : No Max Chained Enclosures : 2 Disable Ctrl-R : No Enable Web BIOS : No Direct PD Mapping : Yes BIOS Enumerate VDs : Yes Restore Hot Spare on Insertion : No Expose Enclosure Devices : Yes Maintain PD Fail History : No Disable Puncturing : No Zero Based Enclosure Enumeration : Yes PreBoot CLI Enabled : No LED Show Drive Activity : Yes Cluster Disable : Yes SAS Disable : No Auto Detect BackPlane Enable : SGPIO/i2c SEP Use FDE Only : Yes Enable Led Header : No Delay during POST : 0 EnableCrashDump : No Disable Online Controller Reset : No EnableLDBBM : Yes Un-Certified Hard Disk Drives : Allow Treat Single span R1E as R10 : Yes Max LD per array : 16 Power Saving option : Don't spin down unconfigured drives Don't spin down Hot spares Don't Auto spin down Configured Drives Power settings apply to all drives - individual PD/LD power settings cannot be set Max power savings option is not allowed for LDs. Only T10 power conditions are to be used. Cached writes are not used for spun down VDs Can schedule disable power savings at controller level Default spin down time in minutes: 30 Enable JBOD : Yes TTY Log In Flash : Yes Auto Enhanced Import : No BreakMirror RAID Support : Yes Disable Join Mirror : Yes Enable Shield State : No Time taken to detect CME : 60s Exit Code: 0x00
Hi, I can confirm, that the new version of firmware (25.4.0.0015), released by Dell on 01 Apr 2016, solves this problem. You can download the new firmware here: http://www.dell.com/support/home/us/en/19/Drivers/DriversDetails?driverId=TMKHJ Tested on PowerEdge R530 with PERC H730P Mini. The hard drives no longer disappear after 'camcontrol identify'.
I see this on the Dell H330 as well, and the latest firmware from Dell for that adapter (25.3.0.0016,A02) was on the system at manufacturing in December. This system also gives the annoying once-every-minute "mrsas0: 3515 (514512868s/0x0002/info) - Unexpected sense: Encl PD 20 Path 5241807048b40a00, CDB: 1c 01 02 80 00 00, Sense: 5/24/00" messages as well, which seem to be both well documented and a WONTFIX on Dell's part.
(In reply to Garrett Wollman from comment #10) I can confirm this on brand new Dell PowerEdge T130 with PERC H330 It works with mfi but once i switched to mrsas I can use camcontrol devlist because it dettaches first 2 of 4 connected disks. So I switched back to mfi. mrsas and mfi both have problems with smartctl but it is unrelated to this annoying bug. I am on FreeBSD 10.3 amd64, details provided by mfiutil once I am back on mfi driver. # mfiutil show adapter mfi0 Adapter: Product Name: PERC H330 Adapter Serial Number: 62J0915 Firmware: 25.3.0.0016 RAID Levels: JBOD, RAID0, RAID1, RAID5, RAID10, RAID50 Battery Backup: not present NVRAM: 32K Onboard Memory: 0M Minimum Stripe: 64K Maximum Stripe: 64K # mfiutil show firmware mfi0 Firmware Package Version: 25.3.0.0016 mfi0 Firmware Images: Name Version Date Time Status BIOS 6.23.03.0_4.16.07.00_0x060C0200 04/23/2015 04/23/2015 active CTLR 5.04-0012 Apr 23 2015 11:03:48 active CPLD 0002 01/22/15 15:28:00 active APP 4.250.01-4405 May 01 2015 12:19:12 active NVDT 3.1411.01-0015 Mar 31 2015 11:30:09 active SGFW 1.00.02 Aug 24 2013 05:06:18 active BTBL 3.02.00.00-0001 May 24 2014 13:04:16 active # mfiutil show drives mfi0 Physical Drives: 0 ( 932G) JBOD <TOSHIBA MG03ACA1 FL2H serial=4684KQ8DF> SATA S0 1 ( 932G) JBOD <TOSHIBA MG03ACA1 FL2H serial=4683K23KF> SATA S1 2 ( 932G) JBOD <TOSHIBA MG03ACA1 FL2H serial=4684KQ8KF> SATA S2 3 ( 932G) JBOD <TOSHIBA MG03ACA1 FL2H serial=4684KQ8IF> SATA S3
Same stuff here. camcontrol identify doesn't lead to topology change, but smartctl -a definitely does. This is reproducible on a fresh 11.[ and 10.3. Controller: LSI MegaRAID SAS 9341-4i Firmware: Product Name : LSI MegaRAID SAS 9341-4i Serial No : SV62122829 FW Package Build: 24.2.1-0021 Console log: [root@san1:/etc]# zpool status pool: esx state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0 in 5h40m with 0 errors on Tue Jun 28 03:27:20 2016 config: NAME STATE READ WRITE CKSUM esx ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da3 ONLINE 0 0 0 da4 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 da6 ONLINE 0 0 0 da7 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 da9 ONLINE 0 0 0 da10 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 da12 ONLINE 0 0 0 da13 ONLINE 0 0 0 mirror-4 ONLINE 0 0 0 da5 ONLINE 0 0 0 da8 ONLINE 0 0 0 mirror-5 ONLINE 0 0 0 da11 ONLINE 0 0 0 da14 ONLINE 0 0 0 mirror-6 ONLINE 0 0 0 da15 ONLINE 0 0 0 da16 ONLINE 0 0 0 errors: No known data errors pool: gamestop state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: resilvered 729M in 0h0m with 0 errors on Thu Jul 7 04:02:53 2016 config: NAME STATE READ WRITE CKSUM gamestop ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da0 ONLINE 0 0 0 da17 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 da18 ONLINE 0 0 0 da19 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 da20 ONLINE 0 0 0 da21 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 da22 ONLINE 0 0 0 da23 ONLINE 0 0 0 errors: No known data errors pool: zroot state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0 in 0h1m with 0 errors on Mon Jun 27 21:21:40 2016 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da1p3 ONLINE 0 0 0 da2p3 ONLINE 0 0 0 errors: No known data errors [root@san1:/etc]# smartctl -a /dev/da2 smartctl 6.5 2016-05-07 r4318 [FreeBSD 11.0-ALPHA5 amd64] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Device Model: ST2000NX0253 Serial Number: S460ED5C LU WWN Device Id: 5 000c50 08ca8b9aa Firmware Version: SN02 User Capacity: 2,000,398,934,016 bytes [2.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 7200 rpm Form Factor: 2.5 inches Device is: Not in smartctl database [for details use: -P showall] ATA Version is: ACS-3 (minor revision not indicated) SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Thu Jul 7 03:04:27 2016 MSK SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART Status command failed: Input/output error SMART overall-health self-assessment test result: PASSED Warning: This result is based on an Attribute check. General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 609) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 330) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x50bd) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 081 063 044 Pre-fail Always - 134895728 3 Spin_Up_Time 0x0003 092 092 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 22 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 8 7 Seek_Error_Rate 0x000f 072 060 030 Pre-fail Always - 19270762 9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 624 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 22 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 068 067 040 Old_age Always - 32 (Min/Max 20/33) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 7 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 0 193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 1280 194 Temperature_Celsius 0x0022 032 040 000 Old_age Always - 32 (0 12 0 0 0) 195 Hardware_ECC_Recovered 0x001a 005 004 000 Old_age Always - 134895728 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 Read SMART Log Directory failed: Input/output error Read SMART Error Log failed: Input/output error Read SMART Self-test Log failed: Input/output error Read SMART Selective Self-test Log failed: Input/output error [root@san1:/etc]# dmesg Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-ALPHA5 #1: Thu Jun 30 22:56:33 MSK 2016 emz@san1:/usr/obj/usr/src/sys/SAN amd64 [...] da2 at mrsas0 bus 1 scbus1 target 7 lun 0 da2: <ATA ST2000NX0253 SN02> s/n S460ED5C detached GEOM_MIRROR: Device swap: provider da2p2 disconnected. [root@san1:/etc]# camcontrol rescan 1:7:0 Re-scan of 1:7:0 was successful [root@san1:/etc]# zpool status pool: esx state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0 in 5h40m with 0 errors on Tue Jun 28 03:27:20 2016 config: NAME STATE READ WRITE CKSUM esx ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da3 ONLINE 0 0 0 da4 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 da6 ONLINE 0 0 0 da7 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 da9 ONLINE 0 0 0 da10 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 da12 ONLINE 0 0 0 da13 ONLINE 0 0 0 mirror-4 ONLINE 0 0 0 da5 ONLINE 0 0 0 da8 ONLINE 0 0 0 mirror-5 ONLINE 0 0 0 da11 ONLINE 0 0 0 da14 ONLINE 0 0 0 mirror-6 ONLINE 0 0 0 da15 ONLINE 0 0 0 da16 ONLINE 0 0 0 errors: No known data errors pool: gamestop state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: resilvered 729M in 0h0m with 0 errors on Thu Jul 7 04:02:53 2016 config: NAME STATE READ WRITE CKSUM gamestop ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da0 ONLINE 0 0 0 da17 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 da18 ONLINE 0 0 0 da19 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 da20 ONLINE 0 0 0 da21 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 da22 ONLINE 0 0 0 da23 ONLINE 0 0 0 errors: No known data errors pool: zroot state: DEGRADED status: One or more devices has been removed by the administrator. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Online the device using 'zpool online' or replace the device with 'zpool replace'. scan: scrub repaired 0 in 0h1m with 0 errors on Mon Jun 27 21:21:40 2016 config: NAME STATE READ WRITE CKSUM zroot DEGRADED 0 0 0 mirror-0 DEGRADED 0 0 0 da1p3 ONLINE 0 0 0 17040910893353730136 REMOVED 0 0 0 was /dev/da2p3 errors: No known data errors [root@san1:/etc]# zpool online zroot da2p3 [root@san1:/etc]# zpool status pool: esx state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0 in 5h40m with 0 errors on Tue Jun 28 03:27:20 2016 config: NAME STATE READ WRITE CKSUM esx ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da3 ONLINE 0 0 0 da4 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 da6 ONLINE 0 0 0 da7 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 da9 ONLINE 0 0 0 da10 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 da12 ONLINE 0 0 0 da13 ONLINE 0 0 0 mirror-4 ONLINE 0 0 0 da5 ONLINE 0 0 0 da8 ONLINE 0 0 0 mirror-5 ONLINE 0 0 0 da11 ONLINE 0 0 0 da14 ONLINE 0 0 0 mirror-6 ONLINE 0 0 0 da15 ONLINE 0 0 0 da16 ONLINE 0 0 0 errors: No known data errors pool: gamestop state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: resilvered 729M in 0h0m with 0 errors on Thu Jul 7 04:02:53 2016 config: NAME STATE READ WRITE CKSUM gamestop ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da0 ONLINE 0 0 0 da17 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 da18 ONLINE 0 0 0 da19 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 da20 ONLINE 0 0 0 da21 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 da22 ONLINE 0 0 0 da23 ONLINE 0 0 0 errors: No known data errors pool: zroot state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: resilvered 16K in 0h0m with 0 errors on Thu Jul 7 03:05:18 2016 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da1p3 ONLINE 0 0 0 da2p3 ONLINE 0 0 0 errors: No known data errors [root@san1:/etc]#
And I also have an error in the LSI log: 07/07/16 3:04:26: C0:EVT#04219-07/07/16 3:04:26: 113=Unexpected sense: PD 07(e0x04/s11) Path 50030480091b514b, CDB: 00 00 00 00 00 00, Sense: 1/00/1d Ъ7/07/16 3:04:26: C0:Raw Sense for PD 7: 70 00 01 00 00 00 00 0a 00 00 00 00 00 1d 00 00 00 00 Exit Code: 0x00 does it mean it's a controller/firmware issue ?
Can't reproduse anymore after flashing the last available firmware, 24.15.0-0016 for my MegaRAID SAS 9341-4i. Looks like this isn't a FreeeBSD driver bug.