Bug 268229 - Logical drives regression with mpr(4) driver
Summary: Logical drives regression with mpr(4) driver
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.4-RELEASE
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-bugs (Nobody)
URL:
Keywords: needs-qa, regression
: 268230 (view as bug list)
Depends on:
Blocks:
 
Reported: 2022-12-07 16:42 UTC by Robert Blayzor
Modified: 2023-04-17 07:38 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Blayzor 2022-12-07 16:42:57 UTC
This issue also appears in FreeBSD 13.1

Regression is seen when updating FreeBSD 12.3 GENERIC kernel to 12.4 kernel.

Logical drives on mpr device driver (LSI Fusion controller 3008i) fails to discover logical drive. If logical drive is removed and drives are in JBOD, drives appear normally. 

This is NOT an issue in FreeBSD 12.3 and updating to 12.4 causes the server to fail boot as it cannot mount as it discovers no drives. Rolling back to 12.3-GENERIC resolves the problem.

In FreeBSD 12.3/12.4/13.1 the controller is discovered:

mpr0: <Avago Technologies (LSI) SAS3008> port 0xe000-0xe0ff mem 0xfb240000-0xfb24ffff,0xfb200000-0xfb23ffff irq 26 at device 0.0 on pci2
mpr0: Firmware: 10.00.03.00, Driver: 23.00.00.00-fbsd
mpr0: IOCCapabilities: 6985c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR,MSIXIndex,FastPath,RDPQArray>
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x0009> enclosureHandle<0x0001> slot 0
mpr0: At enclosure level 0 and connector name (    )
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x000a> enclosureHandle<0x0001> slot 1
mpr0: At enclosure level 0 and connector name (    )


However ONLY in 12.3 we see the logical drive is discovered and system boots normally:

da0 at mpr0 bus 0 scbus0 target 0 lun 0
da0: <LSI Logical Volume 3000> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number 739763315135195629
da0: 150.000MB/s transfers
da0: Command Queueing enabled
da0: 142097MB (291014656 512 byte sectors)


When booting with 12.4/13.1 this logical drive is NOT discovered, system cannot boot.
Comment 1 Robert Blayzor 2022-12-07 18:01:39 UTC
*** Bug 268230 has been marked as a duplicate of this bug. ***
Comment 2 Robert Blayzor 2022-12-14 04:26:16 UTC
Could this issue be related or part of:  FreeBSD-EN-22:26

Cannot verify at this time if this LVD issue exists in 13.1-P3 but did exist in 13.1-RELEASE

This issue does also exist in 12.4-RELEASE

So did the cam change in 13.1-RELEASE make it to 12.4-RELEASE, but the patch did not?

Or was there mpr driver chances between 12.3 and 12.4. Release notes do not state so and kernel boot messages say the driver version is the same..
Comment 3 Robert Blayzor 2022-12-14 04:37:55 UTC
If not CAM, suspect commit ... ?

https://cgit.freebsd.org/src/commit/?id=9d842d84f49af6d4aacaea694dcaea4173522684
Comment 4 cin_nev 2023-04-17 07:38:56 UTC
RELENG-12.3 not support, but I can not upgrade to RELENG-12.4, because I have got same issue after upgrade to RELENG-12.4. What can I do to solve this issue?

Information about hardware (commands ran on RELENG-12.3):
root@host:~ # mprutil show all
Adapter:
mpr0 Adapter:
       Board Name: Asus SAS3008
   Board Assembly:
        Chip Name: LSISAS3008
    Chip Revision: ALL
    BIOS Revision: 8.35.00.00
Firmware Revision: 15.00.04.00
  Integrated RAID: yes
         SATA NCQ: ENABLED
 PCIe Width/Speed: x8 (8.0 GB/sec)
        IOC Speed: Full
      Temperature: 53 C

PhyNum  CtlrHandle  DevHandle  Disabled  Speed   Min    Max    Device
0       0001        0009       N         6.0     3.0    12     SAS Initiator
1       0002        000a       N         6.0     3.0    12     SAS Initiator
2       0003        000b       N         6.0     3.0    12     SAS Initiator
3       0004        000c       N         6.0     3.0    12     SAS Initiator
4                              N                 3.0    12     SAS Initiator
5                              N                 3.0    12     SAS Initiator
6                              N                 3.0    12     SAS Initiator
7                              N                 3.0    12     SAS Initiator

Devices:
B____T    SAS Address      Handle  Parent    Device        Speed Enc  Slot  Wdt
00   02   4433221100000000 0009    0001      SATA Target   6.0   0001 00    1
00   03   4433221101000000 000a    0002      SATA Target   6.0   0001 01    1
00   04   4433221102000000 000b    0003      SATA Target   6.0   0001 02    1
00   05   4433221103000000 000c    0004      SATA Target   6.0   0001 03    1

Enclosures:
Slots      Logical ID     SEPHandle  EncHandle    Type
  08    500112f950000f70               0001     Direct Attached SGPIO

Expanders:
NumPhys   SAS Address     DevHandle   Parent  EncHandle  SAS Level


root@host:~ # dmesg | grep mpr0
mpr0: <Avago Technologies (LSI) SAS3008> port 0x6000-0x60ff mem 0x94240000-0x9424ffff,0x94200000-0x9423ffff irq 16 at device 0.0 on pci1
mpr0: Firmware: 15.00.04.00, Driver: 23.00.00.00-fbsd
mpr0: IOCCapabilities: 6985c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR,MSIXIndex,FastPath,RDPQArray>
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x0009> enclosureHandle<0x0001> slot 0
mpr0: At enclosure level 0 and connector name (    )
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x000a> enclosureHandle<0x0001> slot 1
mpr0: At enclosure level 0 and connector name (    )
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x000b> enclosureHandle<0x0001> slot 2
mpr0: At enclosure level 0 and connector name (    )
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x000c> enclosureHandle<0x0001> slot 3
mpr0: At enclosure level 0 and connector name (    )
da0 at mpr0 bus 0 scbus0 target 0 lun 0
da1 at mpr0 bus 0 scbus0 target 1 lun 0



root@host:~ # dmesg | grep da0
Trying to mount root from ufs:/dev/da0p2 [rw]...
da0 at mpr0 bus 0 scbus0 target 0 lun 0
da0: <LSI Logical Volume 3000> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number 3978164406233064500
da0: 150.000MB/s transfers
da0: Command Queueing enabled
da0: 456809MB (935544832 512 byte sectors)