Bug 244335 - [mfip] Suspecting a problem with mfi/mifp(?) Megaraid SAS driver in 12.1 w/ FTS PRAID CP400i controller
Summary: [mfip] Suspecting a problem with mfi/mifp(?) Megaraid SAS driver in 12.1 w/ F...
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.1-RELEASE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-bugs mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-02-23 11:10 UTC by Palle Girgensohn
Modified: 2020-02-23 11:10 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Palle Girgensohn freebsd_committer 2020-02-23 11:10:25 UTC
Hi!

We updated a Fujisti server Friday morning from FreeBSD 11.2 -> 12.1 and we are seeing problems with the ZFS storage.

It as rather large, ~ 50 TB, ZFS volume. After upgrading, it complains about chksum errors and has rebooted spontaneous twice in ~48 hours.

Since it just happend after upgrading the OS, and since it happens on several disks, I primarily suspect there is a software error. That is because that is all that has changed. I have similar reports from another identical machine that was also upgraded, but I canont verify at the moment. It could also be the controller of course, and in that case triggered by the intentional reboots as part of the upgrade, but I pursuing both options at the moment. I the other machine actually shows the same symptoms, a software problem is even more likely.

It is a FTS PRAID CP400i controller, identified in FreeBSD to use the mfi driver. It is configured as JBOD, no RAID at all, and *not* using gpart.

Any ideas or tips will be greatly appreciated.

Palle


$ grep mfi /boot/loader.conf 
mfip_load="yes"


$ freebsd-version -ku
12.1-RELEASE-p2
12.1-RELEASE-p2


$ uname -iKmoprsUv
FreeBSD 12.1-RELEASE-p2 FreeBSD 12.1-RELEASE-p2 GENERIC  amd64 amd64 GENERIC 1201000 1201000


$ zpool status -v
  pool: tank
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Sun Feb 23 11:36:47 2020
	126G scanned at 117M/s, 27,1G issued at 25,2M/s, 15,8T total
	0 resilvered, 0,17% done, 7 days 13:48:54 to go
config:

	NAME                    STATE     READ WRITE CKSUM
	tank                    DEGRADED     0     0     0
	  raidz2-0              DEGRADED     0     0     0
	    mfisyspd0           ONLINE       0     0    12
	    mfisyspd1           DEGRADED     0     0    16  too many errors
	    mfisyspd2           DEGRADED     0     0    34  too many errors
	    mfisyspd3           DEGRADED     0     0    27  too many errors
	    mfisyspd4           DEGRADED     0     0    59  too many errors
	    mfisyspd5           DEGRADED     0     0    40  too many errors
	  raidz2-1              DEGRADED     0     0     0
	    mfisyspd6           ONLINE       0     0    23
	    spare-1             DEGRADED     0     0   151
	      mfisyspd7         DEGRADED     0     0     0  too many errors
	      mfisyspd11        ONLINE       0     0     0
	    mfisyspd8           ONLINE       0     0    11
	    mfisyspd9           DEGRADED     0     0    40  too many errors
	    mfisyspd10          DEGRADED     0     0    38  too many errors
	logs
	  mirror-2              ONLINE       0     0     0
	    mfisyspd12p4        ONLINE       0     0     0
	    mfisyspd13p4        ONLINE       0     0     0
	cache
	  mfisyspd12p5          ONLINE       0     0     0
	  mfisyspd13p5          ONLINE       0     0     0
	spares
	  11621498950123571546  INUSE     was /dev/mfisyspd11

errors: No known data errors



from /var/log/messages:

Feb 22 14:07:41 hostname kernel: mfisyspd9: hard error cmd=read 2855317293-2855317844
Feb 22 14:07:41 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e80198, status=0x3c, scsi_status=0
Feb 22 14:07:41 hostname kernel: mfi0: sense error 112, sense_key 5, asc 32, ascq 0
Feb 22 14:07:41 hostname kernel: mfisyspd6: hard error cmd=read 2855318195-2855318746
Feb 22 14:07:42 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e83fc0, status=0x3c, scsi_status=0
Feb 22 14:07:42 hostname kernel: mfi0: sense error 114, sense_key 4, asc 207, ascq 0
Feb 22 14:07:42 hostname kernel: mfisyspd5: hard error cmd=read 6047123871-6047124388
Feb 22 14:07:42 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e802a8, status=0x3c, scsi_status=0
Feb 22 14:07:42 hostname kernel: mfi0: sense error 114, sense_key 4, asc 207, ascq 0
Feb 22 14:07:42 hostname kernel: mfisyspd2: hard error cmd=read 6047123872-6047124389
Feb 22 14:07:43 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e80330, status=0x3c, scsi_status=0
Feb 22 14:07:43 hostname kernel: mfi0: sense error 114, sense_key 4, asc 207, ascq 0
Feb 22 14:07:43 hostname kernel: mfisyspd8: hard error cmd=read 2906458877-2906459428
Feb 22 14:07:43 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e82f48, status=0x3c, scsi_status=0
Feb 22 14:07:43 hostname kernel: mfi0: sense error 114, sense_key 4, asc 0, ascq 0
Feb 22 14:07:43 hostname kernel: mfisyspd8: hard error cmd=read 2906459429-2906459980
Feb 22 14:07:43 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e82420, status=0x3c, scsi_status=0
Feb 22 14:07:43 hostname kernel: mfi0: sense error 112, sense_key 5, asc 32, ascq 0
Feb 22 14:07:43 hostname kernel: mfisyspd8: hard error cmd=read 2906459981-2906460532
Feb 22 14:07:43 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e83960, status=0x3c, scsi_status=0
Feb 22 14:07:43 hostname kernel: mfi0: sense error 114, sense_key 4, asc 207, ascq 0
Feb 22 14:07:43 hostname kernel: mfisyspd0: hard error cmd=read 412266548-412267099
Feb 22 14:07:43 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e80220, status=0x3c, scsi_status=0
Feb 22 14:07:43 hostname kernel: mfi0: sense error 112, sense_key 5, asc 32, ascq 0
Feb 22 14:07:43 hostname kernel: mfisyspd0: hard error cmd=read 412267100-412267651
Feb 22 14:07:49 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e815c8, status=0x3c, scsi_status=0
Feb 22 14:07:49 hostname kernel: mfi0: sense error 114, sense_key 4, asc 207, ascq 0
Feb 22 14:07:49 hostname kernel: mfisyspd8: hard error cmd=read 2959727577-2959728128
Feb 22 14:07:49 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e81ed0, status=0x3c, scsi_status=0
Feb 22 14:07:49 hostname kernel: mfi0: sense error 112, sense_key 5, asc 32, ascq 0
Feb 22 14:07:49 hostname kernel: mfisyspd1: hard error cmd=read 7610749043-7610749594
Feb 22 14:07:49 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e82f48, status=0x3c, scsi_status=0
Feb 22 14:07:49 hostname kernel: mfi0: sense error 114, sense_key 4, asc 0, ascq 0
Feb 22 14:07:49 hostname kernel: mfisyspd3: hard error cmd=read 4882180734-4882181285
Feb 22 14:07:51 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e82b90, status=0x3c, scsi_status=0
Feb 22 14:07:51 hostname kernel: mfi0: sense error 114, sense_key 4, asc 207, ascq 0
Feb 22 14:07:51 hostname kernel: mfisyspd9: hard error cmd=read 1041218706-1041219257
Feb 22 14:07:51 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e80c38, status=0x3c, scsi_status=0
Feb 22 14:07:51 hostname kernel: mfi0: sense error 114, sense_key 4, asc 207, ascq 0
Feb 22 14:07:51 hostname kernel: mfisyspd9: hard error cmd=read 1041219258-1041219809
Feb 22 14:07:52 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e837c8, status=0x3c, scsi_status=0
Feb 22 14:07:52 hostname kernel: mfi0: sense error 112, sense_key 5, asc 32, ascq 0
Feb 22 14:07:52 hostname kernel: mfisyspd6: hard error cmd=read 2960038937-2960039488
Feb 22 14:07:52 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e81e48, status=0x3c, scsi_status=0
Feb 22 14:07:52 hostname kernel: mfi0: sense error 114, sense_key 4, asc 0, ascq 0
Feb 22 14:07:52 hostname kernel: mfisyspd6: hard error cmd=read 2960039489-2960040040
Feb 22 14:07:52 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e82d28, status=0x3c, scsi_status=0
Feb 22 14:07:52 hostname kernel: mfi0: sense error 114, sense_key 4, asc 207, ascq 0
Feb 22 14:07:52 hostname kernel: mfisyspd6: hard error cmd=read 2960040041-2960040592
Feb 22 14:08:13 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e83960, status=0x3c, scsi_status=0
Feb 22 14:08:13 hostname kernel: mfi0: sense error 114, sense_key 4, asc 207, ascq 0
Feb 22 14:08:13 hostname kernel: mfisyspd7: hard error cmd=read 2906939820-2906940371
Feb 22 14:08:13 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e80908, status=0x3c, scsi_status=0
Feb 22 14:08:13 hostname kernel: mfi0: sense error 112, sense_key 5, asc 32, ascq 0
Feb 22 14:08:13 hostname kernel: mfisyspd1: hard error cmd=read 412681365-412681916
Feb 22 14:08:42 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e82860, status=0x3c, scsi_status=0
Feb 22 14:08:42 hostname kernel: mfi0: sense error 114, sense_key 4, asc 207, ascq 0
Feb 22 14:08:42 hostname kernel: mfisyspd9: hard error cmd=read 2857339961-2857340492
Feb 22 14:08:47 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e82ec0, status=0x3c, scsi_status=0
Feb 22 14:08:47 hostname kernel: mfi0: sense error 114, sense_key 4, asc 0, ascq 0
Feb 22 14:08:47 hostname kernel: mfisyspd7: hard error cmd=read 1041524253-1041524786
Feb 22 14:09:01 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e82068, status=0x3c, scsi_status=0
Feb 22 14:09:01 hostname kernel: mfi0: sense error 112, sense_key 5, asc 32, ascq 0
Feb 22 14:09:01 hostname kernel: mfisyspd1: hard error cmd=read 7619627390-7619627941
Feb 22 14:09:13 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e838d8, status=0x3c, scsi_status=0
Feb 22 14:09:13 hostname kernel: mfi0: sense error 114, sense_key 4, asc 0, ascq 0
Feb 22 14:09:13 hostname kernel: mfisyspd10: hard error cmd=read 2960045475-2960046026
Feb 22 14:10:27 hostname kernel: mfi0: I/O error, cmd=0xfffffe00a8e80e58, status=0x3c, scsi_status=0
Feb 22 14:10:27 hostname kernel: mfi0: sense error 114, sense_key 4, asc 207, ascq 0
Feb 22 14:10:27 hostname kernel: mfisyspd7: hard error cmd=read 2906360948-2906361499







$ dmesg| grep mfi
mfi0: <Fury> port 0x6000-0x60ff mem 0xc7b00000-0xc7b0ffff,0xc7a00000-0xc7afffff irq 26 at device 0.0 numa-domain 0 on pci3
mfi0: Using MSI
mfi0: Megaraid SAS driver Ver 4.23 
mfi0: FW MaxCmds = 240, limiting to 128
mfi0: MaxCmd = 240, Drv MaxCmd = 128, MaxSgl = 70, state = 0xb73c00f0
mfi0: 1923 (635692256s/0x0020/info) - Shutdown command received from host
mfip0: mfi0: 1924 (boot + 9s/0x0020/info) - Firmware initialization started (PCI ID 005f/1000/1211/1734)
mfi0: 1925 (boot + 9s/0x0020/info) - Firmware version 4.250.01-3869
mfi0: 1926 (boot + 11s/0x0020/info) - Package version 24.5.0-0031
mfi0: 1927 (boot + 11s/0x0020/info) - Board Revision         
mfi0: 1928 (boot + 59s/0x0004/info) - Enclosure (SES) discovered on PD 07(c Port 0 - 3/p1)
mfi0: 1929 (boot + 60s/0x0004/info) - Enclosure PD 07(c Port 0 - 3/p1) communication restored
<SCSI Passthrough Bus> numa-domain 0 on mfi0
mfi0: 1930 (boot + 60s/0x0002/info) - Inserted: Encl PD 07
mfi1: <Invader> port 0xf000-0xf0ff mem 0xfbe00000-0xfbe0ffff,0xfbd00000-0xfbdfffff irq 50 at device 0.0 numa-domain 1 on pci10
mfi1: Using MSI
mfi1: Megaraid SAS driver Ver 4.23 
mfi1: FW MaxCmds = 928, limiting to 128
mfi1: MaxCmd = 928, Drv MaxCmd = 128, MaxSgl = 70, state = 0xb75003a0
mfi1: cmd_tbolt 0xfffff80004c37c00 has invalid sync_cmd_idx=128 - skipping
mfip1: <SCSI Passthrough Bus> numa-domain 1 on mfi1
mfisyspd0 numa-domain 0 on mfi0
mfisyspd0: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 8)
mfisyspd0:  SYSPD volume attached
mfisyspd1 numa-domain 0 on mfi0
mfisyspd1: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 9)
mfisyspd1:  SYSPD volume attached
mfisyspd2 numa-domain 0 on mfi0
mfisyspd2: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 10)
mfisyspd2:  SYSPD volume attached
mfisyspd3 numa-domain 0 on mfi0
mfisyspd3: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 11)
mfisyspd3:  SYSPD volume attached
mfisyspd4 numa-domain 0 on mfi0
mfisyspd4: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 12)
mfisyspd4:  SYSPD volume attached
mfisyspd5 numa-domain 0 on mfi0
mfisyspd5: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 13)
mfisyspd5:  SYSPD volume attached
mfisyspd6 numa-domain 0 on mfi0
mfisyspd6: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 14)
mfisyspd6:  SYSPD volume attached
mfisyspd7 numa-domain 0 on mfi0
mfisyspd7: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 15)
mfisyspd7:  SYSPD volume attached
mfisyspd8 numa-domain 0 on mfi0
mfisyspd8: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 16)
mfisyspd8:  SYSPD volume attached
mfisyspd9 numa-domain 0 on mfi0
mfisyspd9: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 17)
mfisyspd9:  SYSPD volume attached
mfisyspd10 numa-domain 0 on mfi0
mfisyspd10: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 18)
mfisyspd10:  SYSPD volume attached
GEOM: mfisyspd0: the primary GPT table is corrupt or invalid.
GEOM: mfisyspd0: using the secondary instead -- recovery strongly advised.
mfisyspd11 numa-domain 0 on mfi0
mfisyspd11: 3815447MB (7814037168 sectors) SYSPD volume (deviceid: 19)
mfisyspd11:  SYSPD volume attached
mfi0: 1931 (boot + 60s/0x0002/info) - Inserted: PD 07(c Port 0 - 3/p1) Info: enclPd=07, scsiType=d, portMap=00, sasAddr=500605b0000272bd,0000000000000000
mfi0: 1932 (boot + 60s/0x0002/info) - Inserted: PD 08(e0x07/s3)
mfisyspd12 numa-domain 1 on mfi1
mfisyspd12: 114473MB (234441648 sectors) SYSPD volume (deviceid: 0)
mfisyspd12:  SYSPD volume attached
mfisyspd13 numa-domain 1 on mfi1
mfisyspd13: 114473MB (234441648 sectors) SYSPD volume (deviceid: 1)
mfisyspd13:  SYSPD volume attached
mfi0: 1933 (boot + 60s/0x0002/info) - Inserted: PD 08(e0x07/s3) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=5000c5006308fbed,0000000000000000
mfi0: 1934 (boot + 60s/0x0002/info) - Inserted: PD 09(e0x07/s5)
mfi0: 1935 (boot + 60s/0x0002/info) - Inserted: PD 09(e0x07/s5) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=5000c5006309019d,0000000000000000
mfi0: 1936 (boot + 60s/0x0002/info) - Inserted: PD 0a(e0x07/s1)
mfi0: 1937 (boot + 60s/0x0002/info) - Inserted: PD 0a(e0x07/s1) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=5000c5006308fd8d,0000000000000000
mfi0: 1938 (boot + 60s/0x0002/info) - Inserted: PD 0b(e0x07/s0)
mfi0: 1939 (boot + 60s/0x0002/info) - Inserted: PD 0b(e0x07/s0) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=5000c50063090d45,0000000000000000
mfi0: 1940 (boot + 60s/0x0002/info) - Inserted: PD 0c(e0x07/s4)
mfi0: 1941 (boot + 60s/0x0002/info) - Inserted: PD 0c(e0x07/s4) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=5000c5006308f981,0000000000000000
mfi0: 1942 (boot + 60s/0x0002/info) - Inserted: PD 0d(e0x07/s2)
mfi0: 1943 (boot + 60s/0x0002/info) - Inserted: PD 0d(e0x07/s2) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=5000c500630910c1,0000000000000000
mfi0: 1944 (boot + 60s/0x0002/info) - Inserted: PD 0e(e0x07/s8)
mfi0: 1945 (boot + 60s/0x0002/info) - Inserted: PD 0e(e0x07/s8) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=50000c0f01f7efb6,0000000000000000
mfi0: 1946 (boot + 60s/0x0002/info) - Inserted: PD 0f(e0x07/s7)
mfi0: 1947 (boot + 60s/0x0002/info) - Inserted: PD 0f(e0x07/s7) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=5000c50083c1a0a1,0000000000000000
mfi0: 1948 (boot + 60s/0x0002/info) - Inserted: PD 10(e0x07/s6)
mfi0: 1949 (boot + 60s/0x0002/info) - Inserted: PD 10(e0x07/s6) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=5000c50083c19bb9,0000000000000000
mfi0: 1950 (boot + 60s/0x0002/info) - Inserted: PD 11(e0x07/s11)
mfi0: 1951 (boot + 60s/0x0002/info) - Inserted: PD 11(e0x07/s11) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=5000c50083c19e19,0000000000000000
mfi0: 1952 (boot + 60s/0x0002/info) - Inserted: PD 12(e0x07/s10)
mfi0: 1953 (boot + 60s/0x0002/info) - Inserted: PD 12(e0x07/s10) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=50000c0f01e251e2,0000000000000000
mfi0: 1954 (boot + 60s/0x0002/info) - Inserted: PD 13(e0x07/s9)
mfi0: 1955 (boot + 60s/0x0002/info) - Inserted: PD 13(e0x07/s9) Info: enclPd=07, scsiType=0, portMap=00, sasAddr=50000c0f0136664e,0000000000000000
mfi0: 1956 (635692335s/0x0020/info) - Time established as 02/22/20 13:12:15; (66 seconds since power on)
GEOM: mfisyspd1: the primary GPT table is corrupt or invalid.
GEOM: mfisyspd1: using the secondary instead -- recovery strongly advised.
mfi1: 1440 (635692257s/0x0020/info) - Shutdown command received from host
mfi1: 1441 (boot + 9s/0x0020/info) - Firmware initialization started (PCI ID 005d/1000/1212/1734)
mfi1: 1442 (boot + 9s/0x0020/info) - Firmware version 4.250.00-3868
mfi1: 1443 (boot + 10s/0x0008/info) - Battery Present
mfi1: 1444 (boot + 10s/0x0020/info) - Package version 24.5.0-0031
mfi1: 1445 (boot + 10s/0x0020/info) - Board Revision         
mfi1: 1446 (boot + 14s/0x0008/info) - Battery charge complete
mfi1: 1447 (boot + 14s/0x0008/info) - Battery temperature is normal
mfi1: 1448 (boot + 46s/0x0002/info) - Inserted: PD 00(e0xfc/s0)
mfi1: 1449 (boot + 46s/0x0002/info) - Inserted: PD 00(e0xfc/s0) Info: enclPd=fc, scsiType=0, portMap=00, sasAddr=4433221102000000,0000000000000000
mfi1: 1450 (boot + 46s/0x0002/info) - Inserted: PD 01(e0xfc/s1)
mfi1: 1451 (boot + 46s/0x0002/info) - Inserted: PD 01(e0xfc/s1) Info: enclPd=fc, scsiType=0, portMap=01, sasAddr=4433221103000000,0000000000000000
mfi1: 1452 (635692353s/0x0020/info) - Time established as 02/22/20 13:12:33; (82 seconds since power on)
GEOM: mfisyspd2: the primary GPT table is corrupt or invalid.
GEOM: mfisyspd2: using the secondary instead -- recovery strongly advised.
GEOM: mfisyspd3: the primary GPT table is corrupt or invalid.
GEOM: mfisyspd3: using the secondary instead -- recovery strongly advised.
ses0 at mfi0 bus 0 scbus0 target 7 lun 0
pass1 at mfi0 bus 0 scbus0 target 8 lun 0
pass2 at mfi0 bus 0 scbus0 target 9 lun 0
pass3 at mfi0 bus 0 scbus0 target 10 lun 0
pass4 at mfi0 bus 0 scbus0 target 11 lun 0
pass5 at mfi0 bus 0 scbus0 target 12 lun 0
GEOM: mfisyspd4: the primary GPT table is corrupt or invalid.
pass6 at mfi0 bus 0 scbus0 target 13 lun 0
pass7 at mfi0 bus 0 scbus0 target 14 lun 0
pass8 at mfi0 bus 0 scbus0 target 15 lun 0
pass9 at mfi0 bus 0 scbus0 target 16 lun 0
pass10 at mfi0 bus 0 scbus0 target 17 lun 0
pass11 at mfi0 bus 0 scbus0 target 18 lun 0
pass12 at mfi0 bus 0 scbus0 target 19 lun 0
pass13 at mfi1 bus 0 scbus11 target 0 lun 0
pass14 at mfi1 bus 0 scbus11 target 1 lun 0
GEOM: mfisyspd4: using the secondary instead -- recovery strongly advised.
GEOM: mfisyspd5: the primary GPT table is corrupt or invalid.
GEOM: mfisyspd5: using the secondary instead -- recovery strongly advised.
GEOM: mfisyspd0: the primary GPT table is corrupt or invalid.
GEOM: mfisyspd0: using the secondary instead -- recovery strongly advised.
GEOM: mfisyspd1: the primary GPT table is corrupt or invalid.
GEOM: mfisyspd1: using the secondary instead -- recovery strongly advised.
GEOM: mfisyspd2: the primary GPT table is corrupt or invalid.
GEOM: mfisyspd2: using the secondary instead -- recovery strongly advised.
GEOM: mfisyspd3: the primary GPT table is corrupt or invalid.
GEOM: mfisyspd3: using the secondary instead -- recovery strongly advised.
GEOM: mfisyspd4: the primary GPT table is corrupt or invalid.
GEOM: mfisyspd4: using the secondary instead -- recovery strongly advised.
GEOM: mfisyspd5: the primary GPT table is corrupt or invalid.
GEOM: mfisyspd5: using the secondary instead -- recovery strongly advised.