3ware tws driver has TWS_MAX_NUM_LUNS set to 16 but this is not enough. I think the maximum LUN's should be 255. I have the issue on my 3ware 9750-8i controller with a RAID array spanned into 21 volumes. Only volume the first 16 volumes are detected. CAMCONTROL DEVLIST <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun 0 (pass0,da0) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun 1 (pass1,da1) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun 2 (pass2,da2) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun 3 (pass3,da3) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun 4 (pass4,da4) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun 5 (pass5,da5) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun 6 (pass6,da6) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun 7 (pass7,da7) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun 8 (pass8,da8) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun 9 (pass9,da9) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun a (pass10,da10) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun b (pass11,da11) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun c (pass12,da12) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun d (pass13,da13) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun e (pass14,da14) <LSI 9750-8i DISK 5.12> at scbus0 target 0 lun f (pass15,da15) DMESG (probe0:tws0:0:0:10): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:tws0:0:0:10): CAM status: Invalid Lun (probe0:tws0:0:0:10): Error 22, Unretryable error (probe0:tws0:0:0:11): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:tws0:0:0:11): CAM status: Invalid Lun (probe0:tws0:0:0:11): Error 22, Unretryable error (probe0:tws0:0:0:12): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:tws0:0:0:12): CAM status: Invalid Lun (probe0:tws0:0:0:12): Error 22, Unretryable error (probe0:tws0:0:0:13): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:tws0:0:0:13): CAM status: Invalid Lun (probe0:tws0:0:0:13): Error 22, Unretryable error (probe0:tws0:0:0:14): INQUIRY. CDB: 12 00 00 00 24 00 (probe0:tws0:0:0:14): CAM status: Invalid Lun (probe0:tws0:0:0:14): Error 22, Unretryable error In tws(4) driver sources I see: if (ccb_h->target_lun >= TWS_MAX_NUM_LUNS) { ... ccb_h->status |= CAM_LUN_INVALID; } Where TWS_MAX_NUM_LUNS is 16. So it is clear why it does not work. The only question is whether a higher value could provoke issues on other controllers. I changed the value to 32 (to be safe) and it works like a charm.
I don't know what the actual limit for this hardware is, but I'll find out. In the meantime I'll change it to 32. This card is the only thing the tws driver supports, so changes shouldn't affect anything else.
A commit references this bug: Author: jpaetzel Date: Wed May 17 13:22:13 UTC 2017 New revision: 318401 URL: https://svnweb.freebsd.org/changeset/base/318401 Log: Increase the number of LUNs this hardware can support. Experimentally we know this value works, but the hardware may support an even higher value. PR: 213876 Reported by: J.Catrysse@proximedia.be MFC after: 1 week Changes: head/sys/dev/tws/tws.h
Committed 32 LUN support to HEAD, I'll merge it back to 11 so it can be a part of 11.1-RELEASE in a week. I'll merge it to stable/10 as well.
Hello Josh, TWA has the same issue... maybe you could change that one too? (In reply to Josh Paetzel from comment #3)
I'm less inclined to change twa. It's a generation older than tws and the driver supports a much wider range of cards. My feeling is if it hasn't been as issue yet we can ride it out.
I had the issue on one of our servers, is it possible to add some kind of variable option in sysctl.conf? No idea how it's done :-)
Anything is possible with software. It's not trivial to add a sysctl, nor is it hard. Finding time to work on those sorts of changes for 3rd generation decade old hardware is harder to justify. If you could get an answer from Broadcom support as to how many LUNs that entire family of hardware supports I'd make the change in FreeBSD. Otherwise I think your best bet is to patch FreeBSD and build your own twa.
Will do. Thank you for your time. Regs
A commit references this bug: Author: jpaetzel Date: Wed May 24 20:10:38 UTC 2017 New revision: 318805 URL: https://svnweb.freebsd.org/changeset/base/318805 Log: MFC 318401 Increase the number of LUNs this hardware can support. Experimentally we know this value works, but the hardware may support an even higher value. PR: 213876 Reported by: J.Catrysse@proximedia.be Changes: _U stable/11/ stable/11/sys/dev/tws/tws.h
A commit references this bug: Author: jpaetzel Date: Wed May 24 20:13:50 UTC 2017 New revision: 318806 URL: https://svnweb.freebsd.org/changeset/base/318806 Log: MFC 318401 Increase the number of LUNs this hardware can support. Experimentally we know this value works, but the hardware may support an even higher value. PR: 213876 Reported by: J.Catrysse@proximedia.be Changes: _U stable/10/ stable/10/sys/dev/tws/tws.h