Created attachment 240526 [details] Linux output for nvme id-ctrl All the tests were made on SAMSUNG MZWLJ15THALA-00007 15TB NVMe drives on FreeBSD 13.1 (see attachments for nvmecontrol identify outputs). Trying to create a namespace with nvmecontrol do not work with the command : # nvmecontrol ns create -s 117194752 -c 117194752 -f 2 nvme0 nvmecontrol: namespace creation failed: Invalid Field I tried different combinations of parameters, nothing works, even with a bare minimal command ns create fails. In comparison, the same command works fine on Linux : root@rescue:~# nvme create-ns -s 117194752 -c 117194752 -f 2 /dev/nvme0 create-ns: Success, created nsid:1 root@rescue:~# All other ns commands worked normally (delete, detach, etc ...), only the create command fails. These commands were tested on nda drivers, I do not think nvd drivers would change anything, it seems to be nvmecontrol that do not build the correct command to the NVMe controller.
Created attachment 240527 [details] FreeBSD output of nvmecontrol identify
Created attachment 240528 [details] FreeBSD output of nvmecontrol ns identify
are you able to test 14.0-CURRENT, 13-STABLE or 12-STABLE to se if this is a regression / where it starts?
I can test on 12.2/12.3/12.4 and CURRENT to see if it happens on those versions. I will make a new comment with those results.
It was a little more complicated than anticipated but I tested these 3 images : - FreeBSD-14.0-CURRENT-amd64-20230223-fe5c211ba873-261074-bootonly.iso - FreeBSD-13.2-STABLE-amd64-20230223-bc68c55166ad-254667-bootonly.iso - FreeBSD-12.4-STABLE-amd64-20230223-r372946-bootonly.iso and in all 3 images the same command failed, I will attach screenshots of the session this I did that via KVM/IPMI.
Created attachment 240536 [details] nvmecontrol ns create on FreeBSD CURRENT
Created attachment 240537 [details] nvmecontrol ns create on FreeBSD 12.4
Created attachment 240538 [details] nvmecontrol ns create on FreeBSD 13.2
if I transcript what appears on the console : root@:~ # nvmecontrol ns create -s 117194752 -c 117194752 -f 2 nvme1 nvme1: ADMIN COMMAND (0d) sqid:0 cid:15 nsid:0 cdw10:00000000 cdw11:00000000 nvme1: INVALID_FIELD (00/02) crd:0 m:0 dnr:0 sqid:0 cid:15 cdw0:0 nvmecontrol: namespace creation failed: Invalid Field root@:~ # Shouldn't the nsid be 1 ? since the namespace IDs in NVMe start at 1 and not 0 ?
(In reply to florian.millet from comment #9) No, create command is sent with nsid of 0, and controller selects the available identifier. And I would try with nvd, I know that at least reservation commands are handled incorrectly when using nda.
While my original test on 13.1 was done with nda drivers, all following tests with stock ISOs are using nvd since it is the default driver.
Stuck here with the same (?) problem: # nvmecontrol ns create -s 34359738368 -c 34359738368 -f1 nvme0 nvmecontrol: namespace creation failed: Invalid Format No matter if/what '-f' or which other arguments I specify, all 'nvmecontrol ns create' ever returns is "Invalid Format" NS info from another (identical) drive: # nvmecontrol ns identify -n1 nvme1 Size: 3750748848 blocks Capacity: 3750748848 blocks Utilization: 0 blocks Thin Provisioning: Not Supported Number of LBA Formats: 2 Current LBA Format: LBA Format #00 Data Protection Caps: Not Supported Data Protection Settings: Not Enabled Multi-Path I/O Capabilities: Not Supported Reservation Capabilities: Not Supported Format Progress Indicator: 0% remains Deallocate Logical Block: Read 00h, Write Zero Optimal I/O Boundary: 0 blocks NVM Capacity: 1920383410176 bytes Preferred Write Granularity: 8 blocks Preferred Write Alignment: 8 blocks Preferred Deallocate Granul: 8 blocks Preferred Deallocate Align: 8 blocks Optimal Write Size: 8 blocks Globally Unique Identifier: 000000000000000100a0752134194a0b IEEE EUI64: 00a0750134194a0b LBA Format #00: Data Size: 512 Metadata Size: 0 Performance: Good LBA Format #01: Data Size: 4096 Metadata Size: 0 Performance: Best Tested with 13.1-RELEASE and 13.2-RC1.
https://cgit.FreeBSD.org/src/commit/?id=20dc2c4d11ddaa980c491116b65c51ab522028e9 may fix it. Since I don't see neither of -d -p or -l parameters specified, the command send 0xff's to the drive instead, probably confusing it.
(In reply to Alexander Motin from comment #13) I can confirm that your commit fixes the problem : root@cluster99999:~ # nvmecontrol ns create -s 117194752 -c 117194752 -f 2 nvme1 namespace 1 created root@cluster99999:~ # nvmecontrol ns attach -n 1 nvme1 namespace 1 attached root@cluster99999:~ # nvmecontrol ns identify -n 1 nvme1 Size: 117194752 blocks Capacity: 117194752 blocks Utilization: 117194752 blocks Thin Provisioning: Not Supported Number of LBA Formats: 5 Current LBA Format: LBA Format #02 Metadata Capabilities Extended: Supported Separate: Supported Data Protection Caps: Last Bytes, First Bytes, Type 3, Type 2, Type 1 Data Protection Settings: Not Enabled Multi-Path I/O Capabilities: May be shared Reservation Capabilities: IEKEY13, EX_AC_AR, WR_EX_AR, EX_AC_RO, WR_EX_RO, EX_AC, WR_EX, PTPL Format Progress Indicator: 0% remains Deallocate Logical Block: Read 00h Optimal I/O Boundary: 0 blocks NVM Capacity: 480029704192 bytes Globally Unique Identifier: 35343530549007000025384500000001 IEEE EUI64: 0000000000000000 LBA Format #00: Data Size: 512 Metadata Size: 0 Performance: Better LBA Format #01: Data Size: 512 Metadata Size: 8 Performance: Degraded LBA Format #02: Data Size: 4096 Metadata Size: 0 Performance: Best LBA Format #03: Data Size: 4096 Metadata Size: 8 Performance: Good LBA Format #04: Data Size: 4096 Metadata Size: 64 Performance: Degraded root@cluster99999:~ #
(In reply to florian.millet from comment #14) I did not specify on my previous comment, but the fix was tested on FreeBSD 13.1 with nda drivers. Can we expect this commit to be MFC before 13.2-RELEASE ?
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=de0b02cb7b112947191af96dd34e7a95b760019c commit de0b02cb7b112947191af96dd34e7a95b760019c Author: Alexander Motin <mav@FreeBSD.org> AuthorDate: 2023-03-09 15:19:00 +0000 Commit: Alexander Motin <mav@FreeBSD.org> CommitDate: 2023-03-13 13:53:03 +0000 nvmecontrol: Fix default ns create parameters. Instead of passing 0xff's for all unset parameters, prefer reasonable defaults. It is much easier to use it this was without specs in hand. PR: 269912 (cherry picked from commit 20dc2c4d11ddaa980c491116b65c51ab522028e9) sbin/nvmecontrol/ns.c | 56 ++++++++++++++++++++++++++++++++++----------------- 1 file changed, 37 insertions(+), 19 deletions(-)
(In reply to florian.millet from comment #15) Thanks for testing it. I've merged it to stable/13 and requested re@ permission for merge to 13.2.
(In reply to Alexander Motin from comment #17) Excellent. Thank you for the fix !
(In reply to Alexander Motin from comment #13) Sorry for the late reply and thanks for that info and the fix. The manpage lists all arguments but -s and the device-id as optional ("[...]"), so I got the impression those aren't needed and/or have reasonable defaults.
A commit in branch releng/13.2 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=069fb41af5bca224caf6f96b50fed55fa7755c5b commit 069fb41af5bca224caf6f96b50fed55fa7755c5b Author: Alexander Motin <mav@FreeBSD.org> AuthorDate: 2023-03-09 15:19:00 +0000 Commit: Alexander Motin <mav@FreeBSD.org> CommitDate: 2023-03-14 13:58:02 +0000 nvmecontrol: Fix default ns create parameters. Instead of passing 0xff's for all unset parameters, prefer reasonable defaults. It is much easier to use it this was without specs in hand. PR: 269912 Approved by: re (cperciva) (cherry picked from commit 20dc2c4d11ddaa980c491116b65c51ab522028e9) (cherry picked from commit de0b02cb7b112947191af96dd34e7a95b760019c) sbin/nvmecontrol/ns.c | 56 ++++++++++++++++++++++++++++++++++----------------- 1 file changed, 37 insertions(+), 19 deletions(-)
(In reply to Sebastian from comment #19) Yes, it was a bug, obviously not handling default values properly. Fixed now for 13.2.