Bug 243148 - NVME drive Write Error on RK3399
Summary: NVME drive Write Error on RK3399
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: arm (show other bugs)
Version: CURRENT
Hardware: arm64 Any
: --- Affects Many People
Assignee: freebsd-arm mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-01-07 05:14 UTC by me
Modified: 2020-02-15 07:11 UTC (History)
2 users (show)

See Also:


Attachments
DTS patch from Radax's offical linux repo on github (1.21 KB, patch)
2020-01-07 05:14 UTC, me
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description me 2020-01-07 05:14:47 UTC
Created attachment 210492 [details]
DTS patch from Radax's offical linux repo on github

I built FreeBSD with the Current branch on my Rock Pi 4 1.3RevB Today, with a small dts patch in the attachment, and my dmesg is here:

https://dmesgd.nycbug.org/index.cgi?do=view&id=5301

with @mmel's recent PCIE code, my PCIE to SATA adapter(Marvell 9235 base) is working very stable, I've been testing read and write an ZFS Mirror pool with 2 350G hardware for 2 hours, 0 problem.

However the nvme drive have some problem when I try to dd a single 5GB on it(small file like 5M work fine). The error message:

/mnt # dd if=/dev/urandom of=file5G bs=1G count=5
nvme0: WRITE sqid:6 cid:127 nsid:1 lba:106216 len:256
nvme0: COMMAND ID CONFLICT (00/03) sqid:6 cid:127 cdw0:0
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=19ee8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=54362112, length=131072)]error = 5



nvme0: Resetting controller due to a timeout and fatal error status.
nvme0: Resetting controller due to a timeout and fatal error status.
nvme0: resetting controller
nvme0: Resetting controller due to a timeout and fatal error status.
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: Resetting controller due to a timeout and fatal error status.
nvme0: resetting controller
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: Resetting controller due to a timeout and fatal error status.
nvme0: Resetting controller due to a timeout and fatal error status.
nvme0: resetting controller
nvme0: Resetting controller due to a timeout and fatal error status.
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: Resetting controller due to a timeout and fatal error status.
nvme0: Resetting controller due to a timeout and fatal error status.
nvme0: resetting controller
nvme0: Resetting controller due to a timeout and fatal error status.
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: Resetting controller due to a timeout and fatal error status.
nvme0: resetting controller
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: WRITE sqid:6 cid:123 nsid:1 lba:105448 len:256
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:123 cdw0:0
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=19be8 0 ff 0 0 0
nvme0: WRITE sqid:6 cid:118 nsid:1 lba:105704 len:256
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:118 cdw0:0
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: aborting outstanding i/o
g_vfs_done():gpt/disk0[WRITE(offset=53968896, length=131072)]ernvme0: WRITE sqid:6 cid:117 nsid:1 lba:105960 len:256
ror = 5
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:117 cdw0:0
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=19ce8 0 ff 0 0 0
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: WRITE sqid:6 cid:116 nsid:1 lba:106472 len:256
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:116 cdw0:0
g_vfs_done():gpt/disk0[WRITE(offset=54099968, length=131072)]ernvme0: aborting outstanding i/o
ror = 5
nvme0: WRITE sqid:6 cid:122 nsid:1 lba:106728 len:256
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=19de8 0 ff 0 0 0
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:122 cdw0:0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: WRITE sqid:6 cid:115 nsid:1 lba:106984 len:256
g_vfs_done():gpt/disk0[WRITE(offset=54231040, length=131072)]error = 5
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:115 cdw0:0
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=19fe8 0 ff 0 0 0
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: WRITE sqid:6 cid:125 nsid:1 lba:107240 len:256
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:125 cdw0:0
g_vfs_done():gpt/disk0[WRITE(offset=54493184, length=131072)]error = 5
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): WRITE. NCB: opc=11a0e8 0 ff 0 0 0
nvme0: WRITE sqid:6 cid:114 nsid:1 lba:107496 len:256
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:114 cdw0:0
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: aborting outstanding i/o
g_vfs_done():gpt/disk0[WRITE(offset=54624256, length=131072)]ernvme0: WRITE sqid:6 cid:121 nsid:1 lba:107752 len:256
ror = 5
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:121 cdw0:0
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1a1e8 0 ff 0 0 0
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: WRITE sqid:6 cid:113 nsid:1 lba:108008 len:256
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:113 cdw0:0
g_vfs_done():gpt/disk0[WRITE(offset=54755328, length=131072)]ernvme0: aborting outstanding i/o
ror = 5
nvme0: WRITE sqid:6 cid:126 nsid:1 lba:108264 len:256
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1a2e8 0 ff 0 0 0
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:126 cdw0:0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: WRITE sqid:6 cid:112 nsid:1 lba:108520 len:256
g_vfs_done():gpt/disk0[WRITE(offset=54886400, length=131072)]ernvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:112 cdw0:0
ror = 5
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1a3e8 0 ff 0 0 0
nvme0: WRITE sqid:6 cid:111 nsid:1 lba:108776 len:256
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:111 cdw0:0
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: aborting outstanding i/o
g_vfs_done():gpt/disk0[WRITE(offset=55017472, length=131072)]ernvme0: WRITE sqid:6 cid:120 nsid:1 lba:109032 len:256
ror = 5
nvme0: ABborting outstanding i/o
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: WRITE sqid:6 cid:110 nsid:1 lba:109288 len:256
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:110 cdw0:0
g_vfs_done():gpt/disk0[WRITE(offset=55148544, length=131072)]ernvme0: aborting outstanding i/o
ror = 5
nvme0: WRITE sqid:6 cid:109 nsid:1 lba:109544 len:256
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1a5e8 0 ff 0 0 0
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:109 cdw0:0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: WRITE sqid:6 cid:108 nsid:1 lba:109800 len:256
g_vfs_done():gpt/disk0[WRITE(offset=55279616, length=131072)]ernvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:108 cdw0:0
ror = 5
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1a6e8 0 ff 0 0 0
nvme0: WRITE sqid:6 cid:107 nsid:1 lba:110056 len:256
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:107 cdw0:0
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: aborting outstanding i/o
g_vfs_done():gpt/disk0[WRITE(offset=55410688, length=131072)]ernvme0: WRITE sqid:6 cid:106 nsid:1 lba:110312 len:256
ror = 5
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:106 cdw0:0
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1a7e8 0 ff 0 0 0
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: WRITE sqid:6 cid:105 nsid:1 lba:110568 len:256
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:105 cdw0:0
g_vfs_done():gpt/disk0[WRITE(offset=55541760, length=131072)]ernvme0: aborting outstanding i/o
ror = 5
nvme0: WRITE sqid:6 cid:104 nsid:1 lba:110824 len:256
(n
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: WRITE sqid:6 cid:103 nsid:1 lba:111080 len:256
g_vfs_done():gpt/disk0[WRITE(offset=55672832, length=131072)]ernvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:103 cdw0:0
ror = 5
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1a9e8 0 ff 0 0 0
nvme0: WRITE sqid:6 cid:102 nsid:1 lba:111336 len:256
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:102 cdw0:0
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: aborting outstanding i/o
g_vfs_done():gpt/disk0[WRITE(offset=55803904, length=131072)]ernvme0: WRITE sqid:6 cid:101 nsid:1 lba:111592 len:256
ror = 5
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:101 cdw0:0
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1aae8 0 ff 0 0 0
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: WRITE sqid:6 cid:100 nsid:1 lba:111848 len:256
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:100 cdw0:0
g_vfs_done():gpt/disk0[WRITE(offset=55934976, length=131072)]ernvme0: aborting outstanding i/o
ror = 5
nvme0: WRITE sqid:6 cid:99 nsid:1 lba:112104 len:256
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1abe8 0 ff 0 0 0
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:99 cdw0:0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: WRITE sqid:6 cid:124 nsid:1 lba:112360 len:256
g_vfs_done():gpt/disk0[WRITE(offset=56066048, length=131072)]ernvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:124 cdw0:0
ror = 5
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsidd with an error
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:119 cdw0:0
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: aborting outstanding i/o
g_vfs_done():gpt/disk0[WRITE(offset=56197120, length=131072)]error = 5
nvme0: WRITE sqid:6 cid:127 nsid:1 lba:112872 len:256
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1ade8 0 ff 0 0 0
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:127 cdw0:0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: WRITE sqid:6 cid:98 nsid:1 lba:113128 len:256
g_vfs_done():gpt/disk0[WRITE(offset=56328192, length=131072)]error = 5
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:98 cdw0:0
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1aee8 0 ff 0 0 0
nvme0: aborting outstanding i/o
nvme0: WRITE sqid:6 cid:97 nsid:1 lba:113384 len:256
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:97 cdw0:0
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: aborting outstanding i/o
g_vfs_done():gpt/disk0[WRITE(offset=56459264, length=131072)]ernvme0: WRITE sqid:6 cid:96 nsid:1 lba:113640 len:256
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:96 cdw0:0
ror = 5
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1afe8 0 ff 0 0 0
nvme0: WRITE sqid:6 cid:95 nsid:1 lba:113896 len:256
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:95 cdw0:0
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: aborting outstanding i/o
g_vfs_done():gpt/disk0[WRITE(offset=56590336, length=131072)]ernvme0: WRITE sqid:6 cid:94 nsid:1 lba:114152 len:256
ror = 5
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:94 cdw0:0
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1b0e8 0 ff 0 0 114408 len:256
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:93 cdw0:0
g_vfs_done():gpt/disk0[WRITE(offset=56721408, length=131072)]ernvme0: aborting outstanding i/o
ror = 5
nvme0: WRITE sqid:6 cid:92 nsid:1 lba:114664 len:256
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1b1e8 0 ff 0 0 0
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:92 cdw0:0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: WRITE sqid:6 cid:91 nsid:1 lba:114920 len:256
g_vfs_done():gpt/disk0[WRITE(offset=56852480, length=131072)]ernvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:91 cdw0:0
ror = 5
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1b2e8 0 ff 0 0 0
nvme0: WRITE sqid:6 cid:90 nsid:1 lba:115176 len:256
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:90 cdw0:0
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: aborting outstanding i/o
g_vfs_done():gpt/disk0[WRITE(offset=56983552, length=131072)]ernvme0: WRITE sqid:6 cid:89 nsid:1 lba:115432 len:256
ror = 5
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:89 cdw0:0
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1b3e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: WRITE sqid:6 cid:88 nsid:1 lba:115688 len:256
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:88 cdw0:0
g_vfs_done():gpt/disk0[WRITE(offset=57114624, length=131072)]ernvme0: aborting outstanding i/o
ror = 5
nvme0: WRITE sqid:6 cid:87 nsid:1 lba:115944 len:256
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1b4e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with done():gpt/disk0[WRITE(offset=57245696, length=131072)]ernvme0: WRITE sqid:6 cid:86 nsid:1 lba:116200 len:256
ror = 5
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:86 cdw0:0
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1b5e8 0 ff 0 0 0
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: WRITE sqid:6 cid:85 nsid:1 lba:116456 len:256
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:85 cdw0:0
g_vfs_done():gpt/disk0[WRITE(offset=57376768, length=131072)]ernvme0: aborting outstanding i/o
ror = 5
nvme0: WRITE sqid:6 cid:84 nsid:1 lba:116712 len:256
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1b6e8 0 ff 0 0 0
nvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:84 cdw0:0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: WRITE sqid:6 cid:83 nsid:1 lba:116968 len:256
g_vfs_done():gpt/disk0[WRITE(offset=57507840, length=131072)]ernvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:83 cdw0:0
ror = 5
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1b7e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
nvme0: WRITE sqid:6 cid:82 nsid:1 lba:117224 len:256
g_vfs_done():gpt/disk0[WRITE(offset=57638912, length=131072)]ernvme0: ABORTED - BY REQUEST (00/07) sqid:6 cid:82 cdw0:0
ror = 5
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1b8e8 0 ff 0 0 0
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
nvme0: aborting outstanding i/o
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=57769984, length=131072)]error = 5
(nda0:nvme0:0:0:1):or
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=57901056, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1bae8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=58032128, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1bbe8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=58163200, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1bce8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=58294272, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1bde8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=58425344, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1bee8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=58556416, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1bfe8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=58687488, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1c0e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhagth=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1c1e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=58949632, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1c2e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=59080704, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1c3e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=59211776, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1c4e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=59342848, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1c5e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=59473920, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1c6e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=59604992, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1c7e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=59736064, length=131072)]error = 5
(nda0:nvme0:0:0:1): WR): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=59867136, length=131072)]error = 5
(nda0:nvme0:0:0:1): WRITE. NCB: opc=1 fuse=0 nsid=1 prp1=0 prp2=0 cdw=1c9e8 0 ff 0 0 0
(nda0:nvme0:0:0:1): CAM status: CCB request completed with an error
(nda0:nvme0:0:0:1): Error 5, Retries exhausted
g_vfs_done():gpt/disk0[WRITE(offset=59998208, length=131072)]error = 5
5+0 records in
5+0 records out
5368709120 bytes transferred in 200.878442 secs (26726159 bytes/sec)
Comment 1 wcarson.bugzilla 2020-02-15 01:33:19 UTC
I'm having the exact same issue with an NVMe on my Khadas EDGE-V, which is also RK3399 chipset.

Setting hw.ncpu="4" in /boot/loader.conf works around the issue, but disables the two "big" A72 cores.
Comment 2 me 2020-02-15 03:06:52 UTC
Thank for the workaround, I'll give it a try later. I hope someone can solve this bug without disabling cores(especially big cores)
Comment 3 wcarson.bugzilla 2020-02-15 04:47:18 UTC
Just as another point of information, I tried hw.nvme.use_nvd=1 in /boot/loader.conf, which forced using the nvd driver instead of nda, but the same issue occurs:

nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00df sqid:0005 cid:0000 p:1 sc:0b sct:0 m:1 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:0074 sqid:0005 cid:007f p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00df sqid:0005 cid:0000 p:0 sc:0b sct:0 m:1 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00ea sqid:0005 cid:007b p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00fa sqid:0005 cid:007b p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:0077 sqid:0005 cid:007f p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:0092 sqid:0005 cid:007b p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00a9 sqid:0005 cid:007f p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00c6 sqid:0005 cid:007f p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00cf sqid:0005 cid:0076 p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00da sqid:0005 cid:007f p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00fa sqid:0005 cid:007f p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00fb sqid:0005 cid:0076 p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00d9 sqid:0005 cid:0069 p:1 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00e6 sqid:0005 cid:006b p:0 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:0020 sqid:0005 cid:006b p:1 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:005a sqid:0005 cid:006b p:1 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:009e sqid:0005 cid:006b p:1 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00aa sqid:0005 cid:006b p:1 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:00bc sqid:0005 cid:006b p:1 sc:00 sct:0 m:0 dnr:0
nvme0: Resetting controller due to a timeout.
nvme0: Resetting controller due to a timeout.
nvme0: resetting controller
nvme0: Resetting controller due to a timeout.
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: WRITE sqid:6 cid:113 nsid:1 lba:975796480 len:256
nvme0: GUARD CHECK ERROR (02/82) sqid:6 cid:113 cdw0:0
g_vfs_done():nvd0e[WRITE(offset=4677500928, length=131072)]error = 5
nvme0: WRITE sqid:6 cid:110 nsid:1 lba:975797504 len:256
nvme0: PRP OFFET INVALID (00/13) sqid:6 cid:110 cdw0:0
g_vfs_done():nvd0e[WRITE(offset=4678025216, length=131072)]error = 5
nvme0: WRITE sqid:6 cid:105 nsid:1 lba:975799296 len:256
nvme0: PRP OFFET INVALID (00/13) sqid:6 cid:105 cdw0:0
g_vfs_done():nvd0e[WRITE(offset=4678942720, length=131072)]error = 5
nvme0: WRITE sqid:6 cid:100 nsid:1 lba:975801088 len:256
nvme0: PRP OFFET INVALID (00/13) sqid:6 cid:100 cdw0:0
g_vfs_done():nvd0e[WRITE(offset=4679860224, length=131072)]error = 5
nvme0: WRITE sqid:6 cid:95 nsid:1 lba:975802880 len:256
nvme0: GUARD CHECK ERROR (02/82) sqid:6 cid:95 cdw0:0
g_vfs_done():nvd0e[WRITE(offset=4680777728, length=131072)]error = 5
nvme0: WRITE sqid:6 cid:90 nsid:1 lba:975804672 len:256
nvme0: PRP OFFET INVALID (00/13) sqid:6 cid:90 cdw0:0
g_vfs_done():nvd0e[WRITE(offset=4681695232, length=131072)]error = 5
nvme0: WRITE sqid:6 cid:85 nsid:1 lba:975806464 len:256
nvme0: PRP OFFET INVALID (00/13) sqid:6 cid:85 cdw0:0
g_vfs_done():nvd0e[WRITE(offset=4682612736, length=131072)]error = 5
nvme0: WRITE sqid:6 cid:82 nsid:1 lba:975807488 len:256
nvme0: GUARD CHECK ERROR (02/82) sqid:6 cid:82 cdw0:0
g_vfs_done():nvd0e[WRITE(offset=4683137024, length=131072)]error = 5
nvme0: WRITE sqid:6 cid:69 nsid:1 lba:986208768 len:256
nvme0: COMMAND ID CONFLICT (00/03) sqid:6 cid:69 cdw0:0
g_vfs_done():nvd0e[WRITE(offset=10008592384, length=131072)]error = 5
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:009b sqid:0006 cid:0045 p:1 sc:00 sct:0 m:0 dnr:0
nvme0: cpl does not map to outstanding cmd
cdw0:00000000 sqhd:008d sqid:0005 cid:0000 p:1 sc:0b sct:0 m:1 dnr:0
nvme0: Resetting controller due to a timeout.
nvme0: resetting controller
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
nvme0: aborting outstanding i/o
Comment 4 me 2020-02-15 07:11:27 UTC
hope some committer can fix this.