We are seeing the issue in bus_dmamap_load_ccb() function in the case of 1MB IOs. It seems, that IO mapping to DMA mem is taking more time so bus_dmamap_load_ccb is returning EINPROGRESS (error no: 36) more frequently. the DMA attributes passed are below while creating the tag: 1. maxsize: 1MB 2. nsegments: 256 3. maxsegsz: 1MB I have seen in some of the FreeBSD BZ that warner losh has suggested the workaround for this by increasing the hw.maxphys in /boot/loader.conf. I have tried the same and it will fix our issue. but the issue was reported on 13.0 and the fix was supposed to be part of 13.1 but I am still seeing it.
First, EINPROGRESS means that the transaction has been deferred either because the bounce pages it needs to allocate the memory backing the DMA isn't available (which I don't think is in play, but might be if there's a maximum address constraint for the dmamap) or there's not enough iommu contexts available to do the I/O. I'm having trouble recalling the exact context for my suggestion or the comment that the fix should be in 13.1. Can you refresh my recollection on that?
(In reply to Warner Losh from comment #1) Hi warner, Below is the Bug ID link where you mentioned that increasing the maxphys will resolve the issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240145 Couple of queries I have on your response: 1. when bus_dmamap_load_ccb() returns "EINPROGRESS" does it mean DMA memory allocation is failing due to the unavailability of DMA memory and what course of action the driver should take after this. do we need to return those IOs with specific error codes? so that those IOs could be re- tried. 2. Does it mean FreeBSD doesn't support 1MB IOs? in that do we need to reduce the max transfer size to < 1MB? if yes, then what could be the correct max IO size NOTE: hardware supports a max transfer size of 1MB