Created attachment 250209 [details] Kernel module reproducing the issue The bounce pages for allocated DMA maps are never freed properly and leak when destroying DMA tags. The kernel sometimes needs to allocate bounce pages for DMA maps. When unloading the DMA maps with bounce pages those bounce pages are returned to the pool associated with DMA tag for later reuse when new maps are being loaded. The kernel should free the pages when destroying the DMA tag, but this never happens. The issue is quite easy to reproduce by creating a kernel module, in which a DMA tag is created and mapping the memory that does not meet the alignment restrictions set in DMA tag. After this the map should be unloaded and DMA tag destroyed. Doing this in a loop should soon crash the system. The code allocating bounce pages can be seen in [1], when grepping the source I can see that this is the only place when malloc with M_BOUNCE is called, but there is no corresponding free anywhere. I'm attaching a kernel module source for easy reproduction of the issue. The module creates a dma tag which requires 64 alignment, allocates memory aligned to 64 bytes and then immediately moves the pointer by 4 bytes to misalign it and loads the DMA map with misaligned pointer. This forces the bounce page to be created. Everything, including the DMA tag is then cleaned up. This allocation/deallocation loop happens every second. When the module is loaded, calling `vmstat -m | grep bounce` should show that M_BOUNCE malloc usage is rising, even though everything should be freed properly. Additionally, one can observe the bounce page malloc by using dtrace with one-liner `dtrace -n 'dtmalloc::bounce: {}'`. Dtrace should print malloc calls but no frees will be visible. I've seen the issue on arm64 and amd64, but it should be present on every hardware platform. [1] https://cgit.freebsd.org/src/tree/sys/kern/subr_busdma_bounce.c#n293
I posted a patch to fix this plus some related issues: https://reviews.freebsd.org/D47521 However, the problem isn't a leak per se, it's that the check in alloc_bounce_zone() fails to find the existing bounce zone when the DMA tag's alignment is < PAGE_SIZE. In particular, bounce zones are not private to a DMA tag; they're on a global linked list and can be shared, but the check which decides whether to use an existing zone is broken, so we allocate a new zone each time, which manifests as a leak.