Comment 1 Guido Falsi 2026-07-02 11:44:29 UTC

Hi,

I'm using courier-imap myself with a setup similar to yours, and experiencing no issues with version 6.0.1 on 15.1-RELEASE.

This means I'm unable to reproduce your issue right now and further information will be required to try to understand the issue.

Since you are on stable, I need first to ask if you tried to recompile the port in a clean environment, just in case some ABI breakage happened.

Also, you say version "2.5.0" in the subject here, but then say you are using 5.3.2, please correct the discrepancy.

The backtrace you provide is, unluckily, not very useful.

One minor reason is it is unclear from which version it was extracted from, 6.0.0 or 6.0.1 or something else? You state which version works for you, but I will also need clear indication of which one you are testing as not working. Please test the latest 6.0.1

Second, much more substantial reason the backtrace is not very helpful is that it is from anon debugging binary and so gives no indication exactly where in imapd binary the rash is happening.

To help you somewhat I need this information, so you should try to recompile courier-imap with debugging symbols. It can be easily done by adding "WITH_DEBUG_PORTS= mail/courier-imap" to make.conf and recompiling reinstalling the port. Then see if it still crashes and gather backtrace information again which, with some luck, should include the name of the function the crash is happening in.

Sorry if I can't be more helpful right away.

(In reply to Guido Falsi from comment #1)
The packages were built in pouriere on a two-week-old stable/15 (15.2-STABLE) 
Although the affected hosts are also running 1501501, they are running the most recent stable/15.

About backtrace, it was most likely taken from 6.0.1, but 6.0.0 behaves in the same manner.

I will try to build it with DEBUG when I find some time to attach better core dumps.

I had some time to create a couple of very nice core dumps:

(lldb) target create "/usr/local/bin/imapd" --core "imapd.64617.core"
Core file '/tmp/imapd.64617.core' (x86_64) was loaded.
(lldb) bt
* thread #1, name = 'imapd', stop reason = signal SIGABRT
  * frame #0: 0x0000289e4cb43d4a libsys.so.7`__sys_thr_kill + 10
    frame #1: 0x0000289e4b6566f4 libc.so.7`__raise + 52
    frame #2: 0x0000289e4b707e29 libc.so.7`abort + 73
    frame #3: 0x0000289e4a3d7048 libcxxrt.so.1`___lldb_unnamed_symbol_fea0 + 424
    frame #4: 0x0000289e4a3d5179 libcxxrt.so.1`std::terminate() + 41
    frame #5: 0x0000289e4a3d5664 libcxxrt.so.1`___lldb_unnamed_symbol_e580 + 228
    frame #6: 0x0000289e4a3d5579 libcxxrt.so.1`__cxa_throw + 57
    frame #7: 0x0000289e49833e31 libc++.so.1`std::__throw_bad_alloc() + 49
    frame #8: 0x0000289e49842b77 libc++.so.1`operator new(unsigned long) + 71
    frame #11: 0x000028962459be90 imapd`std::__1::allocator<imapscanmessageinfo>::allocate[abi:se210108](this=0x000028962468aa40, __n=141710)
 at allocator.h:105:14
    frame #14: 0x00002896245aa2a2 imapd`std::__1::vector<imapscanmessageinfo, std::__1::allocator<imapscanmessageinfo>>::__append(this=size=0
, __n=141710) at vector.h:957:49
    frame #15: 0x00002896245a102c imapd`std::__1::vector<imapscanmessageinfo, std::__1::allocator<imapscanmessageinfo>>::resize(this=size=0, 
__sz=141710) at vector.h:1361:11
    frame #16: 0x0000289624594599 imapd`do_imapscan_maildir2(scaninfo=0x000028962468aa00, leavenew=0, ro=0, uidplus_vec=size=0) at imapscancl
ient.C:699:17
    frame #17: 0x0000289624592bdb imapd`imapscan_maildir(imapscaninfo*, int, int, std::__1::vector<uidplus_info, std::__1::allocator<uidplus_
info>>&)::$_0::operator()(this=0x00005370f6629098) const at imapscanclient.C:165:11
    frame #23: 0x0000289624580555 imapd`std::__1::function<bool ()>::operator()(this=0x0000289e44b98260) const at function.h:772:10
    frame #24: 0x0000289624563b66 imapd`imapmaildirlock(scaninfo=0x000028962468aa00, maildir=".", callback=0x0000289e44b98260) at imapd.C:184
:9
    frame #25: 0x0000289624591b3a imapd`imapscan_maildir(scaninfo=0x000028962468aa00, leavenew=0, ro=0, uidplus=size=0) at imapscanclient.C:1
61:9
    frame #26: 0x0000289624591a40 imapd`imapscan_maildir(scaninfo=0x000028962468aa00, leavenew=0, ro=0) at imapscanclient.C:155:9
    frame #27: 0x000028962456d5a8 imapd`do_imap_command(tag="69", flushflag=0x0000289e44b9a1ac) at imapd.C:4332:7
    frame #28: 0x00002896245f76f3 imapd`mainloop() at mainloop.C:123:7
    frame #29: 0x00002896245762b3 imapd`main(argc=2, argv=0x0000289e44b9a4b0) at imapd.C:5882:2
    frame #30: 0x0000289e4b62b15f libc.so.7`__libc_start1 + 303
    frame #31: 0x000028962454d341 imapd`_start + 33


(lldb) target create "/usr/local/bin/imapd" --core "imapd.64614.core"
Core file '/tmp/imapd.64614.core' (x86_64) was loaded.
(lldb) bt
* thread #1, name = 'imapd', stop reason = signal SIGABRT
  * frame #0: 0x0000128390731d4a libsys.so.7`__sys_thr_kill + 10
    frame #1: 0x000012838d2aa6f4 libc.so.7`__raise + 52
    frame #2: 0x000012838d35be29 libc.so.7`abort + 73
    frame #3: 0x000012838dbfb048 libcxxrt.so.1`___lldb_unnamed_symbol_fea0 + 424
    frame #4: 0x000012838dbf9179 libcxxrt.so.1`std::terminate() + 41
    frame #5: 0x000012838dbf9664 libcxxrt.so.1`___lldb_unnamed_symbol_e580 + 228
    frame #6: 0x000012838dbf9579 libcxxrt.so.1`__cxa_throw + 57
    frame #7: 0x000012838b4f3e31 libc++.so.1`std::__throw_bad_alloc() + 49
    frame #8: 0x000012838b502b77 libc++.so.1`operator new(unsigned long) + 71
    frame #11: 0x0000127b672cbe90 imapd`std::__1::allocator<imapscanmessageinfo>::allocate[abi:se210108](this=0x0000127b673baa40, __n=141710)
 at allocator.h:105:14
    frame #14: 0x0000127b672da2a2 imapd`std::__1::vector<imapscanmessageinfo, std::__1::allocator<imapscanmessageinfo>>::__append(this=size=0
, __n=141710) at vector.h:957:49
    frame #15: 0x0000127b672d102c imapd`std::__1::vector<imapscanmessageinfo, std::__1::allocator<imapscanmessageinfo>>::resize(this=size=0, 
__sz=141710) at vector.h:1361:11
    frame #16: 0x0000127b672c4599 imapd`do_imapscan_maildir2(scaninfo=0x0000127b673baa00, leavenew=0, ro=0, uidplus_vec=size=0) at imapscancl
ient.C:699:17
    frame #17: 0x0000127b672c2bdb imapd`imapscan_maildir(imapscaninfo*, int, int, std::__1::vector<uidplus_info, std::__1::allocator<uidplus_
info>>&)::$_0::operator()(this=0x00002a84ba629098) const at imapscanclient.C:165:11
    frame #23: 0x0000127b672b0555 imapd`std::__1::function<bool ()>::operator()(this=0x00001283883894b0) const at function.h:772:10
    frame #24: 0x0000127b67293b66 imapd`imapmaildirlock(scaninfo=0x0000127b673baa00, maildir=".", callback=0x00001283883894b0) at imapd.C:184
:9
    frame #25: 0x0000127b672c1b3a imapd`imapscan_maildir(scaninfo=0x0000127b673baa00, leavenew=0, ro=0, uidplus=size=0) at imapscanclient.C:1
61:9
    frame #26: 0x0000127b672c1a40 imapd`imapscan_maildir(scaninfo=0x0000127b673baa00, leavenew=0, ro=0) at imapscanclient.C:155:9
    frame #27: 0x0000127b6729d5a8 imapd`do_imap_command(tag="32", flushflag=0x000012838838b3fc) at imapd.C:4332:7
    frame #28: 0x0000127b673276f3 imapd`mainloop() at mainloop.C:123:7
    frame #29: 0x0000127b672a62b3 imapd`main(argc=2, argv=0x000012838838b708) at imapd.C:5882:2
    frame #30: 0x000012838d27f15f libc.so.7`__libc_start1 + 303
    frame #31: 0x0000127b6727d341 imapd`_start + 33

The key here is a large enough number of messages in one Maildir. It looks like overflow. To reproduce, please have 140k messages in your Maildir. In my case, it is 141710 messages.

    frame #14: 0x0000127b672da2a2 imapd`std::__1::vector<imapscanmessageinfo, std::__1::allocator<imapscanmessageinfo>>::__append(this=size=0
, __n=141710) at vector.h:957:49

FWIW, I have opened an issue on GitHub[1].

[1] https://github.com/svarshavchik/courier/issues/80

Comment 6 Guido Falsi 2026-07-02 16:25:44 UTC

(In reply to Marek Zarychta from comment #5)

Thanks for reporting this upstream, this is what I would have suggested you do.

The author is the best suited one to understand how to address this. I have no maildirs with that many messages.

BTW I'd also take a look at your limits(3) for the imapd process. It is possible the issue is the daemon exhausting the memory it is allowed to allocate.

By default daemons do have some limitations, check using limits(1).

Comment 7 Guido Falsi 2026-07-02 16:27:46 UTC

(In reply to Guido Falsi from comment #6)

Sorry, the  library call in section 3 is "ulimit" not limits.

Additional AI-reported debugging information from further analysis of the crash:

The SIGABRT is triggered by an uncaught std::bad_alloc during IMAP Maildir scanning. However, the allocation that actually fails is not large (~9MB for std::vector<imapscanmessageinfo>::resize(141710) with sizeof(imapscanmessageinfo)=64), which indicates this is not a simple memory exhaustion case.

Key observations from LLDB core analysis:

    The crash consistently happens in do_imapscan_maildir2() during std::vector<imapscanmessageinfo>::resize(__sz=141710).

    __sz directly corresponds to the number of files in a large Maildir (~141k messages), and is confirmed to be correct input data, not a computation error.

    ulimit / resource limits are not restricting memory usage (all relevant limits are unlimited on affected systems).

    The allocation failure occurs despite the small size of the request, which strongly suggests the process is already in a state where a relatively small contiguous allocation fails (heap exhaustion or fragmentation), rather than hitting a hard memory cap.

Additionally, deeper inspection of the scan path shows that the failure is preceded by construction of large intermediate structures (tempinfo_array, sorting, merging, and UID assignment), meaning peak memory usage during scan is significantly higher than the final msgs.resize() call suggests.

This points to a regression where Maildir scanning for large directories (~140k+ messages) leads to excessive memory pressure or fragmentation prior to the final vector allocation, rather than the resize itself being the root cause.

The issue is reproducible only above a certain Maildir size threshold (~140k messages) and is not present in 5.3.2_1,2 under identical workloads.

Comment 9 Guido Falsi 2026-07-02 21:33:43 UTC

(In reply to Marek Zarychta from comment #8)

Please remember I'm only maintaining the port. I do not have a deep understanding of the software sources. Also version 6.0.1 has no patches involving C/C++ source code.

To get this kind of understanding of the software working we need the developer involvement; let's see if he will followup to you bug report on github with some insight.

What I do know is that since version 5.3.2 courier-imap sources underwent some major changes, having migrated most of the code base from C to C++. This can for sure be cause of major memory allocation behaviour changes.

My suggestion to check limits is based on the fact that on my mail server imapd runs with some limits, and I'm using defaults:

Resource limits (current):
  cputime              infinity secs
  filesize             infinity kB
  datasize             33554432 kB
  stacksize              524288 kB
  coredumpsize         infinity kB
  memoryuse            infinity kB
  memorylocked           131072 kB
  maxprocesses            12013
  openfiles              229284
  sbsize               infinity bytes
  vmemoryuse              65536 kB
  pseudo-terminals     infinity
  swapuse              infinity kB
  kqueues              infinity
  umtxp                infinity
  pipebuf              infinity kB


Unless explicitly configured for unlimited values, some limits are applied by default. May not be the cause but worth checking.


The function where the failure happens has multiple resize operations before the failing one, some in tight loops.

I have an unverified theory that such behaviour could not only fragment process address space, but also balloon its requirements, because with such fast resizes, it is unclear if discarded memory is freed immediately or not. Could be garbage collected at a later time, for example.

As I said I maintain the port, do not develop the software, so I'm inclined to look at the "system operator" side of the issue, I can look at the code, but the upstream developer is for sure better suited for managing the coding part.

Increasing IMAP_ULIMITD as suggested by upstream resolves the issue.
So far, I haven't observed any further crashes with IMAP_ULIMITD=262144.

Whether this is the proper fix or just a workaround, it resolves the issue in my case.

Comment 11 Guido Falsi 2026-07-03 20:02:18 UTC

(In reply to Marek Zarychta from comment #10)

Thanks for reporting the result.

I'll be adding a note to pkg-message shortly with this information, so if someone else stumbles upon this there is some indication on how to solve this.

Comment 12 commit-hook 2026-07-03 20:14:51 UTC

A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=705988e58304b8a02435e50c8aa0b9187e48d3a5

commit 705988e58304b8a02435e50c8aa0b9187e48d3a5
Author:     Guido Falsi <madpilot@FreeBSD.org>
AuthorDate: 2026-07-03 20:09:59 +0000
Commit:     Guido Falsi <madpilot@FreeBSD.org>
CommitDate: 2026-07-03 20:14:21 +0000

    mail/courier-imap: Add note on solving possible memory issue

    When scanning maildirs with many files (more than 120000) the imapd
    daemon can crash reaching it's ulimit memory.

    Add a note to pkg-message about a configuration value that can be
    used to increase the limit.

    PR:             296444
    MFH:            2026Q3

 mail/courier-imap/files/pkg-message.in | 6 ++++++
 1 file changed, 6 insertions(+)

Thanks, that seems like a reasonable solution. Even so, it looks like there's still some room for refactoring.