DragonFly Mail Agent DMA from base system (/usr/libexec/dma, version ~0.11) sometimes reports corrupted queue files. When I switch to DMA from binary pkg (/usr/local/libexec/dma, version ~0.13) then errors regarding corrupted queue files are not occuring anymore. Slightly more verbose report in upstream issue tracker: https://github.com/corecode/dma/issues/110
Looks like FreeBSD main branch has this additional commit that is not in FreeBSD 13.1. https://cgit.freebsd.org/src/commit/contrib/dma/mail.c?id=d21e71efce3963b068ccaa807a099b591dc220e9 https://reviews.freebsd.org/D34159
(In reply to Mikko Lehto from comment #1) I just now merged that commit to stable/13 in 2e8403e0215868a13e2bdd1c979fc351971b8d91
(In reply to Ed Maste from comment #2) Thank you Ed, unfortunately still gives "corrupted queue file". Hope this helps, looking forward not to use Sendmail. Below are steps to reproduce: 1. Succesful case with no problems (13.1 binary pkg mail/dma-0.13.20220128_1,1): --- echo -n 'hello world' | /usr/local/libexec/dma -bq -f localuser@localhost remouser@remotehost.example.invalid --- In this OK case queue file /var/spool/dma/M... ends in byte 0x0A, that seem to be the expected queue file body suffix. Remote delivery is succesful. 2. Failing case (13.1 src.txz + new mail.c from 2e8403e0215868a13e2bdd1c979fc351971b8d91): --- echo -n 'hello world' | /usr/libexec/dma_266629 -bq -f localuser@localhost remouser@remotehost.example.invalid --- In this failing case I can see that queue file /var/spool/dma/M... ends in 0x64, that is last letter of "world". Remote delivery fails. I use "hexdump -C /var/spool/dma/M..." to observe last byte in queue file. For fun I also took mail.c from upstream repository and compiled. With this, frankenstein, there is always 0x0A at the end of /var/spool/dma/M... queue file. ( Still comparing readmail() loops, trying to figure something )
Created attachment 236845 [details] Helper script for highlighting failing local submission Attached helper script "test_dma.sh" that aims to highlight problematic local submission scenario.
Created attachment 236848 [details] Sample log Sample output with test_dma.sh script. This run highlights queue identifiers: 5ff8e.801c48000 and 5ff96.800e48000 They are scenarios: "/usr/libexec/dma BODY WITHOUT NEWLINE" and "/usr/libexec/dma_266629 BODY WITHOUT NEWLINE"
OK to be sure I understand correctly, the issue is that when submitting mail where the body does not end with a newline, upstream dma adds a newline to the queued mail while the version in the base system does not? The version in the base system has an additional change to split long lines - see https://github.com/corecode/dma/issues/18 - and presumably this is not adding the newline when the input does not have one.
(In reply to Ed Maste from comment #6) Yes, you just described it the way I also understand this issue, thanks. At first I was uncertain whether "Sendmail pipe interface" expects some special sequence of characters that might cause this, but I think my calling style is expected. Your patch[1] seem to work, I can not reproduce "corrupted queue files" any more, great. [1] https://reviews.freebsd.org/D36763
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=169487223143b1232ec4686b720b028af8d6d42b commit 169487223143b1232ec4686b720b028af8d6d42b Author: Ed Maste <emaste@FreeBSD.org> AuthorDate: 2022-09-27 19:19:19 +0000 Commit: Ed Maste <emaste@FreeBSD.org> CommitDate: 2022-10-12 15:59:01 +0000 dma: restore addition of newline when missing from input If input mail does not have a newline on the last line dma must add one. This was broken by the addition of long-line splitting, with the switch from strlen(line) to linelen returned by getline(). PR: 266629 Reviewed by: bapt, Mikko Lehto Tested by: Mikko Lehto MFC after: 1 week Fixes: b0b2d05fd060 ("Split body of mails not respecting...") Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36763 contrib/dma/mail.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=bd3d597a6bc0e35273669cfd7a99b3a5b437d92a commit bd3d597a6bc0e35273669cfd7a99b3a5b437d92a Author: Ed Maste <emaste@FreeBSD.org> AuthorDate: 2022-09-27 19:19:19 +0000 Commit: Ed Maste <emaste@FreeBSD.org> CommitDate: 2022-10-19 01:03:55 +0000 dma: restore addition of newline when missing from input If input mail does not have a newline on the last line dma must add one. This was broken by the addition of long-line splitting, with the switch from strlen(line) to linelen returned by getline(). PR: 266629 Reviewed by: bapt, Mikko Lehto Tested by: Mikko Lehto MFC after: 1 week Fixes: b0b2d05fd060 ("Split body of mails not respecting...") Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36763 (cherry picked from commit 169487223143b1232ec4686b720b028af8d6d42b) contrib/dma/mail.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
A commit in branch stable/12 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=de0b782545f725e719f9b3f1ec7f2e6442551086 commit de0b782545f725e719f9b3f1ec7f2e6442551086 Author: Ed Maste <emaste@FreeBSD.org> AuthorDate: 2022-09-27 19:19:19 +0000 Commit: Ed Maste <emaste@FreeBSD.org> CommitDate: 2022-11-18 20:57:45 +0000 dma: restore addition of newline when missing from input If input mail does not have a newline on the last line dma must add one. This was broken by the addition of long-line splitting, with the switch from strlen(line) to linelen returned by getline(). PR: 266629 Reviewed by: bapt, Mikko Lehto Tested by: Mikko Lehto MFC after: 1 week Fixes: b0b2d05fd060 ("Split body of mails not respecting...") Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36763 (cherry picked from commit 169487223143b1232ec4686b720b028af8d6d42b) (cherry picked from commit bd3d597a6bc0e35273669cfd7a99b3a5b437d92a) contrib/dma/mail.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)