Bug 239172 - mail/dovecot imap/pop3 stat-writer issue
Summary: mail/dovecot imap/pop3 stat-writer issue
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Only Me
Assignee: Larry Rosenman
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-07-12 17:54 UTC by Marek
Modified: 2019-08-29 14:48 UTC (History)
1 user (show)

See Also:
bugzilla: maintainer-feedback? (ler)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marek 2019-07-12 17:54:25 UTC
Hi all :)

After mail/dovecot upgrade to v2.3.7 it seems something went wrong with stat-writer...

IMAP and POP3 logs:

Jul 12 19:45:34 mail dovecot: imap-login: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/stats-writer), TRUE) failed: Invalid argument
Jul 12 19:45:34 mail dovecot: imap-login: Error: file_ostream.net_set_tcp_nodelay((conn:unix:login), TRUE) failed: Invalid argument
Jul 12 19:45:35 mail dovecot: imap: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/stats-writer), TRUE) failed: Invalid argument
Jul 12 19:45:35 mail dovecot: imap: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/auth-master), TRUE) failed: Invalid argument
Jul 12 19:45:35 mail dovecot: imap-login: Login: user=<someone@domain.tld>, method=PLAIN, rip=192.168.0.28, lip=192.168.0.1, mpid=47811, TLS, session=<aelAen+NKpDAqAAc>
Jul 12 19:45:41 mail dovecot: imap(someone@domain.tld)<47811><aelAen+NKpDAqAAc>: Connection closed (UID SEARCH finished 6.727 secs ago) in=128 out=2015 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0

Jul 12 19:51:21 mail dovecot: pop3-login: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/stats-writer), TRUE) failed: Invalid argument
Jul 12 19:51:21 mail dovecot: auth: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/stats-writer), TRUE) failed: Invalid argument
Jul 12 19:51:21 mail dovecot: pop3-login: Error: file_ostream.net_set_tcp_nodelay((conn:unix:login), TRUE) failed: Invalid argument
Jul 12 19:51:21 mail dovecot: pop3: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/stats-writer), TRUE) failed: Invalid argument
Jul 12 19:51:21 mail dovecot: pop3: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/auth-master), TRUE) failed: Invalid argument
Jul 12 19:51:21 mail dovecot: pop3-login: Login: user=<someone@domain.tld>, method=PLAIN, rip=192.168.0.33, lip=192.168.0.1, mpid=47877, TLS, session=<h+3jjn+NcPrAqAAh>
Jul 12 19:51:21 mail dovecot: pop3(someone@domain.tld)<47877><h+3jjn+NcPrAqAAh>: Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0
Comment 1 Larry Rosenman freebsd_committer 2019-07-12 19:05:49 UTC
I'm seeing this too.  Looking at it. 
Thanks for the report.
Comment 2 commit-hook freebsd_committer 2019-07-12 19:22:28 UTC
A commit references this bug:

Author: ler
Date: Fri Jul 12 19:22:10 UTC 2019
New revision: 506487
URL: https://svnweb.freebsd.org/changeset/ports/506487

Log:
  mail/dovecot: stop spamming the log with EINVAL.

  PR:		239172
  Submitted by:	zillion1@o2.pl
  Obtained from:	dovecot mailing list.

Changes:
  head/mail/dovecot/Makefile
  head/mail/dovecot/files/patch-src_lib_ostream-file.c
Comment 3 Larry Rosenman freebsd_committer 2019-07-12 19:24:31 UTC
Fix committed.
Comment 4 Marek 2019-07-12 20:17:56 UTC
Confirmed, spamming logs has gone :)
Comment 5 VVD 2019-07-15 17:27:00 UTC
Still spam in log:
tail -f /var/log/dovecot/dovecot.log
Jul 15 19:51:24 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 19:52:39 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 19:58:43 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 20:02:23 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 20:03:57 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 20:04:44 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 20:04:44 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 20:05:24 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 20:06:39 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 20:07:27 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 20:08:28 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 20:09:31 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 20:12:23 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer
Jul 15 20:24:11 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer


Name           : dovecot
Version        : 2.3.7_1
Installed on   : Sat Jul 13 19:12:08 2019 MSK
Origin         : mail/dovecot
Architecture   : FreeBSD:10:amd64
Prefix         : /usr/local
Categories     : ipv6 mail
Licenses       : MIT, LGPL21
Maintainer     : ler@FreeBSD.org
WWW            : http://www.dovecot.org/
Comment        : Secure, fast and powerful IMAP and POP3 server
Options        :
        CDB            : off
        DOCS           : on
        EXAMPLES       : on
        GSSAPI_BASE    : off
        GSSAPI_HEIMDAL : off
        GSSAPI_MIT     : off
        GSSAPI_NONE    : on
        ICU            : on
        LDAP           : on
        LIBSODIUM      : off
        LIBWRAP        : off
        LUA            : off
        LUCENE         : off
        LZ4            : on
        MYSQL          : on
        PGSQL          : off
        SOLR           : off
        SQLITE         : off
        TEXTCAT        : off
        VPOPMAIL       : off
Comment 6 Larry Rosenman freebsd_committer 2019-07-15 17:37:11 UTC
that looks like a firewall issue....
Comment 7 VVD 2019-07-18 22:28:08 UTC
Probably it's incorrect disconnected clients.
Comment 8 Larry Rosenman freebsd_committer 2019-07-18 22:34:05 UTC
the latest port (2.3.7_3) doesn't log these any more.

from an upstream patch.
Comment 9 VVD 2019-07-19 02:52:41 UTC
(In reply to Larry Rosenman from comment #8)
Thanks, they has gone now.
Comment 10 commit-hook freebsd_committer 2019-08-29 14:48:28 UTC
A commit references this bug:

Author: ler
Date: Thu Aug 29 14:47:26 UTC 2019
New revision: 510165
URL: https://svnweb.freebsd.org/changeset/ports/510165

Log:
  MFH: r506460 r506487 r506821 r506824 r507181 r507215 r510075

  mail/dovecot, mail/dovecot-pigeonhole: Update to 2.3.7 and 0.5.7 respectively.

  dovecot changelog:
  * fts-solr: Removed break-imap-search parameter
  + Added more events for the new statistics, see
    https://doc.dovecot.org/admin_manual/list_of_events/
  + mail-lua: Add IMAP metadata accessors, see
    https://doc.dovecot.org/admin_manual/lua/
  + Add event exporters that allow exporting raw events to log files and
    external systems, see
    https://doc.dovecot.org/configuration_manual/event_export/
  + SNIPPET is now PREVIEW and size has been increased to 200 characters.
  + Add body option to fts_enforced. This triggers building FTS index only
    on body search, and an error using FTS index fails the search rather
    than reads through all the mails.
  - Submission/LMTP: Fixed crash when domain argument is invalid in a
    second EHLO/LHLO command.
  - Copying/moving mails using Maildir format loses IMAP keywords in the
    destination if the mail also has no system flags.
  - mail_attachment_detection_options=add-flags-on-save caused email body
    to be unnecessarily opened when FETCHing mail headers that were
    already cached.
  - mail attachment detection keywords not saved with maildir.
  - dovecot.index.cache may have grown excessively large in some
    situations. This happened especially when using autoexpunging with
    lazy_expunge folders. Also with mdbox format in general the cache file
    wasn't recreated as often as it should have.
  - Autoexpunged mails weren't immediately deleted from the disk. Instead,
    the deletion from disk happened the next time the folder was opened.
    This could have caused unnecessary delays if the opening was done by
    an interactive IMAP session.
  - Dovecot's TCP connections sometimes add extra 40ms latency due to not
    enabling TCP_NODELAY. HTTP and SMTP/LMTP connections weren't
    affected, but everything else was. This delay wasn't always visible -
    only in some situations with some message/packet sizes.
  - imapc: Fix various crash conditions
  - Dovecot builds were not always reproducible.
  - login-proxy: With shutdown_clients=no after config reload the
    existing connections could no longer be listed or kicked with doveadm.
  - "doveadm proxy kick" with -f parameter caused a crash in some
    situations.
  - Auth policy can cause segmentation fault crash during auth process
    shutdown if all auth requests have not been finished.
  - Fix various minor bugs leading into incorrect behaviour in mailbox
    list index handling. These rarely caused noticeable problems.
  - LDAP auth: Iteration accesses freed memory, possibly crashing
    auth-worker
  - local_name { .. } filter in dovecot.conf does not correctly support
    multiple names and wildcards were matched incorrectly.
  - replicator: dsync assert-crashes if it can't connect to remote TCP
    server.
  - config: Memory leak in config process when ssl_dh setting wasn't
    set and there was no ssl-parameters.dat file.
    This caused config process to die once in a while
    with "out of memory".

  pigeonhole changelog:
  + vacation: Made the subject for the automatic response message produced
    by the Sieve vacation action configurable. Both the default subject
    (if the script defines none) and the subject template (e.g. used to
    add a subject prefix) can be configured.
  - dsync: dsync-replication does not synchronize Sieve scripts.
  - imap_sieve_filter: Reduce FILTER=SIEVE verbosity over IMAP connection.
  - testsuite: Pigeonhole testsuite segfaulted if it was compiled with
    GCC 9

  mail/dovecot: stop spamming the log with EINVAL.

  PR:		239172
  Submitted by:	zillion1@o2.pl
  Obtained from:	dovecot mailing list.

  mail/dovecot: stop whining about TCP_NODELAY errors.

  [PATCH] lib: ostream-file: Don't log any errors when setting
   TCP_NODELAY

  It's likely never useful to log the error, and it seems more and more
  unexpected errors just keep popping up.

  Obtained from:	upstream git.

  mail/dovecot: One should actually TEST their patches.

  Fix previous commit.

  Pointy Hat To: ler

  mail/dovecot: [PATCH] lib-storage: Namespace prefix shouldn't be included in all
   mailbox name validity checks

  Obtained from:	upstream github.

  mail/dovecot, mail/dovecot-pigeonhole: upgrade to 2.3.7.1 and 0.5.7.1 respectively.

  These releases fix the reported regressions in v2.3.7 & v0.5.7.

  Dovecot core:
          - Fix TCP_NODELAY errors being logged on non-Linux OSes
          - lmtp proxy: Fix assert-crash when client uses BODY=8BITMIME
          - Remove wrongly added checks in namespace prefix checking

  Pigeonhole:
          - dsync: Sieve script syncing failed if mailbox attributes weren't
            enabled.

  mail/dovecot,mail/dovecot-pigeonhole: fix CVE-2019-11500

  Changes
  -------
  * CVE-2019-11500: IMAP protocol parser does not properly handle NUL byte
    when scanning data in quoted strings, leading to out of bounds heap
    memory writes. Found by Nick Roessler and Rafi Rubin.

  Security:	CVE-2019-11500

  PR: 	241074
  Approved by:	ports-secteam (delphij, joneum)

Changes:
_U  branches/2019Q3/
  branches/2019Q3/mail/dovecot/Makefile
  branches/2019Q3/mail/dovecot/distinfo
  branches/2019Q3/mail/dovecot/pkg-plist
  branches/2019Q3/mail/dovecot-pigeonhole/Makefile
  branches/2019Q3/mail/dovecot-pigeonhole/distinfo