Bug 210457 - Mail forwarded from list to members fails DKIM/SPF/DMARC authentication on recipient side
Summary: Mail forwarded from list to members fails DKIM/SPF/DMARC authentication on re...
Status: Open
Alias: None
Product: Services
Classification: Unclassified
Component: Mailing Lists (show other bugs)
Version: unspecified
Hardware: Any Any
: --- Affects Many People
Assignee: postmaster
URL: https://wiki.list.org/DEV/DKIM
Keywords: needs-qa, performance
Depends on:
Blocks:
 
Reported: 2016-06-22 08:33 UTC by VK
Modified: 2018-05-25 01:11 UTC (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description VK freebsd_triage 2016-06-22 08:33:29 UTC
It appears the original sender's DKIM is not stripped and of course it fails validating mail, for mails sent to the list and forwarded to list recipients.

This can result in anything from not being a big deal, to FreeBSD's MTA IPs ending up on black lists for bad reputation. My knowledge of the Mailman system is virtually zero, but quick googling revealed something like this:

https://wiki.list.org/DEV/DKIM

Which makes sense. Sender DKIM should be validated by the FreeBSD MTA subsystem, then stripped by Mailman, then FreeBSD's DKIM slapped up by FreeBSD MTA subsystem on outgoing mail.

I'm available to help with testing.
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2016-06-22 08:37:04 UTC
See Also:

DomainKeys Identified Mail (DKIM) and Mailing Lists
https://tools.ietf.org/html/rfc6377
Comment 2 VK freebsd_triage 2016-10-28 17:23:59 UTC
Adjusting summary for a more precise description of the problem.

Postmater, is SRS in effect on FreeBSD servers? (https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme) I don't see it in the headers of mail I receive from the lists.

I believe that should help?
Comment 3 VK freebsd_triage 2016-11-30 09:14:38 UTC
Is anyone from postmaster@ available for a quick looksie and response to comment #2? (cc clusteradm@, not sure if postmaster@ is getting these....)
Comment 4 Peter Wemm freebsd_committer freebsd_triage 2016-11-30 17:11:21 UTC
It looks like Mailman is set to:

# Default action for posts whose From: address domain has a DMARC policy of
# reject or quarantine.  See DEFAULT_FROM_IS_LIST below.  Whatever is set as
# the default here precludes the list owner from setting a lower value.
# 0 = Accept
# 1 = Munge From
# 2 = Wrap Message
# 3 = Reject
# 4 = Discard
DEFAULT_DMARC_MODERATION_ACTION = 1

It doesn't appear to do anything with DKIM unless DMARC is set to hard-fail.
Comment 5 VK freebsd_triage 2016-12-01 10:58:28 UTC
Well, I'm not sure which option is best for that, but if "Munge From" is the current setting, I don't think it's happening. I just checked the last mail I sent to the list and the list sent back to me, "From" is kept intact:

> Return-Path: <owner-freebsd-ports@freebsd.org>
> To: Freebsd Ports <freebsd-ports@freebsd.org>
> Subject: (In)Stability of the Quarterly Branch
> From: "Vlad K." <vlad-fbsd@...>
> X-Sender: vlad-fbsd@...
> X-BeenThere: freebsd-ports@freebsd.org
> X-Mailman-Version: 2.1.23
> Precedence: list
> List-Id: Porting software to FreeBSD <freebsd-ports.freebsd.org>
> Sender: owner-freebsd-ports@freebsd.org

I'm looking at the meaning of DEFAULT_DMARC_MODERATION_ACTION here:

* https://wiki.list.org/DEV/DMARC

I wonder if wrapping the message would make it more correct and more deliverable in today's context of spam protection... Because right now, every time I send to the list I get a ton of DMARC violation reports sent to me.

If SRS is to be used, I don't know if Mailman can do it, but it can certainly be done at the Postfix level.

* https://github.com/roehling/postsrsd
Comment 6 Peter Wemm freebsd_committer freebsd_triage 2016-12-01 19:11:53 UTC
The problem is:
".. domain has a DMARC policy of reject or quarantine."

You have fo=1 (send reports), but have p=none (not p=quarantine or reject) so the From: munging isn't enabled.  It is actually working as documented.

As a counter example of it working:
_dmarc.yahoo.it descriptive text "v=DMARC1; p=reject; pct=100; rua=mailto:dmarc"
..
and on October 21, on freebsd-stable@, "boot0cfg on does not set default selection on gmirror device"

From: Arrigo ... via freebsd-stable <freebsd-stable@freebsd.org>
Reply-To: Arrigo ... <...@yahoo.it>

However, in that message, while the From: was wrapped, the dkim metadata wasn't stripped.  It did trigger a dkim failure, but the dmarc policy didn't force the rejection:
Authentication-Results: myhost...;
	dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=yahoo.it header.i=@yahoo.it header.b=eFPsTyuQ

Hmmm.
Comment 7 Fukang Chen freebsd_committer 2018-05-08 06:11:44 UTC
https://lists.freebsd.org/pipermail/freebsd-current/2018-May/069306.html

Hi, it seems like "Munge From" works for the freebsd-arm@ list, but freebsd-current@ not.

% perl -MNet::NNTP -e '$n=Net::NNTP->new(q|news.gmane.org|, SSL=>1); $n->group(q|gmane.os.freebsd.devel.arm|); print @{$n->article(q|<9673BD00-6874-4C00-8532-115D524786C2@yahoo.com>|)}' | egrep '^(From|Subject|Date|List-Id):'

From: Mark Millard via freebsd-arm <freebsd-arm@freebsd.org>
Subject: Allwinner A83T BananaPi M3 Board v1.2 early boot failures: "USB0:
Date: Sun, 29 Apr 2018 06:13:34 -0700
List-Id: "Porting FreeBSD to ARM processors." <freebsd-arm.freebsd.org>

% perl -MNet::NNTP -e '$n=Net::NNTP->new(q|news.gmane.org|, SSL=>1); $n->group(q|gmane.os.freebsd.devel.current|); print @{$n->article(q|<8E3C5DFF-BC87-4822-9A35-BF206A735EAA@yahoo.com>|)}' | egrep '^(From|Subject|Date|List-Id):'

From: Mark Millard <marklmi26-fbsd@yahoo.com>
Subject: Re: svn commit: r333240 - in head/sys: powerpc/powerpc sys [appears
Date: Sun, 6 May 2018 19:33:34 -0700
List-Id: Discussions about the use of FreeBSD-current
Comment 8 Kurt Jaeger freebsd_committer 2018-05-09 18:17:54 UTC
pi from postmaster@ team speaking:

I checked both list configs, and both have this parameter set to "no":

Replace the From: header address with the list's posting address 
to mitigate issues stemming from the original From: domain's DMARC
or similar policies.

If you look in our archives:

  https://lists.freebsd.org/pipermail/freebsd-arm/2018-April/017864.html
has
  Mark Millard marklmi26-fbsd at yahoo.com 

and

  https://lists.freebsd.org/pipermail/freebsd-current/2018-May/069306.html
has
  Mark Millard marklmi26-fbsd at yahoo.com 

so the header was munged somewhere else (if I did not misunderstood something).
Comment 9 Fukang Chen freebsd_committer 2018-05-10 02:04:38 UTC
Hi Kurt, thanks for checking the configs.
Comment 10 Fukang Chen freebsd_committer 2018-05-10 09:27:02 UTC
(In reply to Kurt Jaeger from comment #8)

Hi Kurt,

Sorry to bug you. You were right, the header was munged somewhere else. The "from_is_list" option munges all the messages, but there are only a few messages have a "via freebsd-arm" From: header in the freebsd-arm@ list, with a sender address like yahoo.com or mail.ru.

I think it's the option "dmarc_moderation_action = 1", it looks up the DMARC record and set the msgdata['from_is_list'] = 1 when there's a p=reject or p=quarantine found:
https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/view/head:/Mailman/Handlers/SpamDetect.py#L102

Thanks,
loader
Comment 11 Mark Millard 2018-05-24 07:10:55 UTC
Since my messages are being used as examples here, I
note the following relative to my yahoo.com context:

I had used a so-called "disposable email address"
(marklmi26-fbsd) instead of the main one (without
the "26-fbsd"). The definition given for the
disposable type of email address is:

"Create an email address to sign up for third-
party newsletters. Delete account to stop
receiving."


I have just swapped my FreeBSD list Email binding to
be just the normal marklmi at yahoo.com one to see how
it goes. This is based on some experiments that Eitan
Adler helped me with: I sent Email with various
combinations for the account name sent from and with
or without a FreeBSD list also being sent to, but
always sending to Eitan directly as well. (This started
when Eitan reported one I'd sent directly and to a
a list as well ended up as spam.)

Initially it appears that marklmi at yahoo.com gave
Eitan no problems for the same list being involved
where, for marklmi26-fbsd at yahoo.com as the sender,
Email was classified as spam in Eitan's context.

(Sending just directly to Eitan, everything went through
as normal Email, not spam. When I added also sending
to the list then there was a spam classification for
marklmi26-fbsd at yahoo.com as the sender.)

We will see how it goes. But there may be a rule-of-use
here: avoid using a yahoo "disposable email address"
as the Email address for joining lists and for sending
to lists.
Comment 12 Mark Millard 2018-05-25 01:00:31 UTC
(In reply to Mark Millard from comment #11)

Multiple people that I did not send directly to but that
got messages indirectly via a list report that the change
to use marklmi at yahoo.com made no difference: still
classified as spam.

So far it looks like the change to avoid the disposable
Email address only helped when there was a mix of both
a direct send and a list being sent to as well.
Comment 13 Mark Millard 2018-05-25 01:11:40 UTC
Gary Jennehohn sent me direct Email reported finding a
"(Client did not present a certificate)". This was for
an example based on marklmi26-fbsd at yahoo.com (the
so-called disposable Email address in yahoo terms).
Shortening his material some (and replacing some @'s
with " at "s):

Received: from sonic307-12.consmr.mail.ne1.yahoo.com
(sonic307-12.consmr.mail.ne1.yahoo.com [66.163.190.35])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client did not present a certificate) **** looks suspicious ****
by mx1.freebsd.org (Postfix) with ESMTPS id E97078874A
for <freebsd-current at freebsd.org>; Mon,  7 May 2018 02:53:53 +0000 (UTC)
(envelope-from marklmi26-fbsd at yahoo.com)
X-YMail-OSG: . . .
Received: from sonic.gate.mail.ne1.yahoo.com by
sonic307.consmr.mail.ne1.yahoo.com with HTTP; Mon, 7 May 2018 02:53:52 +0000
Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.158])
([76.115.7.162])
by smtp424.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID
65a09db5e9e52ef6b35440b2fc441c41; 
Mon, 07 May 2018 02:33:36 +0000 (UTC)
From: Mark Millard <marklmi26-fbsd at yahoo.com>
Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\))