Bug 76626

Summary: 460.status-mail-rejects shows destination domain instead of source IP
Product: Base System Reporter: rnejdl
Component: confAssignee: freebsd-bugs (Nobody) <bugs>
Status: Open ---    
Severity: Affects Only Me CC: Alexander88207
Priority: Normal    
Version: Unspecified   
Hardware: Any   
OS: Any   

Description rnejdl 2005-01-24 19:10:21 UTC
/usr/src/etc/periodic/daily/460.status-mail-rejects

Checking for rejected mail hosts:
  25 atshaw.com (451... resolve)
  24 EMAILHOSTER.COM (550... http://www.spamhaus.org/SBL)
  22 cohesionventures.com (550... denied)
  19 cohesionventures.com (550... server)
  18 matronics.com (550... denied)
  16 ringofsaturn.com (550... server)
  15 atshaw.com (550... denied)
  13 atshaw.com (550... server)
  12 ringofsaturn.com (550... denied)
   9 danicfinancial.com (451... resolve)
   6 cohesionventures.com (553... Corporation)
   5 ringofsaturn.com (553... Corporation)
   5 cohesionventures.com (550... http://www.spamhaus.org/SBL)
   5 atshaw.com (553... IP's)
   4 emailhoster.com (550... denied)
   4 ATSHAW.COM (550... http://www.spamhaus.org/SBL)
   3 tethys.ringofsaturn.com (550... denied)
   3 saturnconsulting.com (550... server)
   3 saturnconsulting.com (550... denied)
   3 cohesionventures.com (553... IP's)
   3 atshaw.com (553... Corporation)
   3 atshaw.com (553... Clients)
   2 tethys.ringofsaturn.com (553... IP's)
   2 ringofsaturn.com (553... IP's)
   2 ringofsaturn.com (553... Brazil)
   2 ringofsaturn.com (550... http://www.spamhaus.org/SBL)
   2 emailhoster.com (550... server)
   2 cohesionventures.com (553... #Spammer)
   2 authentickungfudallas.com (550... server)
   2 atshaw.com (553... Users)
   1 ringofsaturn.com (553... exist)
   1 ringofsaturn.com (550... 218.219.154.210)
   1 ringofsaturn.com (550... 204.9.210.123)
   1 ringofsaturn.com (451... resolve)
   1 ringo.fsbusiness.co.uk (550... [61.11.26.142])
   1 hydrolawn.com (553... IP's)
   1 hydrolawn.com (550... server)
   1 helixdfw.com (553... IP's)
   1 emailhoster.com (553... IP's)
   1 emailhoster.com (553... Brazil)
   1 emailhoster.com (550... 64.14.48.142)
   1 emailhoster.com (550... 64.14.48.133)
   1 dinhglobal.com (550... server)
   1 cohesionventures.com (553... users)
   1 cohesionventures.com (553... exist)
   1 cohesionventures.com (553... bounced.)
   1 cohesionventures.com (553... Brazil)
   1 authentickungfudallas.com (553... Spammer)
   1 authentickungfudallas.com (553... Brazil)
   1 authentickungfudallas.com (550... denied)
   1 atshawdot.ca (550... [62.14.104.36])
   1 atshawdot.ca (550... [61.11.26.142])
   1 atshaw.dotca (550... [202.54.51.5])
   1 atshaw.com (553... exist)
   1 atshaw.com (553... Spammer)
   1 atshaw.com (553... #Spammer)

This is a list of the destination domains.  I want to see instead a list of the hosts that have been rejected.

Fix: I have solved the problem using gawk, which isn't acceptible for normal installs as gawk is a port.  However, perhaps this solution can be adapted to work correctly for the normal install.

[tethys]:/home/rnejdl> diff -u /etc/periodic/daily/460.status-mail-rejects /usr/src/etc/periodic/daily/460.status-mail-rejects


# If there is a global system configuration file, suck it in.
@@ -32,57 +32,27 @@
            echo
            echo Checking for rejected mail hosts:

-       #    rc=$({
-       #       for f in `find /var/log -name maillog\* \
-       #               \( -mtime 1 -o -mtime 2 \) | xargs ls -tr`
-       #       do
-       #               case $f in
-       #                       *.gz)   zcat -fc $f;;
-       #                       *.bz2)  bzip2 -cd $f;;
-       #                       *)      cat $f;;
-       #               esac
-       #       done
-           start=`date -v-1d '+%b %d' | sed 's/0\(.\)$/ \1/'`
+           start=`date -v-1d '+%b %e'`
            n=$(($daily_status_mail_rejects_logs - 2))
            rc=$({
                while [ $n -ge 0 ]
-                do
-                    if [ -f /var/log/maillog.$n ]
-                    then
-                        cat /var/log/maillog.$n
-                    elif [ -f /var/log/maillog.$n.gz ]
-                    then
-                        zcat -fc /var/log/maillog.$n.gz
-                    elif [ -f /var/log/maillog.$n.bz2 ]
-                    then
-                        bzcat -fc /var/log/maillog.$n.bz2
-                    fi
-                    n=$(($n - 1))
-                done
-                cat /var/log/maillog
-           } | /usr/local/bin/gawk '
-               BEGIN {
-                   today=systime();
-                   yesterday=strftime("%b %d", today-86400);
-                   today=strftime("%b %d", today);
-                   gsub(" 0", "  ", today); gsub(" 0", "  ", yesterday);
-               }
-               {
-                   relay=gensub("^" yesterday ".*, relay=([^,]+), reject=.*",
-                               "\\1", 1);
-                   if (relay != $0)
-                       rejects[relay]++;
-                   else if (match($0, "^" today))
-                       exit;
-               }
-               END {
-                   for (relay in rejects) {
-                       printf("%4d %s\n", rejects[relay], relay);
-                       total += rejects[relay];
-                   }
-                   if (total > 0)
-                       printf("%4d TOTAL\n", total);
-               }' | sort -fnr | tee /dev/stderr | wc -l)
+               do
+                   if [ -f /var/log/maillog.$n ]
+                   then
+                       cat /var/log/maillog.$n
+                   elif [ -f /var/log/maillog.$n.gz ]
+                   then
+                       zcat -fc /var/log/maillog.$n.gz
+                   elif [ -f /var/log/maillog.$n.bz2 ]
+                   then
+                       bzcat -fc /var/log/maillog.$n.bz2
+                   fi
+                   n=$(($n - 1))
+               done
+               cat /var/log/maillog
+           } |
+               sed -n -E "s/^$start"'.*ruleset=check_[^ ]+, +arg1=<?([^@]+@)?([^>,]+).*reject=([^ ]+) .* ([^ ]+)$/\2 (\3... \4)/p' |
+               sort -f | uniq -ic | sort -fnr | tee /dev/stderr | wc -l)
            [ $rc -gt 0 ] && rc=1
        fi;;

Exit 1--RXWwNmR05LWlrH5TlulQ9pGKWh22IR4AiHIMFcNNEJXQhrXY
Content-Type: text/plain; name="file.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="file.diff"

--- /etc/periodic/daily/460.status-mail-rejects Sun Oct 10 13:13:34 2004
+++ /usr/src/etc/periodic/daily/460.status-mail-rejects Mon Jan 24 12:55:07 2005
@@ -1,6 +1,6 @@
 #!/bin/sh
 #
-# $FreeBSD: src/etc/periodic/daily/460.status-mail-rejects,v 1.8.2.5 2002/05/13 21:36:44 brian Exp $
+# $FreeBSD: src/etc/periodic/daily/460.status-mail-rejects,v 1.16.4.1 2005/01/24 14:44:47 brian Exp $
 #
How-To-Repeat: Simply execute the command with a default sendmail installation.  Here's an example of a reject line:

Jan 24 12:58:17 tethys sm-mta[79791]: j0OIviDL079791: ruleset=check_rcpt, arg1=<atshaw@atshaw.com>, relay=[210.187.94.17], reject=550 5.7.1 <atshaw@atshaw.com>... Fix reverse DNS for 210.187.94.17,or use your ISP server

The relay should be shown by periodic script, not atshaw.com.
Comment 1 Gregory Neil Shapiro freebsd_committer freebsd_triage 2007-06-15 05:00:04 UTC
Your patch assumes that only the relay= is of interest in the list.
However, in my opinion, the address you are rejecting is more interesting
in most cases.  For example, from my own logs:

i un 14 00:01:32 gir sm-mta[9280]: l5E71S9N009280: ruleset=check_mail, arg1=<tzdelhi@netbizmoms.com>, relay=ip-51.net-82-216-27.versailles2.rev.numericable.fr [82.216.27.51], reject=451 4.1.8 Domain of sender address tzdelhi@netbizmoms.com does not resolve

Jun 14 00:05:17 gir sm-mta[9349]: l5E75ErZ009349: ruleset=check_rcpt, arg1=<benco@example.com>, relay=ful.cnchost.com [297.157.49.28], reject=400 4.0.0 Temporary failure

Jun 14 00:12:13 gir sm-mta[9552]: l5E7C812009552: ruleset=check_mail, arg1=<newtripod.com@wonfuproductions.com>, relay=dsl081-247-036.sfo1.dsl.speakeasy.net [64.81.247.36], reject=450 4.1.2 <newtripod.com@wonfuproductions.com>... MX lookup failure for wonfuproductions.com

In all three cases, I'm more interested in the address that was rejected
instead of the host sending that mail.
Comment 2 rnejdl 2007-06-16 13:01:19 UTC
Gregory,

Well, to me, since most of the time when I am rejecting emails, the sender
email address is spoofed.  I am definitely more interested in knowing what
ISP's are spamming me most.

[tethys]:/home/rnejdl> /etc/periodic/daily/460.status-mail-rejects

Checking for rejected mail hosts:
 289 TOTAL
   4 [208.97.234.204]
   3 thisistoyou.com [208.66.235.120]
   3 dropspecials.com [69.30.230.84]
   3 [203.156.49.110]
   2 mx1.gatetowinner.com [64.71.164.137]
   2 hn.kd.dhcp [61.52.201.38] (may be forged)
   2 chhor.brillianticon.com [70.42.184.61]

So, when I blocked 208.97.234.204, I managed to block 4 spams from that IP
in the last 24 hours.

To be honest, the display as it is shown below is of no use to me.  It
would be great if we could have a way to configure it to show the field
that you wanted displayed.

Sincerely,
Rusty Nejdl

Gregory Shapiro wrote:
> Your patch assumes that only the relay= is of interest in the list.
> However, in my opinion, the address you are rejecting is more interesting
> in most cases.  For example, from my own logs:
>
> i un 14 00:01:32 gir sm-mta[9280]: l5E71S9N009280: ruleset=check_mail,
> arg1=<tzdelhi@netbizmoms.com>,
> relay=ip-51.net-82-216-27.versailles2.rev.numericable.fr [82.216.27.51],
> reject=451 4.1.8 Domain of sender address tzdelhi@netbizmoms.com does not
> resolve
>
> Jun 14 00:05:17 gir sm-mta[9349]: l5E75ErZ009349: ruleset=check_rcpt,
> arg1=<benco@example.com>, relay=ful.cnchost.com [297.157.49.28],
> reject=400 4.0.0 Temporary failure
>
> Jun 14 00:12:13 gir sm-mta[9552]: l5E7C812009552: ruleset=check_mail,
> arg1=<newtripod.com@wonfuproductions.com>,
> relay=dsl081-247-036.sfo1.dsl.speakeasy.net [64.81.247.36], reject=450
> 4.1.2 <newtripod.com@wonfuproductions.com>... MX lookup failure for
> wonfuproductions.com
>
> In all three cases, I'm more interested in the address that was rejected
> instead of the host sending that mail.
>
Comment 3 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:59:56 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped
Comment 4 Graham Perrin freebsd_committer freebsd_triage 2022-10-17 12:34:45 UTC
Keyword: 

    patch
or  patch-ready

– in lieu of summary line prefix: 

    [patch]

* bulk change for the keyword
* summary lines may be edited manually (not in bulk). 

Keyword descriptions and search interface: 

    <https://bugs.freebsd.org/bugzilla/describekeywords.cgi>