DMARC_REJECT?

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

DMARC_REJECT?

Amir Caspi
Hi all,

        Over the past few weeks I've been getting occasional DMARC_REJECT hits on valid mail (e.g., from family, from valid bank emails, etc.).  It's unclear why this is happening since the SA report doesn't really give any information.  The score on DMARC_REJECT is +10, which is enough to throw these legit emails into spam even with BAYES_00.  (Those senders are also in whitelist_auth, but that rule isn't hitting... I'm guessing the DMARC failure is causing this to be considered non-auth, hence not whitelisted?  Note that SPF_HELO_PASS hits fine.)

Can someone point me to a log message I can look up to figure out why those messages are failing DMARC?  In the meantime, I'll probably have to reduce the score on that to avoid it being such a poison pill.

I'm not posting publicly-available samples here because these emails are ham, and therefore can contain sensitive info.  If someone like Kevin or John need specific info I can forward specific samples privately (off-list).

Thanks!

--- Amir

Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Bill Cole
On 14 Nov 2019, at 0:14, Amir Caspi wrote:

> DMARC_REJECT

Is not the name of any rule currently distributed by the Apache
SpamAssassin project...

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Dominic Raferd


On Thu, 14 Nov 2019 at 05:49, Bill Cole <[hidden email]> wrote:
On 14 Nov 2019, at 0:14, Amir Caspi wrote:

> DMARC_REJECT

Is not the name of any rule currently distributed by the Apache
SpamAssassin project...

This comes from an update to KAM.cf in the last few weeks. It briefly caused me problems because I pass all authenticated/local mails through SA and these all started being scored +10. My solution was to add a line in my local.cf:
score DMARC_REJECT 0

This works for me because I run opendmarc as milter - any emails that non-auth/local and which fail DMARC with p=reject will be blocked anyway.
Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Henrik K
On Thu, Nov 14, 2019 at 06:23:32AM +0000, Dominic Raferd wrote:

>
>
> On Thu, 14 Nov 2019 at 05:49, Bill Cole <[1]
> [hidden email]> wrote:
>
>     On 14 Nov 2019, at 0:14, Amir Caspi wrote:
>
>     > DMARC_REJECT
>
>     Is not the name of any rule currently distributed by the Apache
>     SpamAssassin project...
>
>
> This comes from an update to KAM.cf in the last few weeks. It briefly caused me
> problems because I pass all authenticated/local mails through SA and these all
> started being scored +10. My solution was to add a line in my [2]local.cf:
> score DMARC_REJECT 0
>
> This works for me because I run opendmarc as milter - any emails that non-auth/
> local and which fail DMARC with p=reject will be blocked anyway.

Seems KAM.cf doesn't even check if DKIM/SPF plugins are loaded. :-(

meta DMARC_REJECT !(DKIM_VALID_AU || SPF_PASS) && __DMARC_POLICY_REJECT

Prolly should look something like (DKIM_INVALID || SPF_FAIL) && __DMARC_POLICY_REJECT.

Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Benny Pedersen-4
In reply to this post by Amir Caspi
On 2019-11-14 06:14, Amir Caspi wrote:

> I'm not posting publicly-available samples here because these emails
> are ham, and therefore can contain sensitive info.  If someone like
> Kevin or John need specific info I can forward specific samples
> privately (off-list).

how do you make DMARC in default spamassassin ?, and last point is
spamassassin does not REJECT
Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Benny Pedersen-4
In reply to this post by Dominic Raferd
On 2019-11-14 07:23, Dominic Raferd wrote:

> This works for me because I run opendmarc as milter - any emails that
> non-auth/local and which fail DMARC with p=reject will be blocked
> anyway.

why do you run milters for maillist seender ips ?

never seen domains with dmarc reject policy while some maillists breaks
dkim ?

this emails posts to maillists can potently unsubscribe all users if
dmarc policy reject is respected

in my postfix i have make smtpd_milter_maps with all known maillists ips
to prevent reject on dmarc dkim and any other silly things
Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Benny Pedersen-4
In reply to this post by Henrik K
On 2019-11-14 07:49, Henrik K wrote:

> Prolly should look something like (DKIM_INVALID || SPF_FAIL) &&
> __DMARC_POLICY_REJECT.

no hope for AuthRes plugin :=)
Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Kevin A. McGrail-5
In reply to this post by Dominic Raferd
The DMARC Reject rule is about whether a domain has failed DKIM and has
a DMARC reject policy.  I will add descriptions to these rules ASAP. 
Thanks.

We have encapsulated the rules in a check for DKIM and SPF.

Best to report issues with KAM.cf as noted in the file :-)  Happy to
look at samples.  I would imagine you might have something breaking DKIM
in your environment with FPs as my first guess.  We've had this in
production and so does another ISP with no other FPs reported.

Regards,

KAM

--
Kevin A. McGrail
[hidden email]

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Benny Pedersen-4
On 2019-11-15 07:35, Kevin A. McGrail wrote:
> The DMARC Reject rule is about whether a domain has failed DKIM and has
> a DMARC reject policy.  I will add descriptions to these rules ASAP. 
> Thanks.

super, it could drop the pants on mailling lists where spf, dkim, dmarc
breaks mostly, when direct mailling it mostly works, magical :=)

how many mta / esp do still use sid-milter and clame mailling lists
breaks spf ?, bad excusses for beliving sender-id is same as spf

lets see what the new mimedefang will do with it, in spamassassin i like
to see more fokus on AuthRes

> We have encapsulated the rules in a check for DKIM and SPF.
>
> Best to report issues with KAM.cf as noted in the file :-)

i will stop my rants of a developper that have write access to
apache.org and dont want to let stable rules be part of corpa testing
before pubolicy its contents, seem users that have no write access build
better rules that does not hit on breaked dkim on mailling lists but do
hit on direct mailling if dmarc failing

>  Happy to
> look at samples.  I would imagine you might have something breaking
> DKIM
> in your environment with FPs as my first guess.  We've had this in
> production and so does another ISP with no other FPs reported.

how more longer will it take to make 3.4.3 realease by looking ? :=)
Reply | Threaded
Open this post in threaded view
|

re: 3.4.3 release Re: DMARC_REJECT?

Kevin A. McGrail-5

> how more longer will it take to make 3.4.3 realease by looking ? :=)

Right now, I'm working through bugs found by myself and my staff to make
sure the release is up to snuff.

The biggest delay is a lack of feedback and testing.  Please grab rc6
and give feedback.

Regards,

KAM

--
Kevin A. McGrail
[hidden email]

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Reply | Threaded
Open this post in threaded view
|

Re: 3.4.3 release Re: DMARC_REJECT?

David Jones
On 11/15/19 1:02 AM, Kevin A. McGrail wrote:
>
>> how more longer will it take to make 3.4.3 realease by looking ? :=)
>
> Right now, I'm working through bugs found by myself and my staff to make
> sure the release is up to snuff.
>
> The biggest delay is a lack of feedback and testing.  Please grab rc6
> and give feedback.
>

I have been running 3.4.3 rc6 for a few days in production and no
problems so far on my cluster of 12 SA servers with a pretty good volume
of emails (about 600,000 per day hit SA).

--
David Jones
Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

David Jones
In reply to this post by Kevin A. McGrail-5
On 11/15/19 12:35 AM, Kevin A. McGrail wrote:

> The DMARC Reject rule is about whether a domain has failed DKIM and has
> a DMARC reject policy.  I will add descriptions to these rules ASAP.
> Thanks.
>
> We have encapsulated the rules in a check for DKIM and SPF.
>
> Best to report issues with KAM.cf as noted in the file :-)  Happy to
> look at samples.  I would imagine you might have something breaking DKIM
> in your environment with FPs as my first guess.  We've had this in
> production and so does another ISP with no other FPs reported.
>

While I am for this rule helping all SA instances with KAM.cf added,
it's pretty risky to put this rule in with a default score higher than
1.0 as there are so many ways that SA can be launched/integrated.

Perhaps it needs to be named KAM_DMARC_REJECT to make it obvious that it
came from the KAM.cf and have a default score of 0.001?

I have my own rule for DMARC_REJECT that is tied closely to the headers
added by OpenDMARC which is going to be more reliable / less risky due
to it being linked to the MTA as a milter.

If SA is being run post MTA (i.e. inside Thunderbird) then any filtering
can change the content to remove potentially bad attachments, add an
"EXTERNAL" warning to the Subject or body, etc. which will break DKIM
signing.

--
David Jones
Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Amir Caspi
On Nov 15, 2019, at 9:50 AM, David Jones <[hidden email]> wrote:
>
> If SA is being run post MTA (i.e. inside Thunderbird) then any filtering
> can change the content to remove potentially bad attachments, add an
> "EXTERNAL" warning to the Subject or body, etc. which will break DKIM
> signing.

I believe this is what’s happening on my FPs. My mail flow is sendmail to MailScanner to SA (spamc) via procmail, and MS will do some content altering (e.g. to disable web bugs or reveal potential phishing links). That breaks DKIM. I could disable those features but that obviates half the point of MS. If I could swap the order of MS and SA that would resolve this issue... but I’m not sure if that’s possible with my setup. (I know MS can call SA from within its flow but it doesn’t use spamc/spamd and I think can not accommodate per-user prefs.)

The other FP I’ve seen is forwarded mail, I’m not sure why DKIM broke there because I didn’t see evidence of MS munging. Will have to examine more closely.

Cheers.

--- Amir
thumbed via iPhone
Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Kevin A. McGrail-5
In reply to this post by David Jones
Good idea.  This is done.

On 11/15/2019 11:49 AM, David Jones wrote:
> Perhaps it needs to be named KAM_DMARC_REJECT to make it obvious that it
> came from the KAM.cf and have a default score of 0.001?

--
Kevin A. McGrail
[hidden email]

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

RW-15
In reply to this post by Amir Caspi
On Fri, 15 Nov 2019 10:02:48 -0700
Amir Caspi wrote:

> On Nov 15, 2019, at 9:50 AM, David Jones <[hidden email]> wrote:
> >
> > If SA is being run post MTA (i.e. inside Thunderbird) then any
> > filtering can change the content to remove potentially bad
> > attachments, add an "EXTERNAL" warning to the Subject or body, etc.
> > which will break DKIM signing.  
>
> I believe this is what’s happening on my FPs. My mail flow is
> sendmail to MailScanner to SA (spamc) via procmail, and MS will do
> some content altering (e.g. to disable web bugs or reveal potential
> phishing links). That breaks DKIM. I could disable those features but
> that obviates half the point of MS. If I could swap the order of MS
> and SA that would resolve this issue... but I’m not sure if that’s
> possible with my setup. (I know MS can call SA from within its flow
> but it doesn’t use spamc/spamd and I think can not accommodate
> per-user prefs.)
>
> The other FP I’ve seen is forwarded mail, I’m not sure why DKIM broke
> there because I didn’t see evidence of MS munging. Will have to
> examine more closely.


The rule  is

meta DMARC_REJECT !(DKIM_VALID_AU || SPF_PASS) && __DMARC_POLICY_REJECT

DKIM_VALID_AU is too strict for DMARC as it requires strict alignment.
OTOH SPF_PASS requires no alignment at all which should eliminate most
FPs on incoming mail - including most forwarded mail. DKIM should only
rarely make a difference on ham.


Do you have something preventing SPF from working correctly?


It would be useful to include  ALL_TRUSTED in these rules to handle
local and outgoing mail better.

I would also include __RP_MATCHES_RCVD.


Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Benny Pedersen-4
On 2019-11-16 00:35, RW wrote:

> meta DMARC_REJECT !(DKIM_VALID_AU || SPF_PASS) && __DMARC_POLICY_REJECT

this is when not aligned and domain owner want to reject

adding another meta with MAILING_LIST_MULTI included does not harm imho
Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Amir Caspi
In reply to this post by RW-15
On Nov 15, 2019, at 4:35 PM, RW <[hidden email]> wrote:

DKIM_VALID_AU is too strict for DMARC as it requires strict alignment.

Indeed, although I wonder if DKIM_VALID_AU is itself too strict?  In particular, one sender that triggers this issue is coming from a .gov 3rd-level subdomain where the valid DKIM signature specifies only the 2LD, i.e.,

DKIM d=domain.gov

So SA recognizes DKIM_VALID but does _not_ recognize DKIM_VALID_AU even though the sender is from a legit subdomain of the DKIM signer.  (And, the signer is in the DKIM WL, so __DKIM_RELIABLE is also hitting.)  It would seem to me that a situation like the above should hit DKIM_VALID_AU as the sender address is an authorized subdomain, and the signing (higher-level) domain is reputable.

Or, as you say, it should be based on DKIM_VALID and not DKIM_VALID_AU.

OTOH SPF_PASS requires no alignment at all which should eliminate most
FPs on incoming mail - including most forwarded mail. DKIM should only
rarely make a difference on ham. 

Do you have something preventing SPF from working correctly?

Apparently, yes.  It appears that some incoming messages are missing the Return-Path header that sendmail is supposed to insert.  This appears intermittent -- most messages have it, but some do not, and it varies even for messages from the same sender, so I have not yet been able to identify a pattern.

After exhaustive googling, it appears that this is likely related to MailScanner, because it picks up the message from sendmail before final delivery, does its processing, then lets sendmail do final delivery... and in the process it looks like it kills the Return-Path header in some emails, for reasons unknown.  I've filed a bug with them, resolution TBD.

In the meantime, I also discovered that SA provides support for non-standard Envelope-From headers, so I've directed it to use the one that MailScanner inserts (X-*-MailScanner-From), and that seems to be working... so SPF_PASS should be working correctly now for all messages.  I still want to figure out why MS is killing the correct Return-Path header in those intermittent cases, but it appears I at least have a workaround for now...

So, at the very least, this workaround should enable SPF_PASS and eliminate FPs due to that.

But I agree that DMARC_REJECT should be based either on DKIM_VALID (without _AU), or DKIM_VALID_AU should allow qualified subdomains of the signing domain.

Thanks!

--- Amir


Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Dominic Raferd
In reply to this post by Kevin A. McGrail-5


On Fri, 15 Nov 2019 at 21:17, Kevin A. McGrail <[hidden email]> wrote:
Good idea.  This is done.

On 11/15/2019 11:49 AM, David Jones wrote:
> Perhaps it needs to be named KAM_DMARC_REJECT to make it obvious that it
> came from the KAM.cf and have a default score of 0.001?

I believe only the renaming has been done, the default score remains 10; so anyone overriding the default score (that would be, er, me) needs to update their local settings for the new name.
Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

Kevin A. McGrail-5

Yeah, I'm going to lower it while we look into RW's suggestions about the rule.

On 11/16/2019 1:19 AM, Dominic Raferd wrote:


On Fri, 15 Nov 2019 at 21:17, Kevin A. McGrail <[hidden email]> wrote:
Good idea.  This is done.

On 11/15/2019 11:49 AM, David Jones wrote:
> Perhaps it needs to be named KAM_DMARC_REJECT to make it obvious that it
> came from the KAM.cf and have a default score of 0.001?

I believe only the renaming has been done, the default score remains 10; so anyone overriding the default score (that would be, er, me) needs to update their local settings for the new name.
-- 
Kevin A. McGrail
[hidden email]

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171
Reply | Threaded
Open this post in threaded view
|

Re: DMARC_REJECT?

David Jones
In reply to this post by Dominic Raferd
On 11/16/19 12:19 AM, Dominic Raferd wrote:

>
>
> On Fri, 15 Nov 2019 at 21:17, Kevin A. McGrail <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Good idea.  This is done.
>
>     On 11/15/2019 11:49 AM, David Jones wrote:
>      > Perhaps it needs to be named KAM_DMARC_REJECT to make it obvious
>     that it
>      > came from the KAM.cf and have a default score of 0.001?
>
>
> I believe only the renaming has been done, the default score remains 10;
> so anyone overriding the default score (that would be, er, me) needs to
> update their local settings for the new name.

Yes the rename was the only thing done.  The default score should be
0.001 in KAM.cf then local overrides and meta rules could be used to
bump up the score as needed.

The only way to get complete/true DMARC support in SA is to install
OpenDMARC as a milter and then setup local rules to use the headers it
adds that are specific to the AuthservID value in the
/etc/opendmarc/opendmarc.conf.

We should add default rules to the SA ruleset that would utilize
OpenDMARC headers if they were present similar to how SPF checks can use
Received-SPF and Authentication-Results headers on internal headers.

Any perl people out there want to take a shot at a DMARC plugin that
would use Authentication-Results nternal headers?


Examples:

Authentication-Results: smtp.ena.net; dkim=none
Authentication-Results: smtp.ena.net; dmarc=pass (p=none dis=none)
header.from=dmarc.org
Authentication-Results: smtp.ena.net;
      dkim=pass (1024-bit key) header.d=dmarc.org header.i=@dmarc.org
Authentication-Results: smtp.ena.net; spf=none (mailfrom)
Authentication-Results: smtp.ena.net;
      dkim=pass (2048-bit key) header.d=dmarc.org header.i=@dmarc.org
Authentication-Results: smtp.ena.net; spf=pass (mailfrom)
smtp.mailfrom=ncas.us-cert.gov (client-ip=208.42.190.161;
helo=mailer190161.service.govdelivery.com;
envelope-from=[hidden email]; receiver=[hidden email])
Authentication-Results: smtp.ena.net; dmarc=pass (p=reject dis=none)
header.from=ncas.us-cert.gov
Authentication-Results: smtp.ena.net;
      dkim=pass (2048-bit key) header.d=ncas.us-cert.gov
header.i=@ncas.us-cert.gov

--
David Jones
12