Ask Slashdot: How Do You Handle SPF For Spam Filtering? 187
An anonymous reader writes "Our organization had had a decent SPF record of our own for a long time. Recently, we decided to try using SPF for filtering inbound mail. On the up side, a lot of bad mail was being caught. On the down side, it seems like there is always a 'very important' message being caught in the filter because the sender has failed to consider all mail sources in writing their record. At first, I tried to assist sending parties with correcting their records out of hope that it was isolated. This quickly started to consume far too much time. I'm learning that many have set up inaccurate but syntactically valid SPF records and forgotten about them, which is probably the worst outcome for SPF as a standard. Are you using SPF? How are you handling false positives caused by inaccurate SPF records?"
Spamassassin (Score:5, Interesting)
Spamassassin handles SPF, reasonably intelligently, that is, not trusting it completely, not giving it more weight than it deserves.
Hanging your spam fighting hat on any single hook is problematic. and SA uses a wealth of tools with constantly updating itself via
scripts. Its been largely trouble free, and we have it set up so that it will learn false positives and false negatives when users
move these to the corresponding folders.
I've been well served by Spamassassin [apache.org] for some time now, it runs quietly
on our mail server. SA does not block mail. It flags it. Our mail server will evaluate these flags and trash outright the most
egregious spam, but we have the limits set low enough such that we will allow the questionable things through.
We error on the side of caution, but we still dump a lot of mail right after SA flags it. (Our business can do that, your business
may not be able to do that.)
DMARC (Score:4, Interesting)
That's what DMARC is for. It let's companies specify exactly how to handle their SPF (and DKIM) rules based on how thoroughly they have covered their bases. The company I work for deals with a ton of phishing against our user base and implemented SPF, DKIM, and DMARC with great success.
Google has excellent documentation on the protocol.
Re:Use it for scoring, not blocking (Score:4, Interesting)
Less expected by whom exactly? I was part of the major discussions a decade ago surrounding SPF and there was a large contingent of damned good mail admins and SMTP experts who said the whole thing was flawed.
Re:Forget about them (Score:4, Interesting)
The way I do is is that I send an NDR on rejected mail caused by bad SPF records (with some anti-flood limits). So far, as far as I know, bounced emails always eventually reached us. What happens is the person gets a NDR mail ("We have rejected your email due to bad SPF records"). The person getting the bounce forwards it to their IT dept, where it's usually taken care of rather quickly. If it's a small business without a dedicated IT staff, I'm sometimes asked to explain how this works, but usually, such companies do not have SPF records at all, which means the mail won't bounce.
Re:Forget about them (Score:4, Interesting)
And meanwhile in the real world where nailing some important email because the sender was sending all his email through a local MTA because his ISP doesn't have an externally accessible MTA, your boss is right now handing you your walking papers.
The only sane way to use SPF is to drop a spam score of an email. Outright filtering on bad or missing SPF records is just a recipe for a large number of false positives.
You don't know how SPF works, right? - Because that would excuse your statement...
Basically, SPF is a fail-open system:
A) No SPF: Allow the mail (fail-open)
B) SPF present and the mail fails the check: Refuse the mail
C) SPF present and the mail passes the check: Allow the mail
Option B has an exception for 'soft-fail' if the SPF uses the ~all. It will allow all mail through but tags those that fail SPF.
There's absolutely no reason not to refuse mail if the SPF check fails.
Remember, an incorrect SPF will result in a lot of mail lost, and it won't take many minutes before this is noticed. The fastest way to fix it is not to contact every single communication peer and have them bypass their SPF check; it is of course to just fix the SPF record. A 10-second dialogue with a competent postmaster at just one of the failed recipients should yield the reason for the fail, like the missing mail source. All that's left is to fix the SPF by adding the relevant "a:xxxxx.xxx" or "ip4:xx.xx.xx.xx" to the record.
Re:don't reject based solely on SPF (Score:3, Interesting)
When I worked at a spam company (I got hired to write software to handle 20 million mails per day on a single server - not proud of it, but I needed the job), we had SPF and DKIM on everything. Except the corporate exchange server, at least for a while. But the spam mails, having both SPF and DKIM on those was very important.
Ever since, I've considered blocking mails with SPF or DKIM headers. Only spammers care enough to set those up correctly. And a few of the big mail companies (like GMail and Hotmail), but they're in it with the spammers anyway - at least Hotmail (our main focus) is - we got status updates every day about how close we were to getting blocked, and when to switch IP addresses around, though not directly from Microsoft, but from a company (SenderScore) claiming to be cooperating very closely with Hotmail on this - and even if they were lying about Microsoft cooperating, they still had the data.