Are You Using SPF Records? 263
gravyface writes "I've been setting up proper Sender Policy Framework records for all my clients for past year or so, hoping to either maintain or improve their 'reputation' in the email universe. However, there's a lot of IT admins I speak with who either haven't heard of SPF records or haven't bothered setting them up. How many of you are using SPF records for your mail domains? Does it help? How many anti-spam vendors out there use SPF records as part of their 'scorecard'?"
Re:No, and I won't (Score:5, Informative)
Unless something has changed in the last few years, DKID doesn't verify that the server is allowed to send email for that particular domain, just that the email itself wasn't tampered with which has its own issues. For instance, while it does verify that the message hasn't been tampered with, it does not help people that are sending legitimate cold emails to a server of say the sales rep for the company.
My point here being that any technology has issues when one tries to use it in a way for which it isn't designed, but saying that because it can be used improperly that it's therefore dangerous is an argument which lacks merit.
Re:No, and I won't (Score:2, Informative)
Of course, people to whom email is just another way to make a quick buck may have different ideas.
Re:yes (Score:3, Informative)
If you're using a lack of SPF records as a determinant in whether or not a message is spam, then I can guarantee you that you're losing mail to false positives. Maybe that isn't a big deal to you, but for the organization I work for, it would be absolutely nuts. The chief reason I have SPF records for my domains is so that the big boys like hotmail.com and GMail don't reject my emails. I use greylisting as my chief anti-spam weapon, and it's far more reliable and far more effective than SPF.
Anti-spam vendor's perspective (Score:4, Informative)
I work for a major anti-spam vendor.
Yes, SPF records are part of the mix at many anti-spam vendors.
However, they aren't part of reputation. Reputation, to describe it simply and without giving away any secrets, is determined by the kind of mail a host or network emits. Whether it has SPF records and/or DKIM-signs its mail does not affect reputation; if you emit junk, your reputation will be junk.
SPF and (moreso) DKIM have value in assessing whether a given mail is a forgery or not (think phishing and related scams). They are not weighted overly much, since people do foolish things like put their work email address into their webmail account all the time, and it causes FPs, for some value of false positive. That is, it's not an FP per se, but the mail is technically legit, so dropping it on the floor isn't the desired action.
In short, don't expect having SPF and DKIM to improve your deliverability much, if at all. That's not where the value-add is. The value-add is helping to separate the sheep from the goats among mail that purports to be from your domain. If you want recipients to be able to (theoretically, since most of them don't/won't check) have greater confidence that a mail that claims to have come from your organization really did so, then yes, implement both SPF and DKIM.
If you're an organization whose customers might be phishing targets, definitely do both. Orgs I've seen targeted for phishing include financial institutions of any size (even a single branch!), various government agencies, educational institutions (not just universities, either), BBB, auto clubs, World of Warcraft accounts, Vonage, Craig's List, all the free webmail providers. If it has a login, and anything a phisher could find to be of value (for practically any value of "value"), there will be phishing attempts.
If your company is one of those - or even if it's not, really - I recommend both SPF and DKIM.
Better for Sent Items then Received (Score:5, Informative)
I use them, and what I've found is that they have a very marginal effect (if any) on spam catch rates on your inbound mail. However, they do have a great side benefit. They significantly reduce backscatter, keep yourself off of blacklists, and provide some control of you, your employer, or your client's identity on the web. SPF records provide a mechanism to limit who can spoof as you (as long as recipient servers adhere to them). If you have a risk to yourself or interested parties that someone might spoof your domain (banks!), then SPF provides a means to insure the chain of custody (to an extent).
I do think overall SPF has helped to prevent forged domain letters, but those are less and less common (for those that publish spf). The spammers now either rely on forged domains without DKIM or SPF (why not use both!!) or they send from their own controlled botnet domains and publish legit SPF for themselves as well.
dkim; convincing individual mail providers (Score:3, Informative)
DKIM (formerly known as Domain Keys) is more sophisticated and worth looking into in addition to SPF. I'm using an implementation called DKIMproxy, which runs as a daemon and is specifically designed to work with postfix. I've been fairly happy with it. What's helpful about it is that if I get mail from someone who implements DKIM, I can be sure that it's really from them, and likewise if I get joe-jobbed, I can convince the recipient that the spam wasn't actually from me. The biggest and best known users of DKIM are gmail and yahoo, but I'm seeing it used elsewhere as well. For example, I recently got spam from lulu.com, and the good news was that it was DKIM-signed, so I could be sure it wasn't a joe job.
I understand what you mean about establishing a good reputation in terms of the email you send. Actually many of the big email providers have a policy of blacklisting all domains by default these days, and waiting for the domain operators to contact them and ask to be allowed to send mail to them. Both AOL and yahoo seem to do this. With yahoo, you can fill out a form to convince them you're not evil, and if the info on the form satisfies them, they stop blacklisting you. One of their criteria is that they're more likely to approve you if you implement DKIM. If you tell them you're using DKIM, then they won't accept mail from your domain that isn't DKIM-signed; this is to your advantage, because then their users won't be clicking on the spam button on mail that claims to be from you but isn't.
Re:no (Score:3, Informative)
Re:yes (Score:2, Informative)
Folks who do a lot of mail find out the hard way that without SPF records, there are plenty of places that bounce them. I've had them on my domains for years.
For my old network, where we got a huge amount of spam, we used both graylisting and our own custom blacklist. I didn't trust the blacklist providers, so we did rolling blacklists based on the amount of detected spam (with mailscanner and friends), which worked with the firewall. It set it's own firewall rules, so all traffic was dropped from that IP. On the first offense from an IP, it was blocked for a day. If there were multiple spams detected from the same /24, the whole /24 got blocked. If they were repeat offenders, the durations increased. It protected the mail server from about 90% of the spam, and didn't generate a single complaint. There was a tremendous amount of inbound mail also that was legitimate, so we would have had complaints after their automatic block was lifted.
It also used some honeypotting. Messages to old dormant accounts that only received spam automatically had the sender blocked. It's not like the accounts were a few days unused, we're talking about more than 5 years, and they were some of the highest traffic accounts on the server.
An offense was carefully defined, so as not to block legitimate traffic. It worked amazingly well. For it to work though, you have to have a high-load environment, that the spammers are already hitting hard. We would receive upwards of 100k emails/day, which was then reduced to 10k and most were legitimate.
Re:Use DomainKeys.. (Score:3, Informative)
I don't have DomainKeys set up, and I've never had any difficulty getting mail to users of any of those services.
Does your mail server deliver tens of thousands of messages per hour to those services? If you're talking about the occasional email, you're probably not hitting the threshold at which your delivery will be affected.
Yes! Prevents forged Froms (Score:3, Informative)
SPF is great. It's one of the technical means of making sure that the IP address that is trying to send you a message is authorized to use the sender that it claims to be from. That means you can automatically reject spam that claims to be from any of the big mailers.
One common problem right now, is misconfigured mail servers. An e-mail admin configures the SPF entry in DNS, and then forgets about it. Then they change their IP address, or they outsource their e-mail to a third party, and suddenly, SPF is saying that all of their legit mail is not legit. The other problem is when a company has (for example) an order fulfillment system that generates its own e-mails, instead of routing them through the proper mail server. If that system isn't identified in the SPF entry, those messages can be rejected.
Another "problem" is when organizations send messages on behalf of other individuals or organizations (like the legit message that avon.com tried to send me this morning that was being generated by filltek.com, but without the permission of avon.com's SPF entry). I put "problem" in quotes, because really, third party messaging services should not forge the From line of the message.
On the other hand, it's great, in that it blocks all those stupid e-cards, because they claim to be from your.friend@gmail.com, when really they're being sent by stupid-e-card.com.
The biggest problem is dealing with "forwarding" services, like your @acm.org e-mail address. On my server, I have to keep a list of domains that "bypass" SPF checks, because any message sent to a forwarded address is going to arrive at your mail server from the forwarded (i.e. mail.acm.org), but it's going to have the header information associated with the original message. OpenSPF.org talks about some ways to deal with this, but I haven't look at it in a while.
Since SPF is still not universally accepted, it has a "soft fail" option that you can use for testing, until you're sure that it works the way you want it to. It's not the be-all-and-end-all, but it is a useful piece of the puzzle.
Re:Yes (Score:2, Informative)
But I can tell that Hotmail still ignores SPF since almost all the backscatter that still comes through is from Hotmail. They should know better.
I believe you, but really? Hotmail was THE reason I've implemented SPF for a few domains connected to sites that send alert emails to users. Nothing - from email confirmations to status update type stuff - was getting through to Hotmail accounts until I set up SPF. Some kind of Left Hand / Right Hand mess going on over there?
Re:Nope (Score:3, Informative)
A quick check of mail volume:
151,000 messages checked in the log files that I looked at
58,800 (39%) did not have SPF records ('none')
So 61% of our inbound mail has SPF records that we can test. That is a pretty decent rate of adoption. (Note that we only test SPF for emails that make it past our first set of filters.)
Of the 92,200 messages with SPF records
51,650 (56%) passed their SPF check ('pass')
30,700 (33%) failed (-all)
5,150 (5.6%) resulted in a softfail (~all)
3,100 (3.4%) resulted in 'neutral' (?all)
1,500 (1.6%) permerror (DNS contains an invalid SPF record)
Killing off 33% of those messages with SPF records due to being obvious forgeries is still worth quite a bit. That lets us drop 20% of inbound spam at SMTP time without getting into the really heavy lifting of spam scoring or content analysis.
(We don't block on softfail, neutral or permerror conditions.)
SPF is good stuff. (Score:3, Informative)
SPF allows me to publish information about what systems will legitimately send e-mail using that domain. It also allows me to act on that information published by other third parties.
What this means is that I have to deal with dramatically less backscatter spam. Since implementing SPF, I have not woken up to find 100,000 messages in my box that were bounces or outraged replies to spam sent by someone else. Back in 1995 that exact issue happened to me, and to a lesser degree it happened regularly until SPF.
There are, of course, some difficulties with SPF, but despite those I have chosen to use and advocate SPF.
You do have to deal with legitimate third-parties sending mail from your domain. We use an outsourced accounting package and have had to include their servers in our SPF records. No big deal.
As a recipient, if you have one account forwarding to another, and the destination account implements SPF, then you either need to white-list the forwarding machine(s), or you need to implement SRS there.
DKIM and it's variants is, IMHO, useless because it only allows you to prove that e-mail came from an authorized sender for a domain, it does *NOT* allow you to tell if e-mail came from an UNAUTHORIZED system for a domain. You cannot use DKIM to tell if a sender address is forging the domain.
So DKIM is *NOT* a "better SPF". They *ARE* compatible though. If you get a message claiming to be from a specific domain which fails the SPF check, you probably still want to allow it if it passes DKIM. I don't know of any mail programs that do that though. The unfortunate thing about this is that SPF-only can be implemented entirely at SMTP time (RECV FROM) where SPF+DKIM would have to be implemented after receiving the message (after DATA).
Sean
Re:No, and I won't (Score:2, Informative)
rfc1123 removed the relay function whereby your mail server may list other server's names in the return path.
The replacement is that only your mailserver is listed in the MAIL FROM entry. The RFC never said anything about adding some other server's address to a MAIL FROM line, that was never allowed by the standard. Your mail server can only add its own address as it is known to the mail server it is sending to period (for a new mail from: line).
To list the address in MAIL FROM: means that you (the sending mail server) are claiming you can take responsibility for being able to return mail to the address you list.
That is: by listing an address in mail from, you PROMISE you can immediately return mail to the host you received it from, if there is an error.
There are traditionally only two ways a mail server can take that responsibility: either (1) the item you list in MAIL FROM is you, and you list a local user, then of course you can return the message to your local user, OR:
(2) You are relaying the item, and you received that address in MAIL FROM. You can live up to the promise by delivering the error message to the server that relayed you the message with that address in MAIL FROM, and that server can live up to their promise, and so on.
Well, without source routing (2) is actually a stretch.
The elimination of option (2) as a way to satisfy the promise you make means that you are left with option (1) only.
Re:No, and I won't (Score:3, Informative)
The domain name given in the EHLO command MUST be either a primary host name (a domain name that resolves to an address RR) or, if the host has no name, an address literal, as described in Section 4.1.3 and discussed further in the EHLO discussion of Section 4.1.4.
If your mail server talks to the outside world, it must introduce itself properly. There's no wiggle room there.
Re:No, and I won't (Score:3, Informative)
This is utter nonsense and a major misunderstanding of the standard on your side. Also RFC821 was deprecated by RFC2821 eight fricking years ago. It is telling that you apparently did not know that, because otherwise you won't argue with RFC821 and RFC1123 which updates RFC821 instead on looking at the later standard which incorporated the changes proposed by RFC1123. It is also important to understand that a RFC is what it names tell: A Request for Comments, not a binding standard. The pure existence of RFC1123 does not mean that everyone has to follow it ideas.
Having said that, and ignoring your misleading statements, the case is very simple. Regarding to RFC2821 - which actually is widely accepted as an standard - the MAIL FROM command specifies " The reverse-path consists of the sender mailbox." It is a no-brainer that a relay is a relay and not the sender of the message and thus does not change the "sender mailbox". So to comply with RFC2821 as relay can NOT change this value and HAS TO use the original address (which is by the way not a server address as you called it). While this is fully standard compliant, the existence of SPF and other spam filtering messages nowadays make it impossible to follow the standard in this, thus mechanism like SRS (sender rewriting scheme) were invented. But while the use of such hacks is widely accepted and understood as a necessity, it is in fact a violation of RFC2821 which mandates to keep the original value.
And for the issue of what a RFC means, you really should read RFC 2026 "The Internet Standards Process". As said, a RFC is not a standard, you might have confused RFC with the BCP (best current practice) and STD (standard) series of documents.
Re:No, and I won't (Score:2, Informative)
RFC1123 is a standard, STD 3 is currently RFC1123. RFC2821 is currently not a standard of any kind. Check the listing in the RFC Index [ietf.org] next time.