Gmail, SPF, and Broken Email Forwarding? 300
alek writes "I recently stopped getting Email from a friend ... which turns out to be related to his use of SPF records and my forwarding to gmail. This 'lost Email problem' may get worse with
Google implementing Domain Keys." Alek is looking for a non-complicated solution to this non-trivial problem; read on below for more details.
"Background: Like many people, I have me@mydomain.com as my public facing Email address. When Email comes into my server, I forward it to me@gmail.com. But since my friend has published SPF (Sender Policy Framework) records that say only his server is allowed to send Emails for friend@frienddomain.com, gmail apparently rejects (silently buries actually!) the Email since it is forwarding through my server. Please note that this is exactly what SPF is designed to prevent — spammers from sending Emails with your address — but it breaks forwarding and has other problems.
What's *really* strange is that if I look at the raw sendmail logs on my server, the Email from friend@frienddomain.com comes in, and is forwarded to gmail ... with an "OK" as the response — i.e. the gmail MTA doesn't reject the message as it ideally should. However, the Email then disappears — it's not even in my gmail spam filter ... so there is no trace of it at all. If my friend sends directly to me@gmail.com, it shows up ... since his domain sends directly and the SPF test is passed. Note that on my gmail account, I associate me@mydomain.com with my me@gmail.com account ... so perhaps there should be a recipient test applied before SPF is tested on the sender ... although this arguably defeats the purpose of SPF.
The logical solution is to configure sendmail on my server to do Sender Rewriting — anyone have an easy FAQ to do this? But many people/domains aren't doing this ... and my Email forwarding to gmail is quite common, so I'm surprised that this issue hasn't gotten more attention. Is there another solution?"
Simple answer: stop forwarding (Score:5, Insightful)
Effective spam filtering for forwarded email is pretty much impossible, as you lose vital information in the forwarding. Either get rid of your forwarding address, or have it hosted at Google as well. Probably the largest single reduction in spam I've ever made was the week that I got rid of years-old forwarding addresses. If the forwarding address is more important, just get it hosted at Google directly, or tell people to stop using it!
Re:silently dropping is not unexpected (Score:4, Insightful)
It violates RFCs and causes problems like we are reading about now. It needs to stop.
Support SPF (Score:4, Insightful)
I put SPF on my domain not because I think that it'll solve the world's spam problem, but because it helps reduce the (large) number of bogus returns that come back to my domain (the more recipients that have SPF checking on, and realize that some sender in China isn't a legitimate source for emails from my domain, eats and discards the message rather than bouncing back some wasteful return spam to me).
SPF is great. It isn't a total solution, and there are negatives, but it certainly is better than the anyone is anyone free for all.
Re:Easy answer (Score:5, Insightful)
That's outstandingly unhelpful. How about attaching a link to a decent SRS implementation [srs-socketmap.info]? Or sending them to OpenSPF [openspf.org]?
Randomly throwing down on people legitimately asking for some technical help is a big problem in the OSS community. Whether or not /. is the appropriate place to ask this question is debatable, but since it made the front page and there is no helpful SRS faq on this site, might as well direct them somewhere.
Re:Please adhere to RFC (Score:3, Insightful)
Technically you're right. But I'm pretty sure that if some idiot chose "me@mydomain.com" as his personal email address, he's already used to getting mountains of spam.
Re:I knew .. (Score:5, Insightful)
At first I was wondering why they hell someone that had a working email server would shuttle it through Gmail, but then I read about using the spam filters, etc.
While that sounds good on the surface, is anyone out there not a little apprehensive about having all your email, particularly if you're a business, going through and being stored on their servers? I mean, someday Google will bend completely for govt. wanting to search all emails for 'terrorists' activities, and God knows who else will too.
I guess I'd want a bit more privacy on my emails, especially if they contained sensitive or proprietary information. I know...they're in plain text and could be intercepted if not encrypted, but, this is altogether different. It is stored on google's servers and there for easy data mining.
I'm getting ready to dig out my old email server post Katrina...can you not use procmail and spamassassin to filter spam as effectively as Gmail does?
It's just an example in a text message (Score:4, Insightful)
For God's sake. It's just text! RFC 2606 doesn't specify what you're allowed to write in a text message.
If you're actually going to do some testing then it might matter. What matters here is can the reader understand the question. I can. Can you?
Re:I knew .. (Score:3, Insightful)
1) stop using email altogether.
2) you need to get to a drug rehab center... cocaine is a hell of a drug
Re:silently dropping is not unexpected (Score:5, Insightful)
What they should do is reject the email immediately, in which case they don't have to send a bouce email but the mail is properly logged as being rejected. Ofcourse this does mean google will have to do all of their checks before accepting the message which is a bit harder to do but it is the only correct solution for the bounce problem.
Re:silently dropping is not unexpected (Score:4, Insightful)
Either way, you should never silently discard an email unless you are 110% sure it's spam. In all other cases it should either be dropped in the spam folder or be properly rejected. Anything else makes email totally unreliable. (And frankly, you shouldn't entrust your email to a company that thinks it ok to silently drop something addressed to you, but that another issue.)
Re:Please adhere to RFC (Score:2, Insightful)
So I guess you are expecting that /.'ers are going to start sending email to that domain to try it out? I
No, but a spambot will, you insensitive clod.
Re:silently dropping is not unexpected (Score:3, Insightful)
The rfc is broken, as it assumes no one would lie in their 'MAIL FROM' field.
Will you fix it for us?
--jeffk++
Re:silently dropping is not unexpected (Score:5, Insightful)
It violates RFCs
I'm giving up mods to post this, but it really needs to be said.
People need to stop blaming things on services who pragmatically choose to violate selected aspects of decades-old standards that don't address today's realities. The problem with modern e-mail is that the standard is hopelessly out of touch with modern demands. There should long ago have been a consistent standard that covered things like sender authentication, encryption and signing, formatted messages ("HTML e-mails"), smart handling of errors without treating them all as e-mails in their own right, and numerous other fundamentally broken parts of the original e-mail specs. But there isn't, so people try to do reasonable things and stay as true to the standard as they can without being dogmatic about it when it's obviously a stupid thing to do.
So no, I don't think silent dropping needs to stop under all circumstances. E-mail has never had useful reliability of delivery (another thing a replacement standard should deal with) so you can't count on it anyway. On the other hand, I'm sick and tired of getting a deluge of hundreds of unwanted e-mails in ten minutes because someone sent out a mail with webmaster@my.domain as the sender, and loads of people who were confident enough that the message was spam to block it still sent back a bounce message to an address that is 99.99% likely to have been faked as well in that case. I'm sorry, but that's just antisocial behaviour, and responsible sysadmins should take steps to avoid it: if you're confident enough to refuse delivery, why aren't you confident enough not to reverse-spam the innocent bystander? If you're running a sensible service where a user can whitelist specific senders or switch off spam filtering altogether for specific receiving addresses if they want to guarantee receiving everything, and they've opted in to your spam filtering, this shouldn't be a problem.
Re:I knew .. (Score:3, Insightful)
I agree with cayenne8, but not quite for the same reason. I've been using my GMail account for a while now and loving it. There's nothing incriminating in the email, per se, but there probably would be enough to do a bang-up job of identity theft. More than the government, I'm worried about Google misplacing an unencrypted backup tape with my account on it.
The reasons I still use them are that I think the quality and utility outweigh the risk, and because my much-smaller web hosting company is more likely to do something bird-brained than Google is.
Re:I knew .. (Score:3, Insightful)
The reasons I still use them are that I think the quality and utility outweigh the risk, and because my much-smaller web hosting company is more likely to do something bird-brained than Google is.
That's actually a foolish remark. Use google to search for things like "gmail outage" or "gmail issue". My favorite is "gmail security issue" with over 100k results.
I've heard stories personally about people logging into gmail and ending up in someone else's Inbox. Yes, that's right, full access to someone else's email. [slashdot.org] Or how about another goodie: mass deletes of random emails. [oreillynet.com]
I don't understand why people have the idea that Google is better then competent system administrators - it's just plain foolish and naive.
Regards,
Re:Please adhere to RFC (Score:2, Insightful)
Indeed, as the former abuse/e-mail guy for Dotster, who own mydomain.com, I can't even express how annoying it is to see it used as an example and the deluge of bogus e-mail we had to reject as a result.