Gmail, SPF, and Broken Email Forwarding? 300
alek writes "I recently stopped getting Email from a friend ... which turns out to be related to his use of SPF records and my forwarding to gmail. This 'lost Email problem' may get worse with
Google implementing Domain Keys." Alek is looking for a non-complicated solution to this non-trivial problem; read on below for more details.
"Background: Like many people, I have me@mydomain.com as my public facing Email address. When Email comes into my server, I forward it to me@gmail.com. But since my friend has published SPF (Sender Policy Framework) records that say only his server is allowed to send Emails for friend@frienddomain.com, gmail apparently rejects (silently buries actually!) the Email since it is forwarding through my server. Please note that this is exactly what SPF is designed to prevent — spammers from sending Emails with your address — but it breaks forwarding and has other problems.
What's *really* strange is that if I look at the raw sendmail logs on my server, the Email from friend@frienddomain.com comes in, and is forwarded to gmail ... with an "OK" as the response — i.e. the gmail MTA doesn't reject the message as it ideally should. However, the Email then disappears — it's not even in my gmail spam filter ... so there is no trace of it at all. If my friend sends directly to me@gmail.com, it shows up ... since his domain sends directly and the SPF test is passed. Note that on my gmail account, I associate me@mydomain.com with my me@gmail.com account ... so perhaps there should be a recipient test applied before SPF is tested on the sender ... although this arguably defeats the purpose of SPF.
The logical solution is to configure sendmail on my server to do Sender Rewriting — anyone have an easy FAQ to do this? But many people/domains aren't doing this ... and my Email forwarding to gmail is quite common, so I'm surprised that this issue hasn't gotten more attention. Is there another solution?"
silently dropping is not unexpected (Score:5, Interesting)
What's *really* strange is that if I look at the raw sendmail logs on my server, the Email from friend@frienddomain.com comes in, and is forwarded to gmail ... with an "OK" as the response -- i.e. the gmail MTA doesn't reject the message as it ideally should. However, the Email then disappears -- it's not even in my gmail spam filter ... so there is no trace of it at all.
While the RFCs specify that an MTA that is dropping should notify the sender in various ways, modern MTAs often violate these parts of the spec, pretending to accept and then dropping the mail and/or failing to send bounce notifications.
This is deliberate. Not sending bounce messages reduces the load on the servers and net (now that most mail traffic bounces). Pretending to accept mail which is actually dropped is a defense against guessing email addresses and probing filters to see what gets past them.
Re:Please adhere to RFC (Score:5, Interesting)
^---- what jeffmeden said. (Score:4, Interesting)
Another satisfied google hosted apps customer chiming in. I have a reseller webhosting account that I keep about 10-15 domains on for myself/friends/family which does acceptable e-mail, but I advise everyone to just shove their e-mail over to gmail/a instead.
You get your own hosted mail/webmail service with (currently) 7gb of storage per/account, no preset account limit, POP and IMAP, as well as great spam-filtering.
All free.
And for $50/acct/year you can have 25gb/acct storage, API access to customize it for single-signon and/or gateways, a full Postini implementation, and 99.9% uptime guarantee.
Hate to sound like a shill, but it's a fantastic service and I don't mind pimping it.
Re:I knew .. (Score:4, Interesting)
Short answer is, no. Google's large amount of incoming email, their patented algorithms, and the huge data mine they're sitting on give them a unique ability to provide very through and high-quality spam filtering.
Of course, that isn't to say that one can't do a half decent job with spamassassin, it just won't be as good as Google's filter.
Re:There is an easy way to do e-mail forwarding... (Score:3, Interesting)
How do you deal with the problem of being blacklisted as a spammer if you end up forwarding lots of spam mail off of your domain? Remember, SPF itself doesn't address the problem of spam, so the fact that you're checking SPF doesn't matter a lot in this regard.
Re:I knew .. (Score:4, Interesting)
I have a gmail account, I get a handful of spam a week slipping through. I don't ever advertise my gmail account, however it's a common enough username with no numbers so dictionary attacks would hit it.
I have a private email server, with clamav running spamassassin and postfix tuned to prevent spam (Simple settings really), I get even less spam than my gmail. This address has been published for years on multiple websites, I use for just about everything, in cleartext on websites that are spidered.
In my experience you can do just as well or better than gmail without any headaches and a simple setup. Expect a few hours initial setup, and maybe an hour every 6 months to check if you're missing something the auto-updates can't update. It's been like this for a few years so far.
Re:silently dropping is not unexpected (Score:3, Interesting)
People violate the RFC because spammers spoof the sender as the people they are spamming, so the bounce goes back to that person and they get the spam. The RFC does not account for this, so fuck it.
Only if the mail admin is incompetent. This comes up every time there's a story about something in mail that's been screwed by spammers.
Receiving mail server should not be sending ANYTHING to the sender's mail address, faked or not. The Receiving server's responsibility is to generate a 5xx error on a permanent error and send that back to the SENDING MTA. The Sending MTA has the responsibility to generate the appropriate mailer-daemon message
Re:Pull instead of push? (Score:3, Interesting)
... IMAP, which google now also supports.
Gmail claims to support IMAP, but if you try really using it, its awful.
Eg deleting an email from my mail client inbox only removes it from the Inbox label, it still stays in AllMail. And deleting from AllMail is impossible, the email reappears in thunderbird in a minute or two. Deleting attachments doesnt work. Etc.
I understand they want to keep my data as long as possible and also that they want to make IMAP work with their Labels, but I don't care I just want an IMAP compliant email account...
Re:Easy -- sign up for Google Apps for your Domain (Score:1, Interesting)
Funny thing is on the "Learn more" section of Google Apps, when talking about the Gmail capabilities "Google" yes the ever mighty Google uses "jsmith@yourdomain.com" as an example in its wording.
How dare they ... we need to get the example.com police on their ass!