CAN-SPAM One Year Later?

BigPoppaT asks: "Computerworld has an article reviewing the effectiveness of CAN-SPAM one year after it passed. In the article several anti-spam companies cite spam as a huge (and increasing) percentage of the total e-mail load. Most state that it is more than 50%, and some are saying as much as 75%. (This matches what I see in other articles on the subject.) Are these figures reasonable? I do not work for an ISP or maintain a mail server, but speaking as an end-user, I do not have anywhere near this much spam - more like 5 to 10 items a week (out of a few hundred messages). This is in my personal email - I do not recall ever receiving any spam in my work inbox. If the numbers above are reasonable, I wonder why I get so little spam? I am on a number of mailing lists, and have purchased things online, so it is not as if I have gone too far out of the way to hide my email address. I am not complaining, mind you, I just think it would be useful for the Slashdot readers who deal with this in an administrative capacity to explain it to the rest of us. Are the spam numbers being inflated by these anti-spam groups as a marketing tool? (This is not a rhetorical question - I really am not in a position to evaluate this, so those who know, please fill the rest of us in.)"

Users and their Spam

No. "Users" like their free crap. So they are willing to give out their e-mail address because it seems inocent enough. Then 2-3 months later they get their "free" spam and still haven't learned their lessons.

75 % accurate

I'm a faculty member at a large university, and about 75% of my email is spam. (This is based on the number of emails in my spam folder versus the number of emails in my inbox.) My email is on multiple web pages, on every syllabus I hand out, and in various directories.

By playing around with permutations of my email address, I find that a large chunk comes from infected colleagues' and students' computers. Relatively little comes from web crawlers. I also get a burst at around 8:00-8:15 when the staff members turn their machines on, and another burst a little later as faculty drift in. During the holidays, the rate goes way down.

Admins and generic addresses get it worst

Have you ever registered a domain? Nearly all the spam I get is to an address I only use for registering domains. I'm careful with my primary addresses, and receive nearly nothing on them.

A lot of spam that hits the system you'll never see as well. A big chunk of spam lists have bad or nonexistent addresses in them. There's usually some poor schmuck (here, that's me) that has to check and see if an Important Business Contact just can't type, or if all those emails to [e-mail], [e-mail], etc. are aimed at insecure men.

Other popular targets for spam are sales@, info@, support@....etc. so unless you're responsible for one of those, that's more spam you won't see.

Lucky bastard.

Spam levels vary widely

A year ago, I was in the same boat as the poster, with about 5-10 spams a week. Now, I'm getting closer to that many a day. It's annoying, but not unmanageable. For my part, I'm grateful that my spam load is much lower than some people have reported. The key benefit (besides less spam, of course :) is that all the anti-spam tools that have been developed to handle more spam easily take care of the compartatively little spam amount I get. In any case, I don't doubt that the huge numbers given for spam loads are at least close to accurate (unless those numbers come from AOL, which classifies way too much non-spam mail as spam).

However, I do wish the anti-spam leaders would finally start encouraging people to PGP sign their emails. While perhaps not perfect, it has all the benefits of systems like hashcash and allows for much easier verification of senders.

But what do I know -- I'm not an anti-spam leader. And I run my own mail server, so in their eyes, I *am* a spammer (just ask the more radical of them).

Accurate figures

Accurate figures are difficult to come by. But some of us do get those kinds of volumes of spam. One of my mailboxes averages almost 10 an hour. A few others approach that rate, I'm not really sure as I've got several layers of spam filtering in place now. Other accounts that have not been as well exposed online get much less spam.

You may have successfully protected your email address and have ordered from businesses with some degree of integrity. You may also have a spam filter in place somewhere.

Maybe not inflated, but certainly skewed

Let's face it, if spam isn't a big problem for you, then why would you want to pay money to BrightMail or some other spam filtering service in the first place? I think it's a pretty reasonable assumption that a large percentage of spam filtering services' customers have a problem with spam that they feel unable to cope with themselves. By definition then, they will have an above average percentage of spam in their legitimate email.

some get it, other don't

I must be lucky, or have good filters :)

I rarely get spam, whereas my workmates get an average of 100 spams a week

Alot of spam..

I get approx 3000-4000 spam per day.
Training spam filters are taking some time.

anecdotes != data...

...but I get about 125 spams a day, and about 20 real e-mails. I'm pretty sure my e-mail address has been harvested from at least the following sources:

domain name registrations
online fora and blog comments
usenet

Yeah, I leave my real e-mail address in all of those places. I used to be more careful, but SpamBayes is so good, spam just isn't a problem for me.

Numbers are accurate.

I work in academia. My email ends up on things like conference abstracts and journal articles, not to mention the University's on-line directory.
I get, on average, 300 emails per day, Over 250 of which are spam. Spam-assassin catches maybe 90% of those.

You probably have filters at the ISP level

I have several email accounts. The ones that I have through my ISP get a few dozen spams a day, on the order of about 5% of my email. Just for grins, I set up a completely unfiltered account using my own hosting service, and attached it to PopFile (a free Bayesian filtering program for email) to see the relative spam load. It is roughly 98% spam, or roughly 500 spams/day. I use that address for some mailing list subscriptions, and it is posted on my hosting service site, which probably where it got harvested. Unfortunately, PopFile is still getting some false positives, so I still check the spam bucket for improperly marked messages. (I'm definitely looking for something that won't give me false positives; I would rather get a small amount of spam than lose any legitimate messages.)

From your description, I would guess that your ISP is nuking most of the spam before you see it.

My ratio

One of the many duties I hold at my company is managing the email flow.

We receive between 60k-80k messages a day into our company and of that, about 90% is spam.

I have found the people who get most of the spam are those who have their addresses in other people's address books. I think that spammers get lists of emails gathered by viruses that collect address books.

Of course my boss is the worst because his email is set up as the billing email for all of our domains. The benefit of this is I have a great control subject for my home grown spam solution. I can tell when it is working well by how much spam gets through to him. He gets about 1000 spam messages a day.

Time to amend CAN-SPAM

The "expert" estimates on spam percentages do vary. But one thing seems pretty clear. CAN-SPAM hasn't perceptibly reduced the flow of junk email since it went into effect 1/1/2004. That's why I have suggested [betanews.com]that Congress seize a simple way to put some teeth into the law. Give U.S. citizens a right to private action. Why save the privilege of suing spammers just for ISPs, attorneys general, and the FTC?

Yup, it's that bad

I'm the mail admin at a company with a little more than 500 active mail accounts. We get about 110,000 Internet messages per week, and about 80% of those are spam. We're using SpamAssassin to detect it, and running a script against syslog to get those numbers.

Our SpamAssassin server correctly detects over 99% of the spam, and rejects about 92% of it outright at our Internet gateway. The 8% least-spammy-looking-spam is tagged and allowed through to allow for false positives, though none have yet been reported.

Public Email Addresses

I am the editor of a mid-size magazine (hard copy, not web). By necessity, my email address and those of the various department editors are published in the magazine and on our website so readers can contact us. Obviously, this is one of the worst possible scenarios, but necessary to address the lowest common demoninator among readership.

Due to this, I and the department editors that work for me (as well as the advertising and circulation departments) receive hundreds of spam messages daily.

I eliminate most of mine at the server level by filtering all email from non-U.S. servers based on IP (APNIC, LACNIC, and RIPE registry). The remainder get diverted to a spam folder by SpamBully, and are then reported to the FTC and to the originating ISP via SpamCop (not because I think it does any good, but because it makes me feel better).

Bottom line: about 80-plus percent of email is spam (except on deadline day).

There are currently...

... 2795 spams in my GMail, to which I redirect three or four other addresses. Last delete was on Dec 1st (logically).

So I get roughly 100 spams per day, of which gmail will let one, maybe two through every fifth day or so. pretty good. I now use my gmail account pretty much exclusively.

Thinking back, my spam volumes appear to have gone UP since CAN-SPAM went into effect. As for my work address, 3 a day or so, but we run a lot of spam filtering here, and I don't have access to the figures blocked. I've certainly not seen any marked effect of recent legislation on the amount of crap I get in my inbox.

75% Accurate

I'm the Network Administrator of a moderately-sized University, and we have a Barracuda spam appliance as our mail gateway. It tags about 75% +/- 3% of all incoming mail as spam, and has a very, very low false positive.

Yes, spam volume really is that bad.

perspective

My Department manages mail servers with ~400 mail accounts. We would say that the spam problem has increased (along with the virus-generated-email problem) because we see the reports generated by the mailserver and filter. Our users, however, seem to have forgotten that spam is a problem at all. They have forgotten that mere months ago they received dozens (or in some cases hundreds) of spam messages per day. Now they receive few or none, and when they do they send them to us as trouble tickets! At the same time, looking at the growing number of messages hitting our servers (we filter out ~90% as spam at this point) it's pretty clear that spam has gotten worse since CAN-SPAM rather than better.

So it really depends on who you ask. Users may not even realize that their ISP or employer is aggressively filtering. To them it just looks as if spam has evaporated.

I wonder if we're actually filtering TOO well. With bosses having only slightly pointier hair, it might be hard to justify the budget amount we plow into spam/virus filtering. I've been tempted to knock the filter down a few percent to admit more spam, just to keep people remembering it's a problem! (except then I'd get more too)

~

maybe you just don't see it

Many ISPs (and the webmail providers) have taken to just blocking the most egregious spam before it even gets to users. So your mailbox could be getting some spam that you don't even see. It still gets sent and clogs up the network though.

Some plots

I think you'll find the percentage depends a lot more on how much ham they receive than how much spam they receive. For example, if I got 100 spams/day, I'd be happy. Given my 300 hams/day, that would put me at 25%. But others, who get only 10 hams/day, would claim seeing 91% spam. Maybe counting raw spams per account would be a more useful metric.

To get a rough idea of trends, I've been plotting stats on a mailserver I manage. In general, we see spam and viruses are increasing, while ham is decreasing. Spam is about 67% of incoming mail. [uiuc.edu]

I also plot my personal spam stats [uiuc.edu] but obviously an individual account is hardly representative.

Some Figures

I recently activated a spam collector on my inbox. Since December 12th (time of activation), I have received about 1200 spam messages. Not counting mailing lists, this is at least %75 (if not more) of my total mail income. So yes, I would say those figures are accurate.

False positives are the new new problem

With the dramatic improvements in spam filtering software, getting rid of spam is no longer the technical problem it once was. In my experience as a consultant to email administrators and as a market research in the messaging industry, other, derivative problems are now taking over. And these problems are the result of filtering.

There are several problems that now plague email administrators: 1) satisfying the vast resource requirements of a modern email filtering system, 2) handling an increased flow of end-user complaints (yes, increased), and 3) dealing with false positives.

Everyone knows that spam is an enormous problem. The 75% number quoted in this article is conservative. Many organizations I work with receive in excess of 90% spam. Dealing with a problem of this magnitude is of course absolutely necessary -- and most large companies have by now installed a spam solution.

Unfortunately, implementing a large scale spam filtering solution requires rolling out sophisticated enterprise software and managing expensive, complicated, and high maintenance storage devices. This storage is mostly eaten up by the spam quarantine (or "junk mail folder") -- something that is necessary to deal with the possibility of false positives.

Even assuming that the system is correctly installed, maintaining it is an ongoing nightmare. And with a spam filter in place, end-users tend to assume that any spam that does get through is the result of a system failure that should be reported immediately as a trouble ticket -- adding to the email administrator's burden.

Finally, even though the latest spam filters are pretty good at what they do, if you're looking for a 95% spam rejection ratio, getting a false positive rate of less than 0.5% in the real world is a challenge. And while most false positives are things like newsletters that you don't normally care about, occasionally something critical is eliminated. When that happens, the email administrator can lose his job.

So what does he do? He tunes down the capture efficiency of the filter to drop the false positive rate. In a recent survey, Sophos PureMessage (one of the big iron enterprise anti spam solutions) had a capture rate of 90% and a false positive rate of 0.04% (Network World Spam Survey from December 2004 [nwfusion.com]). IMHO, 90% is a terrible capture rate that would result in an unacceptable flow of end-user complaints. Why did Sophos tune their product this way? Because false positives are the number one concern of email administrators.

Bar none. Number one.

False positives get you fired. Spam gets you a few more trouble tickets. You decide.

Spam filtering will always be necessary, but a complete rethink is required to take the problem resolution to the next level without the attendant drawbacks of filtering. The rethink involves end-user authentication (read: this is not the same thing as SenderID's domain authentication), something that can be implemented today using an aliasing system.

ASSP stats

I use ASSP for any of my customers who've implimented spam filters. It keeps global stats [sourceforge.net] for anyone who wants to report back to them. My spam hovers around 60%, more on holidays when there is less legitimate mail. Oddly, within the first few weeks after installing the filter, my spam dropped down from 80% to 70%. I guess the spammers realized they weren't getting through.

I used to get 50-100 spams per day; now almost 0.

I've had email addresses rendered useless by the sheer volume of spam. 50-100 spams per day, with maybe 10 legitimate emails hidden among the noise.

Thanks to MS-Outlook worms, even internal corporate email lists started receiving some really offensive porno-spam.

Today I get only a few spams per month, but to achieve this I ended up abandoning my old domain and setting up a system of aliases whereby I give a different email address to every person or organization that asks me for one. I now have several hundred entries in my /etc/aliases file. Whenever one of these aliases starts receiving spam, I delete the alias. Problem solved.

Yes, I even give aliases to my family members, since they'll inevitably divulge my address to e-card companies and so on.

Not an exaggeration

Admittedly, this is only my particular case. However...

In January 2004, I received roughly 1,020 spams. Last month (December 2004), I received over 3300 spams. And the number has not decreased in any month since March 2004.

Effective law, my a**.

Sources of spam

Culprits?

http://www.spamhaus.org/rokso/ [spamhaus.org]

We have unique WHOIS addresses and a lot of the spam comes from here but also from website scraping.

You can also see the source of SPAM migrate around the world, as new lists are produced and the old ones sold on. Our oldest unique addresses now receive almost all their SPAM from Asia in non English Languages.

75% seems a little low to me

I work for a corporation, our email scanner recieved 120,000 emails within the past 12 days. It only sent 10,000 that it determined wasn't spam to our email server.