How Do You Store and Reconcile Email Archives?

heyitsjustme wants to know how you deal with old email. "I delete most of what I get but keep the stuff from friends and relations as an archive. Unfortunately I have these email archives from the late 80's through today in the form of macintosh, linux and windows mailboxes including AOL 1.0 mailboxes. What does everyone use to archive email across multiple platforms and non-standard mailbox formats? Is there an easy solution out there? Does anyone archive IM?"

Disk space is cheap. Why bother deleting?

[heypete]

Save it all. With the exception of some mail archives lost to catastrophic disk failures (I keep archives for my own convenience, not for any official purposes, so I don't back them up), I keep all my email.

Thunderbird is able to import all my old mail archives (from years and years of Eudora) and search it effectively. If I were inclined to export all my archives from my Mac to my Windows machine, I could use Google Desktop Search to really search through it all.

I'm afraid...

[joNDoty]

the best way to consolidate various types of emails may be to email them to a common source. Then archive from there.

PDF

[DisasterDoctor]

I delete almost everything, and only save a few very important or personal emails. For those I do keep, I print to PDF, and archive by date and person/subject. It works exceptionally well for me. It is all electronic, takes very little disk space, and keeps the clutter to a minimum, and eliminates most of the cross platform nightmares.

One Word

[Zone-MR]

One word: IMAP. If you can read your email using any decent email client, it should support moving it to an IMAP server. If you are using web-based email or some crappy client which can't export emails to a standard/raw format, you'll have to write a script to convert the messages.

It's simple: plain text

[Faust7]

Ever since I first got acquainted with e-mail on my Apple IIe in the '80s, I've used e-mail programs that offer plain-text storage as at least an option. It's one of the most universal formats in existence, and can be read one way or another on computers both decades old and brand new. I encountered some weird proprietary clients in the '90s that still stored e-mail in this format, because from a corporate perspective, this stuff was still in its infancy, plus HTML hadn't yet mucked everything up. To this day I still store in plain text from Eudora 6.2.

I burn it to CD-Rs that I know won't get moved around or scratched. They stand a good chance of lasting the rest of my life.

of course

[Anonymous Coward]

Of course people archive IM. I archive everything, but in particular conversations with customers (I do contract programming.)

See if one of the desktop search products, such as Beagle or Google's will index will index your archives. That might be all that you need.

If not, the first step is to convert everything to a real format. Eudora and Thunderbird can read in some of the non-standard mail formats and convert them to unix mbox.

How I archive my mail

[gpscc]

I only use e-mail clients that store mail in ascii with standard headers. This means no Outlook mail. I still use the Netscape e-mail client to view and organize my mail. Also I have various perl scripts that can access the e-mail archive. I have 22 years of e-mail, archived on my PC. It gets backed up with the nightly backup onto a swapable firewire drive. I swap the backup every morning and have one of the drives with me.

Convert to MBOX format

[Jason Earl]

Almost every email client around can import and export mbox formats. Getting your email in a format that is going to be readable in 20 years is the first step, otherwise why bother?

Worse comes to worst mbox is readable as plain text.

Re:One word

[Padrino121]

Gmail?

I don't know about you but I generate about 6GB of email archives per year. Besides that having my email potentially available for searching doesn't sit well with me. I'm not sure where it stands now but there were a lot of potential privacy issues with Gmail.

No I don't receive hords of email, just a lot of engineering related with source code,research, white papers attached. If you do anything business related it's important to keep all of the original emails received so there is an electronic paper trail.

Re:One Word

[pasamio]

IMAP isn't really a word, its an anocrym. But I agree, IMAP is the way I use, it helps for the relevant email and on my network I use both Linux and Windows (with a dedicated Linux box). I have Evolution set up to continually check and sort my email into IMAP folders and I generally read them off my linux box. If I need to click on links, I generally open up any Windows email client (from Thunderbird to Outlook) and it'll connect to IMAP and my emails will all appear (nicely sorted too!). If I need webmail, I have squirrelmail (which I use) to access my IMAP system remotely using any web browser and I can get at my hotmail email (from the old days, but my accounts are still active) using freepops or some other Web Email to POP3 gateway. Everything (but gmail, my mailing list archive), is in my IMAP server - I just backup one area.

Spotlight and Tiger

[goombah99]

Combine this with spotlight/tiger in mac os. Spotlight indexes PDF content. print it to pdf and it will be searchable. Assuming you have a Mac that is.

Delete it

[exp(pi*sqrt(163))]

That way it won't be subject to a sub poena. You'll regret it one day if you don't. Do you realize how much incriminating stuff you have in there?

Re:Disk space is cheap. Why bother deleting?

[Libraryman]

Why delete?

Because if you delete early and often, you've committed no crime. If you wait to delete it until someone (feds, cops, *IAA, UN-black-helicopter troopers, whoever) demands you turn it over to them, you're screwed.

After all, you break laws too (everybody does, they are written that way). You just haven't been caught yet. (I know this because if you had, you wouldn't have all you email archived!)

Re:Delete it

[man_ls]

Generally, most people don't have to fear subpoenas unless they're doing something illegal or nefarious in some way.

Good point, though.

Re:Convert to MBOX format

[snaptography]

This is always quite an interesting discussion, especially because MS Entourage can not even import from a MS Outlook database...doesn't seem to make sense. Although I would say that MBOX formats are probably your best bet...unless you want to write your own exporting,parsing,sorting system.

Re:Disk space is cheap. Why bother deleting?

[aussie_a]

On the other hand, laws are made retroactively, there could be regime change, or other unpleasentness.

For the majority of slashdotters this wouldn't be a problem, as I'm pretty sure American laws can't be retroactive. If I ate a chicken today and e-mailed someone saying I ate a chicken, and tomorrow it became illegal to eat chicken, I can proclaim to the world "I ate a chicken on the 12th of March" and I won't be able to be charged with any crimes.

However given your choice of words (regime change) I figure your talking about other countries, I can't comment on their legal systems. IANAL

I run my own personal IMAP server

[EMR]

I have a machine that runs a dedicated IMAP server with one account on it (mine) which has my 2GB+ of e-mail since 1996. (minus the spam of course). That way I can easily switch between different clients and not have to worry about converting my e-mail all the time.

PDFs more durable than mail, perhaps

[goombah99]

One assumes, perhaps wrongly, that pdfs are a more durable format than mail. This of course is what the entire "ask slashdot" question was about. How do you deal with past mail in different mail programs. If you keep it in pdf format then it probably will be readable regardless of the mail program that generated it. However then the problem is wading through 10,000 old e-mail pdfs. Spotlight solves this. Now that spotlight exists one assumes no operating systme in the future will ever be without something like spotlight.

Re:Disk space is cheap. Why bother deleting?

[operagost]

"No Bill of Attainder or ex post facto Law shall be passed." - U.S. Constitution, Article I, Section 9.

Re:Delete it

[pohl]

That's a rather optimistic viewpoint. It's entirely possible for someone to do nothing illegal and still find themselves at the blunt end of a civil suit.

Re:Disk space is cheap. Why bother deleting?

[zcat_NZ]

So Steamboat Willie is NOT actually protected by copyright, since at the time it was created copyright protection only lasted fourteen years!! The constitution says so!!

Re:Disk space is cheap. Why bother deleting?

[Tango42]

Wow... you really are american, aren't you? Your regime can change just as much as anyone elses... stop being so arrogant and open your eyes.

Re:IMAP all the way - agree 100%

[cheros]

BTW, it's also the only way to reconcile an enforced corporate suffering of /barf/ Outlook and more sane programs like Evolution (slooow) and kmail (not quite as pretty) or pine (good for SSH on slow dialups ;-).

With an IMAP backend you can try it all without tying yourself into one format - that's Open Standards for you!