Ask Slashdot: Best Way To Archive and Access Ancient Emails? 282
An anonymous reader writes "I started using email in the early 90s and have lost most of that first decade due to ignorance, botched backups, and so on. But since about 2000, I've got most — if not all — of my email in some form or other. I run Linux, so this has mainly been in a mix of various programs: Kmail, Evolution, Thunderbird. The past 2-3 years are still on the IMAP servers. My problem is that I only rarely NEED to look back to email of 5 years ago. But sometimes it's nice. Or I just want to reminisce about something...or find an old attachment that I was sent. But I do not want to be clogging my current email client of choice with vast backups and even more, I don't know if it will even easily convert. The file structures are different, some are mbox, others maildir, etc., and I would ideally like a way to 1) store and archive these emails, 2) access them, and 3) search by Sender, Subject, Date, Attachments. Is there anything I can do or do I just have to keep legacy applications on hand for this? Should I keep trying to upgrade and pull old files into the new applications? Any help or suggestions about what YOU do would be great."
Just dump them (Score:5, Interesting)
Had the same need 20 years ago when migrating from VAX/VMS to Unix. The old emails were saved in a not quite readable format, but I figured I could recover them if necessary. In the end, never bothered. Yes, there are a few (actually, only two) that I'd like to resurrect now, but life moves on.
Use a database! (Score:5, Interesting)
You could export your emails to some parsable format, write an importer to extract the basics that you want to keep (from/to/subject/body,attachments/entire binary blob/etc) and then bulk insert that mess into on a mysql/sql server tucked away somewhere locally or "in the cloud" (EC2, Azure). Just another option as I'm sure you'll see here many here. At least with this route you are in full control of how you index, what you can search, encryption, performance, level of backups, etc. Maybe not the best way for some but I know if I had over 100000 emails that I wanted searchable very very quickly with advanced SQL like searching, this would be a cool way to do it (time permitting). Good luck! And to the pedantry to ensue...Yes. Good day.
Re:Stop being a hoarder (Score:5, Interesting)
It's kind of like photos, you know? Or letters, and such. People like to store those things, because they serve as a memory aid for what the mind no longer holds. It is also quite useful for history reconstruction/when you are old and have nothing else to do but a box full of photos/letters/etc.
Not to say that you are wrong on your point, except on the weird part. Unless you are okay with double standards, or you also consider anybody who keeps photos of parties/graduations/etc weird... Just saying.
Re:IMAP (Score:2, Interesting)
IMAP offers "server side" storage with "client side" viewing.
Besides the privacy implications, are you seriously suggesting that the OP
a) find or create an IMAP server,
b) force feed that server all his archived emails (presuming that there is some way to bulk import email into the IMAP server), and
c) change his current email setup so that, from now on, his email is sent to the mail server on which the IMAP server runs?
How is that any easier to manage than his current predicament? ISTM that your suggestion forces him to go into a big "migration" phase, and change his email provider/provisioning. Not the simple solution that I would suggest.
Re:Look to the past (Score:4, Interesting)
Re:Use a database! (Score:5, Interesting)
And you could make a doilie, and a hat, and a casserole, and wallpaper with the headers, and knit the .signatures into a fancy flying cape.
Just use IMAP and Maildir. Modern systems are fast enough to allow you to search the content directly, and not vulnerable to the database support wackiness this sort of "I can pre-organize it now and make my life better by wasting it pre-programming my queries" approach.
Re:maildir + mutt + maildir-utils (Score:2, Interesting)
To those saying keeping email forever is hoarding: not if it's done right.
That's like saying neck deep rooms of newspapers/magazines in a house isn't hording if you stack them neatly with little paths running through it.
Re:Maildir (Score:4, Interesting)
The main problem that I personally have is where I get emails from people that reference images and such from other servers. Most of the time it's commercial messages that I delete, but sometimes there's a newsletter that I want to save, and the images themselves turn out to be relatively important. Kind of annoys me to have to print them to PDF so that the formating gets preserved.
Re:Stop being a hoarder (Score:5, Interesting)
But why would I waste time manually finding and copying individual emails, when I can just let the backup script archive them all for virtually no cost?
Re:IMAP (Score:4, Interesting)
What's wrong with MBOX? I've been using it with gigabyte-sized folders for over a decade and nothing bad has ---@@@ From MAILER-DAEMON Fri Jul 8 12:08:34 2011
Seriously, though. Even when mbox gets trashed due to disk corruption, it is still every bit as useable as a trashed maildir: Messages get lost, not whole folders...and even then, that's what backups are for (right?).
Or at least, that was my experience over the past many moons. As a static archive (Sent-mail-2012), I can't think of a single thing wrong with mbox. And it's easier to cat to tape than maildir.
Comment removed (Score:4, Interesting)
Re:IMAP (Score:4, Interesting)
[......] are you seriously suggesting that the OP
a) find or create an IMAP server,
Ridiculously simple. They're already running Linux, they just have to install dovecot and they've got a fully functional IMAP server (no configuration required) - which has access to all their local mail boxes.
b) force feed that server all his archived emails (presuming that there is some way to bulk import email into the IMAP server)
Ridiculously simple. Fire up Thunderbird, configure it to access your local IMAP server, select all, drag and drop.
c) change his current email setup so that, from now on, his email is sent to the mail server on which the IMAP server runs?
Why would they need to do that? Thunderbird (or other mail reader of choice) can access multiple accounts.
Re:IMAP (Score:4, Interesting)
There aren't any privacy implications. If you there were any, then you would have named one or mentioned an example. The situation prior to typing "sudo apt-get install dovecot" is that he had the data (so it's already subpoena-able or whatever you're trying to imply) and after that he'll also have the data. Nothing changes. Are you complaining that he's keeping the data rather than deleting it? I don't get your point at all.
Yes, because it's easy. This can be done in literally ten minutes. Maybe a little more if he doesn't already have good storage allocated for it (e.g. a Reiser formatted ~/Maildir, or whatever your own religion commands).
Yes, because it's trivial. It's highly likely that whatever he is using to read each of his different mail archives, can also talk IMAP, because everything talks IMAP. You say "force feeding" as though literally selecting and dragging in a GUI, or picking "copy" or "import" off some menu, is hard. It's not.
As for step c (changing how he receives email), I don't think that's being suggested but it may be a good idea. He can decide later, whether or not he wants his archive server to become his main/active server. That decision can wait and is not part of the scenario being discussed; it's an opportunity for the future.
Because then he'll have his archive stored in a system that is specialized for handling the problem, accessible and searchable by any client he wishes to use, possibly even the very same tool he uses for his day-to-day non-archive mail reading. Or he can pick some other IMAP client if it handles mass/archive use case better than the routine use case. Everything Just Works, all together. All of his complexity and exceptions disappear. And at virtually no cost; there's no downside to counter any of the advantages.
This is one of the easiest no-brainer Ask Slashdots, ever. There is one right objective best simple easy-and-fast-and-good(!) answer, and setting up an IMAP server is it. Probably because email storage is an old, very-solved problem.