Best Way To Store Digital Video For 20 Years? 805
An anonymous reader writes "My kid is now 1 year old and I already have 100G of digital video (stored on DVDs, DVD quality) and photos. How should I store it so that it's still readable 10 to 20 years from now? Will DVDs stil be around, and readable, 10 years from now? Should I plan for technology changes every 5 to 10 years (DVD->Blue-ray->whatever)? Is optical storage better, or should I try to use hard drives (making technology changes automatic)? And, if the answer is optical, how do you store optical disks so that they last?"
My method (Score:5, Interesting)
Movies: Taken on MiniDV, backed up to HDD.
The only worry I have is that the MiniDV's and HDD are in the same house although they are stored in separate locations. But every picture is backed up offsite.
HDDs (Score:2, Interesting)
Re:CDs are still readable (Score:5, Interesting)
Only professional CDs have that sort of shelf life, because they're physically stamped. The consumer grade ones use a type of photosensitive dye that DOES decompose in less than a decade.
How much could you store? (Score:5, Interesting)
At some point, I came to the realization that I had downloaded over 6 solid months worth of music. This doesn't include CD's, LP's, or 7 inch records, of which I probably have 1000 total. If I were able to put all that music on a big loop, and not repeat anything, I'm thinking it would last over 12 months. Some of these I'll probably never listen to. I'm thinking the same is true for the submitter's videos.
My parents have a big box of photographs from their childhoods, as well as those of their parents. There are some great photos in that box, and I could and have spent hours going through them. Each time I do, I make a mental note that one day I'll scan them and make them digital. Then I realize that we only drag out that box once or twice a year, and never do anything with the photos anyway, and resign to scan them once it gets even cheaper.
Without meaning any offence... (Score:1, Interesting)
Without meaning any offence (and I honestly mean that), what makes you think your kid, or anybody else, is going to be interested in so much video from when they were that age?
I appreciate the desire to record the life of your pride and joy, but aside from the personal impulse you have that this is important, what is the point?
My mom probably has a few hundred individual photographs of me from when I was a child, and although I haven't ever looked at them, I'm sure a time will come when I will. Nostalgia is like that. Still, if I was born in 2007 and this was twenty-six years from now, there is no way on earth that I would be reviewing terabytes of video. I wouldn't have the time or interest.
I have sat down with relatives and watched their holidays videos and found it to be the most tedious experience of my life.
Photographs are great because they give you a glimpse at a moment in time, and the person (presumably, somebody you care about if you are looking at their photos) will tell you the associated story. Its interesting, its interpersonal and it is succinct. Videos are boring as hell because aside from what is on screen, there is no extra story told by your friend/loved one, or if there is, it is the same story you would get from a photograph, except you had to watch five minutes of a baby crawling in the kitchen instead of a snapshot of same.
I know you posted this to get advice on storage media, but for what its worth, here is some advice on a related issue. Stop recording so much video. Record a few, were the video enhances your story in a way a photo couldn't, but after that, take lots of snapshots and look forward to hours of story telling with your nearest and dearest.
Re:CDs are still readable (Score:3, Interesting)
I don't have a single CD that has succumbed to "CD rot". I've had some rendered unuseable by scratches or by being left in the heat/sun too long by accident, but other than that all my CDs, even from the late 80s and early 90s are completely fine. The data CDs I burned in the early-mid 90s are also still fine.
I'm not saying that it doesn't happen, but people make far more of a big deal about it than is really warranted.
That said, anything I wanted to make sure was still good I'd "refresh" every 5 years or so.
Where can I get these mythical disks? (Score:2, Interesting)
If anyone knows of a way to read my old CD-R's, even if it is [b]one[/b] time only, let me know.
Re:My parents wedding (Score:3, Interesting)
Qualilty digital tape will last 30 years. There's "100 year" digital optical media (if you believe it), but it's very expensive per-byte.
DARPA did a research project to create a storage medium that would last for centuries after a nuclear war, and be readible with very low-tech gear. They invented a metal punch-tape format - very cool.
Re:Without meaning any offence... (Score:5, Interesting)
I suspect that in a few decades, estate planning will include what to do with the family terabytes.
Re:How about print on paper? (Score:2, Interesting)
We're talking about archiving, not backup... (Score:2, Interesting)
HTH.
And tape is an abysmal archival medium.
How long have you worked in the industry?
Re:CDs are still readable (Score:3, Interesting)
CDs, perhaps. CDR's? No, they're not.
I paid a ton of money for one of the first 1x CDR drives for PCs back about 14-15 years ago. Recently I've been moving all the data I've built up over that time onto HDs for longer term storage.
What I've found is nearly all of the discs from back then I can read -- the $20 a pop gold discs.
Starting with the discs from the very late 90's, I'm getting about a 50% failure rate (on discs stored in climate controlled conditions away from light). With some brands (and not necessarily low end ones), I'm getting nearly 90% failure rates after just seven or eight years.
(And I consider failure to mean a clean disc, at least one file can't be copied anymore...)
Since he's not going to be pressing glass masters and casting pressed CDs, I'm not sure non-recordable media longevity matters one bit.
Re:I appreciate your enthusiasm (Score:2, Interesting)
This is a very good point. The bits you want to keep are the disgustingly cute/embarrassing pictures and videos. These will come in handy when the kid becomes older.
"Do what I say or I'll show that embarrassing picture of you to your friends." Instant compliance.
It's also wonderful to use when they bring over boyfriends/girlfriends...
Re:CDs are still readable (Score:4, Interesting)
Re:What I've found to work... (Score:2, Interesting)
I've had to do this at my workplace which existed when 9-track reel-to-reel tapes were around. What we discovered that we needed to transfer data from one format to another as new better formats are available. All media types will become obsolete and will degrade over time so what we discovered you will have to migrate your old data from format to another since that old format will become obsolete if we like or not. Also as storage media gets denser and denser we can fit 46 old 100MB per 9-track tapes in one 4.5GB DVD so we can store more data in smaller space and longer lasting. In theory a DVD-R properly stored will last about 20 years but will the equipment that read these will exist in 20 years is another question. In our case of 9-track tapes will readable (stored under the best conditions) but the equipment will not be so we decided that migrating from one format to another is best way to prevent obsolescence of data.
We migrated one roomful of 9-track tapes to one box of DVDs several years ago and we are happy for all of space we got back in our off-site storage location.
Re:Gold Disks (Score:5, Interesting)
Re:CDs still a pain. Keep it alive and available. (Score:3, Interesting)
Well , you could hire webspace to store the files. A good contract would ensure regular backups and redundancy . That way , you don't have to worry about it , but it's probably the most expensive option. And off course , in 20 years , that company might go bankrupt , leaving you with nothing
Re:What I've found to work... (Score:3, Interesting)
Flash Storage (Score:5, Interesting)
I'm amazed that no one mentioned it. Just get 16gb usb flash disks.
It has theoretically unlimited life for archiving. The only time it deteriorate is when you continuously write/erase it.
current media + open formats (Score:1, Interesting)
Exactly right. Optical media will be gone soon enough. Personally I copy from my dvd camcorder to hard disk using a program I wrote:
http://www.pixelbeat.org/programs/dvd-vr/
The fact that I had to reverse engineer the DVD-VR format emphasizes that the data format is at least as important for long term access as the media.
After extracting the video data from the DVD I then reencode and compress it further using the open dirac codec from the BBC which is specifically designed for this purpose.
Re:Without meaning any offence... (Score:4, Interesting)
I already have instructions on how to access my archives, what is available to whom, and what to destroy as part of my will.
Re:My method (Score:4, Interesting)
You're exactly right. I used to work for one of the largest banks in the country, large enough that I can say with confidence that every American, and probably 75% of the people in the world, have heard of them. Our backup system was amazingly well designed. A lot of very smart people drew very large salaries for a very long time just to design it, not to mention the millions of dollars in hardware, and gigantic fees to hardened facilities for offsite storage.
They had a major data loss once because some douchebag forgot to change the backup tapes when he was supposed to.
The best laid plans and so on.....
(I'll throw in that, while working for the same company, I learned that no amount of money spent on ultra high end UPS systems and backup generators can protect you from an incompetent technician replacing a battery without following proper procedures.)
Re:Hard Copies (Score:2, Interesting)
What you need is PaperBack [ollydbg.de], which prints the bits on paper. And the data can actually be recovered using a decent scanner.
Re:Look at history. . . (Score:3, Interesting)
You're not thinking in a sufficiently long term. Yes, my wife and I only watch videos of our kids when they were babies a couple times a year. Our kids are 4 and 5 now, and they love seeing themselves as babies, that's one of the best parts of the experience.
*But* - the real value of those videos will come much farther on. My wife enjoys seeing what I looked like as a baby and a young kid, and I enjoy seeing what she was like then, too. Our kids' future spouses may enjoy seeing their baby videos, but even that isn't a long enough term
The real value of those videos will come long after the OP is dead and gone. For example, my mom's dad died relatively young (his early 50s), before my parents even knew each other. He was an extremely skilled hunter and fisherman, and I marvel at the stringers of fish I see in pictures of him with his friends, for both the size and the quantity of the fish. He hunted all sorts of birds, raccoons, just about anything but deer. My mom says he wouldn't hunt deer because too many deer hunters would shoot at anything that moved in the bush without even seeing what it was. But he shot enough raccoons that my mom and my grandmother both had raccoon coats (a fashion at the time, but theirs were all made from coons my grandfather shot with his side by side 12-gauge).
I'm the only one in my family who fishes. I taught myself. I would have loved to have learned to hunt, too, and I'm sure I would have learned from my grandfather if he'd lived longer. I love all those old pictures of him. Sadly, my mom sold his old fishing gear, and shotgun, and marbles (what a collection! Like I've never seen before or since) when I was too young to even realize that I could/should object and say "Hey, keep that stuff! I want it!" It all went to an antique dealer. A bamboo baitcasting rod. Original Creek Chub Bait Company lures from the 1920s and 1930s, most still with their original boxes. His Pikie Minnow was in near-new condition, I remember.
So (everyone) by all means, preserve those family videos on a number of media. Hard drive. DVD. Blu-Ray. Whatever comes after Blu-Ray. DLT (been around a long time, and will be for a long time to come), flash drive, etc. If possible, pass down to your children not only the media, but devices capable of reading them.
In the even longer term, like hundreds of years from now, if a lot of video from the present day is preserved, the archeologists of the future will have a much easier time seeing what our times were like than archeologists today have of seeing what times just a few hundred years gone were like.
And yes, that means I think humans will be around for a long time to come. We're the most successful species in the history of the planet and we're not going away. I'll even make a bold prediction: 20 years from now, the air and water will both be cleaner than they are today. If anyone doubts this, let me tell you that I grew up in southern California in the 1970s, and despite the fact that California's pupulation has roughly doubled in that time, the number of cars on the roads has more than doubled, the air is better now than it was then. We've yet only scratched the surface of alternative fuel vehiclesf, and emissions of internal combustion engines can still be improved. 20 years from now, vehicles that use only an internal combustion engine for power will be in the minority. They might even be downright rare. And the air then will be as much better than the air now as the air now is better than the air in the 1970s in LA.
Don't know if I'll be around to see it, since I'm almost 50, but it's the world I want to leave for my kids. Along with their baby videos :)
Re:Good luck with that! (Score:3, Interesting)
Still, I don't think anyone who has mentioned hard drive has meant just sticking a hard drive full of data in a shoe box and burying it.
Re:We're talking about archiving, not backup... (Score:2, Interesting)
Okay, so I'm having trouble understanding the solution you propose. Let me see if I get this right...
You should "constantly archive" which means that you regularly make copies of your data for long term storage (preferably off-site). But you shouldn't use any removable media. So you're left with Hard Drives as your only option (remember, USB keys are still removable storage).
Did I get that right? Or did you skip the idea of off-site storage all together?
What "formats you will always be able to use" are there? Hard drives? And what makes hard drives so special? They still require an interface to read the data. Someone posted that IDE has been around over 20 years. That's great, but it's also currently end of life. How long is SATA going to last? Fibre Channel? Hopefully your Linux distro in 2030 will still be able to read NTFS5 partitions. Hard drives are heavy and delicate.
Or are you talking about replicating your entire system to ensure that you can read the data? And then shipping the whole server to Iron Mountain? Heck, while you're at it you might as well plug the thing in, give it an internet connection and set it up for on-line fail-over.
Or maybe you have a different definition of "archive" than the rest of us.
Re:Simple - disguise it as porn (Score:4, Interesting)
That's not quite good enough. When I wanted to back up drafts of my master's thesis, this is what I did.
1.) Assign each revision (or tape, in your case) a unique word combination of bizarre sexual acts. For instance "Ostrich feces smeared by Horny Redhead Orangutan Schoolgirls."
2.) Keep the list of mappings of backed-up files to unique names very, very safe. Keep the list, written down is fine, in a safe deposit box at one or more locations.
3.) Upload the "porn videos" to Usenet, Kazaa, Gnutella, etc.
I think you'll find this backup method more than sufficient to withstand fire, flood, meteors, and other acts of God. It sure saved my butt several times when I needed to find old versions of my thesis to build on in future work. If you want to see the final draft of my thesis, just search for "Crazy teen Lllama Sucking Blonde Elephant". There's about a million copies out there, just rename to .pdf.
For extra points, is anyone out there willing to write automated software to perform such backups? I'm thinking, you have the user enter a few dozen unique animal names, sexual acts, etc. Then, everytime you do an SVN commit, the backup manager chooses a unique combination of words, renames to .avi, and uploads to the usual locations.
Re:CDs are still readable (Score:3, Interesting)