Best Way To Store Digital Video For 20 Years?

An anonymous reader writes "My kid is now 1 year old and I already have 100G of digital video (stored on DVDs, DVD quality) and photos. How should I store it so that it's still readable 10 to 20 years from now? Will DVDs stil be around, and readable, 10 years from now? Should I plan for technology changes every 5 to 10 years (DVD->Blue-ray->whatever)? Is optical storage better, or should I try to use hard drives (making technology changes automatic)? And, if the answer is optical, how do you store optical disks so that they last?"

CDs are still readable

[Calinous]

CDs are still readable, after almost 20 years

Multiple times, repeatedly

[Anonymous Coward]

Different media, copied over to new media after a few years.

CDs still a pain. Keep it alive and available.

[ibane]

If you strive to keep it all accessible all the time, you will move with format changes as they occur. US networks are not capable of HD video streaming, so I put OGG Theora in my video blogs with links to better quality for those who want it. Disk storage will improve in time to keep up with your vorracious demands. Raid would be good to have. Optical storage media that has to be loaded one disk at a time is a last ditch archive that you should keep in a seperate physical location, just in case.

What I've found to work...

[Mordok-DestroyerOfWo]

In the department I work for we typically use archival DVDs in a temperature and humidity controlled room (also used to store photos, slides, and vellum). For the really important ones I'll copy the disc onto a server in the same room as an ISO. Every month I mirror the data drive onto an offsite server in another building on campus. It's not fool-proof and it's pretty expensive but it has worked for about 8 years now.

groovy man

[FizzGiGG]

Store the media on whatever the current standard is. Think about it, what if you had a closet full of tape reals that had all of your old sweet groovy 60's music? What would you do with that now?

buy an external eSATA RAID5 array

[circletimessquare]

a few years ago, this would have been exorbitantly expensive overkill, but this stuff keeps getting cheaper by the day

with raid5, your videos will last forever, as long as someone keeps replacing the dead drives

any other media format is physically static, which can degrade. raid5 ensures that the files live on after the physical components degrade, as long as new drives are continually added to the system

and when the technology becomes ancient and archaic, simply move the files over and upgrade (obviously to a new file format as well)

as long as some continually performs low level maintenance, your videos will last forever

Do what IT departments do

[morgan_greywolf]

Use multiple different media, with redudancy.

Store it on HDDs. Mirrored RAID like RAID 1 or RAID 10 is preferred. but even RAID 5 buys you some extra integrity protection.

Then back it up. CDs. DVDs. BluRay. Tape. Whatever. Multiple times, multiple ways. Every few years do some copies onto new media.

Keep at least one copy off of your premises. A safe deposit box might be good.

Diversify.

[Lux]

If you can afford it, I'd recommend a utility computing platform, like Amazon S3 or whatever Google's offering in that space. Verify that they're built out for long-term, fault-tolerant storage (ie: replication + automated verification and repair.)

I wouldn't trust that 100%, though, so keep them locally as well.

Look at it this way:

[Anonymous Coward]

Instead of hiding behind the camera the whole time, actually interact and play with your kid. The videos and memories aren't as interesting as who the kid will become.

Wow, that's a lot.

[TheQuantumShift]

I sure hope you're shooting in 1080p and RAW, because otherwise your kid must think everyone has cameras growing out of their skulls... Seriously, put down the camera and live a little.

As for storage, I would personally go through and put together maybe a movie and and picture viewer DVD for each year. And then have those professionally mastered onto pressed discs. Keep those in your fire-proof storage and use burned copies for everyday (I hope not) use and sending to relatives and what not.

Still readable

[CastrTroy]

As the other guy mentioned, CDs are still readable, almost 20 years later. However, they didn't have a viable alternative until about 10 years ago. I think that you will easily be able to find a DVD drive for many years to come, at least the next 20. The problem becomes ensuring that the actual media doesn't get scratched. I wouldn't trust DVDs to last that long, even if you just leave them on a shelf, away from the sunlight. If I was really interested in saving the stuff, I would put it on hard disks with at least 1 redundant copy, if not 2, stored in different places, and transfer over every 3-4 years. Still, it's going to be a lot of data. Your kid is only 1, and you already have 100 GB of stuff. Just think about how much that will balloon to once the kid has an attention span of more than 43 seconds. The first hockey game, all the school plays, all the other junk you could record.

Personally, I just don't bother with recording much. My wife gets on my case for not taking a lot of pictures with the kids, but I'd rather be interacting and paying attention, rather than trying to ensure we have everything recorded. Sure sometimes like during school plays you can record and not miss anything, but a lot of times, I find when I'm trying to take videos, or photos, I end up missing out on the actual fun.

The strength of digital in archiving....

[Chyeld]

isn't long term storage, though it may have it. The strength is that you can reproduce it with high fidelity to the original numberous times.

The best way to store digital vidio for 20 years is to make numerous copies of it. 10Gigs is about 3 DVD's at the lowest density. Add a dvd of checksum files (something like a PAR [wikipedia.org]) and you should still be able to make five sets for under $20 if you are shopping around for DVD media.

Once a year or three, load up one of the sets and run it through the checksums. Correct any errors discovered via the checksums and copies from the other sets, and make another five sets.

Volia. Repeatable as long as there is any sort of cheap digital recording media that can easily fit your files out there.

The real question is how you do this when you have 1,000 Gig to backup.

So when he's 20...

[Anonymous Coward]

You'll have 2 Tb? Or a lot more, if technology goes asymptotic?

I suggest you invest in an editor, and slim your storage down to what is reasonable given current technology - perhaps 5 CDs?

Then transfer to new technology as it appears, keeping only the amount that each new technology can reasonably handle

Alternatively, buy yourself a data centre..

Oh, there is one other way. Just send a few messages off to Iran asking about nuclear materials, and then send all your kids' photos over the net. The governments of the US, Europe and the Middle East will then keep all your data in a high security storage facility, free of charge...

Re:CDs still a pain. Keep it alive and available.

[lgw]

If you're not trying to go cheap, get a tape drive (DLT, LTO, or AIT, not the quarter-inch or DAT crap). If your time isn't worth that much, migrate from optical format to optical format every few years. Either way, keep your backups off-site.

Hard drives and just not suitable for (home) archiving - one robbery, fire, or natural disaster and everything's gone forever. If you add backup to those hard drives, then we're back to "what format?".

Why oh why?

[Anonymous Coward]

Who is going to want to watch 100 gigabytes of your kid? At 5 GB / DVD, that's 20 movies = 40 hours. I don't care if my kid is the next Beethoven, I'm never going to watch 40 hours of diaper footage in my life.

I suggest keeping it on CDs in different places. Hopefully about 15 seconds of footage will survive.

Re:Live your life, don't record it

[Anonymous Coward]

It's also important to remember that your kid isn't special or important, even though you feel like he is. Unfortunately, all this constant recording of every little thing he does will turn him into a self-important jackass.

Use S3

[42forty-two42]

With S3 [amazonaws.com] you'd pay $15/mo (+bandwidth) to have it hosted online, instantly accessible. Will it still be around 20 years from now? One can't be certain, but if not, I'm sure you'll have enough warning to copy things off to another medium, and I'm sure there'll be similar services to take its place if need be.

Re:HD unreliable

[Firehed]

I was under the impression that unspun drives tend to deteriorate relatively quickly - the heads clashing with the platter or some such nonsense. Just spin them up once a month and you're fine, from what I've heard.

Re:Wow, that's a lot.

[everphilski]

100G is less than 10 hours of MiniDV footage. Birthday parties, time with the grandparents, 10 hours over the course of a year goes by fast.

Re:HD unreliable

[Al Kossow]

The only way this could be true is if the data were rewritten.
Reading alone has no effect on the data.

Re:CDs are still readable

[Darkness404]

On CDs, the rot becomes visually noticeable in two ways: 1. When the CD is held up to a strong light, light shines through several pin-prick sized holes.[1] 2. Discoloration of the disc, which looks like a coffee stain on the disc (see also CD bronzing).[1]

In audio CDs, the rot leads to decreased audio quality, chatter, scrambled audio, and static. A Philips press officer has declared CD rot to be an isolated problem affecting only an "absolute minority" of cases. PDO has offered to replace any discs affected by CD bronzing if supplied with the defective disk and proof of purchase. However, according to the website of one of the affected record companies, Hyperion, PDO's helpline was discontinued in 2006 after a change of ownership, and defective CDs are now no longer replaced by the manufacturer, even though some of the affected record labels continue to offer replacements.[2]

CD bronzing is a specific variant of CD rot, a type of corrosion that affects the reflective layer of audio CDs and renders them unreadable over time. The phenomenon was first reported by John McKelvey in the September/October 1994 issue of American Record Guide.[1][2] Affected discs will show a uneven brownish discoloring that usually starts at the edge of the disc and slowly works its way towards the center. The top layer is affected before the bottom layer. The disc will become progressively darker over time; tracks at the end of the disc will show an increasing number of audio problems due to disc read errors before becoming unplayable. CD bronzing seems to occur mostly with audio CDs manufactured by Philips and Dupont Optical (PDO) at their plant in Blackburn, Lancashire, UK, between the years 1988 and 1993. Most, but not all of these discs have "Made in U.K. by PDO" etched into them (see image). Discs manufactured by PDO in other countries do not seem to be affected. A similar, if considerably less widespread problem occurred with discs manufactured by Optical Media Storage (Opti.Me.S) in Italy. PDO acknowledged that the problem was due to a manufacturing error on its part, but gave different explanations for the problem. The most widely acknowledged explanation is that the lacquer used to coat the discs was not resistant to the sulphur content of the paper in the booklets, which led to the corrosion of the aluminium layer of the disc, even though PDO later said it was because "a silver coating had been used on its discs instead of the standard gold."[3] Peter Copeland of the British Library Sound Archive confirms that silver instead of aluminium in the reflective layer of the CD would react with sulpheriferous sleeves, forming silver sulphate, which has a bronze colour.[4] A combination of the two factors seems likely because, as Barbara Hirsch of the University of California points out, the oxidation could only have occurred if the protective lacquer did not seal the metal film and substrate well enough.[2]

Those were from Wikipedia, fact is, though CD rot can be a problem, it isn't as bad as people make it out to be.

How many pictures are you going to keep?

[ninjapiratemonkey]

Honestly, you're probably not going to look at most of the pictures in 20 years anyways. Sure it's good to have a lot, but with 100 GB of pictures/videos, that's pushing it a lot; babies don't exactly do much. Go through the collection and cut out the bad/redundant pictures, print off the especially good ones, and put em on the wall: they'll get more use that way. But... with what you do save, try burning to CD/DVD/blu-ray, since they don't degrade too much over time, and if you have space/money, archive it to tape: it'll last the longest. HDD's won't last the 20 years, but if you want to, try a RAID, but it'll have to be recopied and replaced every so often. And... with whatever media you choose, keep it away from light/dust/too much humidity.

100G in one year?

[saccade.com]

100G of video in the first year? I guess you should archive it...he can show it to his therapist when he gets older. One way to solve the archiving problem is to do some editing (serious editing) so there's much less to store.

Good luck with that!

[cpct0]

People who say HDD have their heads in the sand. 20 years. Think about that. 1988. SCSI-1 40 pins. Nearing the end of MFM/RLE. Parallel.

People who say CDs and DVDs again have their heads in the sand. That's the Floppy Era.

The best format IMHO is the "current" format. DVDs + HDDs along with a live copy on your computer. DVDs and HDDs should be at two of your friend's houses.

5-10 years later, once one of the formats is obsolete (EXT3 is now EXT8, DVDs are now expensive again in drug stores), it's time to copy these to the new "current" format, and repeat the process.

That's a lot of video...

[OpCode42]

So in his first year of life, you've recorded around 34-35 hours of footage? (going on single layer dvd capacities and mpeg2) Ask yourself, when are you going to watch all of that?

Continually Transfer

[UnknowingFool]

Unfortunately there isn't a guarantee on any technology. CD/DVDs were supposed to last 100 years until that pesky mold and poor quality make them unsuitable for long term storage. HD-DVD was promising until it lost the format wars. HDs reliability varies with manufacturer and model. My suggestion is to back it up every 5 - 10 years onto new media to keep ahead of the curve. It's more work but you'll make sure it gets saved.

8mm -> VHS -> DVD -> Bluray -> Profit!!

Re:HDDs

[lgw]

No, no, no! Mirroring is not backup. One theft, fire, or natural disaster and you're toast; plus mirroring provides no protection at all against the 85% case for data loss: "Oops! Dammit, undo, undo!"

A tried and true method

[SpacePunk]

Chisel what you want to keep onto stone tablets, or use clay tablets then encase them in a clay envelope. Then bury them in a pit in your back yard.

Thanks for the memories indeed!

[ADRA]

Is it just me, or do you find that besides a few photos meant to invoke memories, its better to remember something than to record it? I find If I over-indulge in 'capturing the moment', all I seem to have left was the content and I forget what it was like to -be- there.

Re:CDs are still readable

[twistedcubic]

Hell, why not go the whole hog and bury a small PC with the drive and DVDs? Though, I do like the other suggestions of using a tape drive. Some of my CDRs from the 90s degraded after just a few years packed in my closet.

Re:Look at it this way:

[FroMan]

Sometimes that video isn't just for you, who is able to be there every day with your child. Grandparents who are unable to be there every day with the child really like to see videos also. Also, not every waking moment needs to be spent actively interacting with your child. Sometimes they need some time to play on their own or with other children. Some of the cutest moments with my son have been spent watching him explore the world around him on his own (and subsequently get stuck in the tupperware drawer).

That's one of the problems with Linux people..

[raehl]

Build a simple storage array with RAID from a barbones PC, your favorite Linux distro, configured for fault-tolerant RAID.

If you're not a Linux person, that *IS* complicated by default.

Not that it would really be less complicated with Windows, but only a Linux person wouldn't recognize the inherent complication of RAID.

Quality not quantity

[NotQuiteReal]

Edit down a "best of" video once in a while. If you don't do it as you go along you never will.

A few good pictures and a handful of short videos become "precious memories".

A slag heap of hundreds of hours of raw material become a burden that someone will eventually stop maintaining because it is such a chore.

Lots of pictures are less of a problem than video both because they are smaller, but because you can look at them faster to see if there are any worth copying, printing etc.

Send copies of your "best of" to friends and family so that you have off-site storage should your house burn down.

Re:for some years: dvd + raid-1

[Kamokazi]

That's a pretty good idea, but I would change it up a little.

Raid 1=Good, keep that going...gives you on-the-fly data protection.

Skip the DVDs and get 1 or 2 more external HDDs (preferrably 2) for off-site storage. Every month or so (However often you feel you need), backup the RAID 1 array to the single HDD, and take it to a relative/friend's house...someone near enough you can do this regularly, but far enough away that if a tornado or something hits, they will probably not be affected. I said 2 extras drives for 2 reasons: 1) Easier to swap and only make one trip, and 2) you can backup more frequently and keep it at home in a fireproof safe.

Also, I would keep really important files (not the video) in guaranteed remote storage. Just in case. Lots of services online offer this.

The key is to archive less...

[Dzimas]

My wife and I have taken thousands of digital photos and videos since 2002, but the cold, hard truth is that the vast majority of them just aren't very good. I suspect the OP is in the same boat. The solution is to take the time to identify the best images and videos. This should result in a relatively compact archive only a few hundred digital photos and videos. This "best of" collection will be much easier to duplicate in different format and making physical prints on archival quality paper won't put a huge dent in your pocketbook. Remember that just because you can take 2000 photos of your spittle-covered toddler doesn't mean that you need to keep them all.

Chances are pretty good that your great grandchildren aren't going to give a damn about inheriting a massive archive of pictures and videos starring weirdly dressed dead relatives they've never met, and if it's physically large it stands a good chance of ending up in a garden shed or unprotected lunar storage pod. If you condense your family album into something more manageable, it will be more accessible and enjoyable for all - now and in the future.

Re:CDs are still readable

[joelwyland]

CD-Rs are not the same thing as stamped CDs. With CD-Rs you're lucky if they last 5 years.

This very week I pulled out some CDs I burned back in 1999 with old email on them. The CDs were in beautiful condition, all of the data was perfect. If you want to be sure that data will still be there, burn it 3 times, put each one in a different Case Logic book. Keep two of them in different areas of your home and a third off-site. Upload all your photos to Google's Picasa Web Albums as a 4th backup if you want. I'd be willing to bet that 10 years from now that Google will still be operating and will not have lost any of your data. The service may have changed, but old data will be brought along to the new service, etc.

Re:CDs are still readable

[Lumpy]

Low grade garbage consumer CD-R's do that. you can buy high end CD-R's that have a gold substrate and a permanent dye that are guarenteed to last decades. I have a couple of TDK archival quality CD-R's from the very early 90's that were burned on a god-awful-expensive 1X CD burner that are still readable.

Do I store them on my car's dashboard? nope. I store them in a cool climate controlled media safe. I can still buy high end archival quality CD-R and DVD-R disks that I am sure will last a long time.

And if the 3.5" floppy drive is any example, CD and DVD drives will be around for another 10 years at least.

Re:CDs are still readable

[Lumpy]

SCSI

I can read a SCSI-I drive in a Ultra 320 controller easily.

Re:HDDs

[Kjella]

One important thing: As well as all of this, on each drive store a copy of the software (e.g. codecs), as well as any registration information to make them work. Backups of your files are no good if you can't play them later. Try to avoid storing video long term in any format that requires a codec with online activation. Will that company still be around in 20 years time?

DVDs are played by all open-source tools today and mplayer will play all of MPEG2/VC1/H.264 (though slowly and not a problem in the far future), what kind of codec wouldn't you find?

Re:CDs are still readable

[negRo_slim]

no they're not. ever hear of cd rot?

There is in fact higher quality archival media. [cdr-zone.com] Taiyo Yuden media has been mentioned to me many times over the years in that regard...

Personally I keep 2 sets of copies a secondary hard disc, originals & edits. I also keep a master copy on standard Maxwell DVD+R that I check periodically generally when I add another volume, roughly 6 months time. I also keep another set on my gf's computer on the network, keeps her out of my hair in Photoshop ;) And I'm always assured to have at least 2 straight from the camera collections on physically separate devices and media.

Re:Wow, that's a lot.

[Dolohov]

That doesn't really apply here. It's really hard to tell what's good quality and what isn't if you're only within a few years of taking the family footage. All it takes is the death of a family member to make you wish you'd kept every scrap of video of your kid interacting with them.

The original poster will want to edit it down eventually, sure, but for the moment those edits should be along the lines of getting rid of dead air, finding the right encoding quality, getting rid of repetitive stuff in favor of a good sample (Two minutes of the kid putting a square block into a round hole is amusing. Thirty, not so much) and LABELING.

Re:Gold Disks

[trentfoley]

The wording here is what is important.

Up to 300 years includes 1 day. Since there is no minimum given, it is a semantically void promise. The only thing guaranteed is that your data will not last 300 or more years.

It is like the "Save up to 50% and more" sales. What does that really mean?

Re:buy an external eSATA RAID5 array

[TrevorB]

RAID5 is beaten by:

A) Disaster: Fire. Flood. Earthquake.
B) Carless mistake or Idiocy: "rm -r /" can happen to the best of us.

Nothing but an offsite backup will do.

I'd burn a copy to good quality DVD's in an offsite location (an office drawer at work does the trick for me) as you accumulate data. Replace the media every 5 years, as new media come out.

Re:CDs are still readable

[CamerasInAirplanes]

While in school for photography, I learned that CD's are not a long-term archivable medium. I wish I had the documentation in front of me to back up my claim, but unfortunately I don't (Yes, I kept all the information material from every class in college). Since the CD is light sensitive, similar to silver-halide film (don't take this analogy too seriously), the CD will degrade over time. DVDs are very similar to CDs in that they are 'laminated' layers of CDs and, as a result, will suffer more than CDs. I would never use DVDs to archive my photos. Kodak USED to make archivable CDs that supposedly lasted a lifetime+. Buy HDDs and don't look back...especially if the data is something you treasure.

Reading alone has no effect on the data.

[Anonymous Coward]

O RLY?

Regards,
Werner Heisenberg

Re:How much could you store?

[dwye]

> Sure, it's nice to have every single event in
> your child's life on demand at the touch of a
> button/click of a mouse, but aren't just plain
> old memories ok?

No. Auto accident. Child dead. Now what?

Also, if the original poster is smart, he will include his wife and himself in some of the videos, and his children and grandchildren can see what grandfather Surname was like if something happens to him, instead. Let his wife take a few of him, or it will be like our family, where we have just one half inch high photo of my one grandfather, who took all the photos of everyone else.

> Each time I do, I make a mental note that one day
> I'll scan them and make them digital. Then I realize
> that we only drag out that box once or twice a year,
> and never do anything with the photos anyway, and
> resign to scan them once it gets even cheaper.

Scan them before a leaky roof or basement ruins them. Annotate them, while someone still lives who can identify who is who. Then you have a backup to the photos, as well.

Re:CDs are still readable

[DaveV1.0]

Let me introduce you to a concept I call down-engineering. What happens is something is made really well. So well that the thing lasts a long, long time. This is bad for profits. So, the company has a choice of two:
1) develop something new
2) make the thing less reliable

So, the company making the thing begins to use poorer quality material to increase the failure rate over time.

What lasted 10 years soon only lasts 5, then 3, then 1 year. And people buy more.

Re:CDs are still readable

[russellh]

If you want to be sure that data will still be there, burn it 3 times, put each one in a different Case Logic book. Keep two of them in different areas of your home and a third off-site.

No no no no. Burn it 9 times, with three different brands of media on three different burners.

Is your kid named Truman?

[new death barbie]

you're collecting 100 GIGS per YEAR?

When do you plan on WATCHING this stuff?

Odds are, by the time he's three, you'll be so sick of watching him grow up through a camera viewfinder you'll toss the camera into the back of the closet.

And if you ever have another kid, he'll grow up thinking he's adopted, because he can't find any photographic evidence of his childhood.

I speak from experience :)

Re:CDs are still readable

[tgd]

If your files are small enough to fit on DVDs -- I've got individual video files over 50GB.

Sure, you can split them, but then you have to make damn certain you don't lose or otherwise damage a disk, or you have to generate a parity disk or twenty...

Its good for some stuff, not for massive amounts of data.

Re:Thanks for the memories indeed!

[JBFrobozz]

I have 13 DVDs of my life. From my first steps to high school graduation. 8mm to MiniDV, all converted to DVD now. One of the things that my dad quickly learned was that the video of birthday parties and school musicals is boring to watch. The good stuff is just random everyday life, such as I when I ran my bike straight into the back of our neighbor's truck. Everyone who has ever watched my DVDs has been jealous that they don't have something similar.

I have about 30 hours of childhood video over the course of my childhood which was in the neighborhood of 160,000 hours. I certainly had plenty of time to interact with my parents, but I also have some great footage that is fun to watch.