Best Way To Store Digital Video For 20 Years?

An anonymous reader writes "My kid is now 1 year old and I already have 100G of digital video (stored on DVDs, DVD quality) and photos. How should I store it so that it's still readable 10 to 20 years from now? Will DVDs stil be around, and readable, 10 years from now? Should I plan for technology changes every 5 to 10 years (DVD->Blue-ray->whatever)? Is optical storage better, or should I try to use hard drives (making technology changes automatic)? And, if the answer is optical, how do you store optical disks so that they last?"

My method

[everphilski (877346)]

Pictures: Backed up to HDD, DVD and Flickr. For $24.95, it's cheap offline backup and the grandparents love it.

Movies: Taken on MiniDV, backed up to HDD.

The only worry I have is that the MiniDV's and HDD are in the same house although they are stored in separate locations. But every picture is backed up offsite.

Gold Disks

[stretchpuppy (1304751)]

Claim up to 300 years.

http://www.smarthouse.com.au/Home_Office/Storage/U9P4F7L2 [smarthouse.com.au]

Re:Gold Disks

[Kjella (173770)]

Claim up to 300 years.

On a very nice theoretic assumption on accelerated aging from disks stored less than a year. I expect at that age it's more than simple temperature and humidity factors that come to play, but I'm sure the company is happy to be long gone with your money before you start complaining in a few decades. Sure it might be good but it's hardly a proven technology.

Re:Gold Disks

[uncqual (836337)]

Be careful though... When burning these, it's critical that you only use gold [USB, SATA, PATA] cables to connect your burner to your computer. Otherwise the bits on the media will degrade quickly due to galvanic corrosion.

for some years: dvd + raid-1

[boldi (100534)]

Currently, There is no better way than store a backup on DVD and store the main data on a raid-1 disk set. Move the raid disk set to new disks every few years.

All the other technologies are more expensive, and even possibly more dangerous (loss of data due incompatibilies or for any other reason).

What I've found to work...

[Mordok-DestroyerOfWo (1000167)]

In the department I work for we typically use archival DVDs in a temperature and humidity controlled room (also used to store photos, slides, and vellum). For the really important ones I'll copy the disc onto a server in the same room as an ISO. Every month I mirror the data drive onto an offsite server in another building on campus. It's not fool-proof and it's pretty expensive but it has worked for about 8 years now.

Optical

[PawNtheSandman (1238854)]

Well why not optical platters? We have 50 year retainment requirements for certain documents and were looking at Plasmon optical devices. They claim it will still be readable and are the only type of backup media that survived both 9/11 and Katrina. Although when I asked if it was the same cartridge that survived both, the vendor gave me a dirty look. I think though you would be fine with dvd-r and just make a new copy every 5 years.

Optical?

[Anonymous Coward]

Only wimps use optical media, _real_ men just upload their important stuff on ftp and let the rest of the world mirror it.

Storage array.

[palegray.net (1195047)]

Build a simple storage array with RAID from a barbones PC, your favorite Linux distro, configured for fault-tolerant RAID. It doesn't have to be complicated, and it doesn't have to be powered on unless you're actually pushing data to it.

Every couple of years, you can add an extra couple of drives. With drive capacities increasing as fast as they are, cost shouldn't be a huge issue.

Use backups

[z00_miak (1305831)]

Regardless of the methods you choose, I would highly recommend using at least two different media.

If these videos are important enough to be stored for 10 to 20 years, then they are important enough to be backed up - it is always difficult to foresee long term failures in any technology. If you read the article on tin whiskers [slashdot.org] they mentioned that some failures can not be tested using short time span methods.

Do what IT departments do

[morgan_greywolf (835522)]

Use multiple different media, with redudancy.

Store it on HDDs. Mirrored RAID like RAID 1 or RAID 10 is preferred. but even RAID 5 buys you some extra integrity protection.

Then back it up. CDs. DVDs. BluRay. Tape. Whatever. Multiple times, multiple ways. Every few years do some copies onto new media.

Keep at least one copy off of your premises. A safe deposit box might be good.

Diversify.

[Lux (49200)]

If you can afford it, I'd recommend a utility computing platform, like Amazon S3 or whatever Google's offering in that space. Verify that they're built out for long-term, fault-tolerant storage (ie: replication + automated verification and repair.)

I wouldn't trust that 100%, though, so keep them locally as well.

One word, USENET.

[Roskolnikov (68772)]

Video I posted 20 years ago is still there....

Look at it this way:

[Anonymous Coward]

Instead of hiding behind the camera the whole time, actually interact and play with your kid. The videos and memories aren't as interesting as who the kid will become.

Re:Look at it this way:

[FroMan (111520)]

Sometimes that video isn't just for you, who is able to be there every day with your child. Grandparents who are unable to be there every day with the child really like to see videos also. Also, not every waking moment needs to be spent actively interacting with your child. Sometimes they need some time to play on their own or with other children. Some of the cutest moments with my son have been spent watching him explore the world around him on his own (and subsequently get stuck in the tupperware drawer).

Simple - disguise it as porn

[pwnies (1034518)]

1. Rename to "xxx 18yr old bj strip"
2. Upload to P2P protocol of choice.
Let it proliferate around the internet and retrieve it when necessary.

Wow, that's a lot.

[TheQuantumShift (175338)]

I sure hope you're shooting in 1080p and RAW, because otherwise your kid must think everyone has cameras growing out of their skulls... Seriously, put down the camera and live a little.

As for storage, I would personally go through and put together maybe a movie and and picture viewer DVD for each year. And then have those professionally mastered onto pressed discs. Keep those in your fire-proof storage and use burned copies for everyday (I hope not) use and sending to relatives and what not.

Re:Wow, that's a lot.

[everphilski (877346)]

100G is less than 10 hours of MiniDV footage. Birthday parties, time with the grandparents, 10 hours over the course of a year goes by fast.

Re:Wow, that's a lot.

[Dolohov (114209)]

That doesn't really apply here. It's really hard to tell what's good quality and what isn't if you're only within a few years of taking the family footage. All it takes is the death of a family member to make you wish you'd kept every scrap of video of your kid interacting with them.

The original poster will want to edit it down eventually, sure, but for the moment those edits should be along the lines of getting rid of dead air, finding the right encoding quality, getting rid of repetitive stuff in favor of a good sample (Two minutes of the kid putting a square block into a round hole is amusing. Thirty, not so much) and LABELING.

Re:Wow, that's a lot.

[pruss (246395)]

I have not been able to find anybody willing to press discs in quantities lower than about 150. Pressing discs in quantities of, say, 5 would be a nice service for archival purposes, but it would presumably be expensive, since I think the setup costs for pressing discs are high.

Still readable

[CastrTroy (595695)]

As the other guy mentioned, CDs are still readable, almost 20 years later. However, they didn't have a viable alternative until about 10 years ago. I think that you will easily be able to find a DVD drive for many years to come, at least the next 20. The problem becomes ensuring that the actual media doesn't get scratched. I wouldn't trust DVDs to last that long, even if you just leave them on a shelf, away from the sunlight. If I was really interested in saving the stuff, I would put it on hard disks with at least 1 redundant copy, if not 2, stored in different places, and transfer over every 3-4 years. Still, it's going to be a lot of data. Your kid is only 1, and you already have 100 GB of stuff. Just think about how much that will balloon to once the kid has an attention span of more than 43 seconds. The first hockey game, all the school plays, all the other junk you could record.

Personally, I just don't bother with recording much. My wife gets on my case for not taking a lot of pictures with the kids, but I'd rather be interacting and paying attention, rather than trying to ensure we have everything recorded. Sure sometimes like during school plays you can record and not miss anything, but a lot of times, I find when I'm trying to take videos, or photos, I end up missing out on the actual fun.

The strength of digital in archiving....

[Chyeld (713439)]

isn't long term storage, though it may have it. The strength is that you can reproduce it with high fidelity to the original numberous times.

The best way to store digital vidio for 20 years is to make numerous copies of it. 10Gigs is about 3 DVD's at the lowest density. Add a dvd of checksum files (something like a PAR [wikipedia.org]) and you should still be able to make five sets for under $20 if you are shopping around for DVD media.

Once a year or three, load up one of the sets and run it through the checksums. Correct any errors discovered via the checksums and copies from the other sets, and make another five sets.

Volia. Repeatable as long as there is any sort of cheap digital recording media that can easily fit your files out there.

The real question is how you do this when you have 1,000 Gig to backup.

Use S3

[42forty-two42 (532340)]

With S3 [amazonaws.com] you'd pay $15/mo (+bandwidth) to have it hosted online, instantly accessible. Will it still be around 20 years from now? One can't be certain, but if not, I'm sure you'll have enough warning to copy things off to another medium, and I'm sure there'll be similar services to take its place if need be.

How much could you store?

[veganboyjosh (896761)]

No one's brought it up yet, so I will... As the price/convenience/long term compatibility and viability of storage goes down and down, I wonder to what end we will end up keeping this stuff? How many hours of video that you're paying (in time, money, security against fire/damage/loss, etc) to keep up you're actually going to watch? Sure, it's nice to have every single event in your child's life on demand at the touch of a button/click of a mouse, but aren't just plain old memories ok? Does his entire life have to be recorded and watchable?

At some point, I came to the realization that I had downloaded over 6 solid months worth of music. This doesn't include CD's, LP's, or 7 inch records, of which I probably have 1000 total. If I were able to put all that music on a big loop, and not repeat anything, I'm thinking it would last over 12 months. Some of these I'll probably never listen to. I'm thinking the same is true for the submitter's videos.

My parents have a big box of photographs from their childhoods, as well as those of their parents. There are some great photos in that box, and I could and have spent hours going through them. Each time I do, I make a mental note that one day I'll scan them and make them digital. Then I realize that we only drag out that box once or twice a year, and never do anything with the photos anyway, and resign to scan them once it gets even cheaper.

Good luck with that!

[cpct0 (558171)]

People who say HDD have their heads in the sand. 20 years. Think about that. 1988. SCSI-1 40 pins. Nearing the end of MFM/RLE. Parallel.

People who say CDs and DVDs again have their heads in the sand. That's the Floppy Era.

The best format IMHO is the "current" format. DVDs + HDDs along with a live copy on your computer. DVDs and HDDs should be at two of your friend's houses.

5-10 years later, once one of the formats is obsolete (EXT3 is now EXT8, DVDs are now expensive again in drug stores), it's time to copy these to the new "current" format, and repeat the process.

Hard Copies

[rhesuspieces00 (804354)]

Print the bits out on paper to be scanned later, as necessary. You should make several copies and store them in different locations incase of fire or water damage. To answer your next question: Land in Montana and the Yukon territory is cheap.

Continually Transfer

[UnknowingFool (672806)]

Unfortunately there isn't a guarantee on any technology. CD/DVDs were supposed to last 100 years until that pesky mold and poor quality make them unsuitable for long term storage. HD-DVD was promising until it lost the format wars. HDs reliability varies with manufacturer and model. My suggestion is to back it up every 5 - 10 years onto new media to keep ahead of the curve. It's more work but you'll make sure it gets saved.

8mm -> VHS -> DVD -> Bluray -> Profit!!

NARA and Library of Congress

[Krieger (7750)]

Forget Wikipedia, ask the people who spend their lives trying to figure this out.

http://www.digitalpreservation.gov/you/digitalmemories.html [digitalpreservation.gov]

http://www.archives.gov/preservation/technical/guidelines.html [archives.gov]

http://www.archives.gov/preservation/family-archives/digitizing-photos.html [archives.gov]

http://www.archives.gov/preservation/storage/ [archives.gov]

Flash Storage

[ady1 (873490)]

I'm amazed that no one mentioned it. Just get 16gb usb flash disks.
It has theoretically unlimited life for archiving. The only time it deteriorate is when you continuously write/erase it.

Is your kid named Truman?

[new death barbie (240326)]

you're collecting 100 GIGS per YEAR?

When do you plan on WATCHING this stuff?

Odds are, by the time he's three, you'll be so sick of watching him grow up through a camera viewfinder you'll toss the camera into the back of the closet.

And if you ever have another kid, he'll grow up thinking he's adopted, because he can't find any photographic evidence of his childhood.

I speak from experience :)

Re:CDs are still readable

[Anonymous Coward]

no they're not. ever hear of cd rot?
store everything on hard drives, with duplicate backups stored off site.

Re:CDs are still readable

[Darkness404 (1287218)]

On CDs, the rot becomes visually noticeable in two ways: 1. When the CD is held up to a strong light, light shines through several pin-prick sized holes.[1] 2. Discoloration of the disc, which looks like a coffee stain on the disc (see also CD bronzing).[1]

In audio CDs, the rot leads to decreased audio quality, chatter, scrambled audio, and static. A Philips press officer has declared CD rot to be an isolated problem affecting only an "absolute minority" of cases. PDO has offered to replace any discs affected by CD bronzing if supplied with the defective disk and proof of purchase. However, according to the website of one of the affected record companies, Hyperion, PDO's helpline was discontinued in 2006 after a change of ownership, and defective CDs are now no longer replaced by the manufacturer, even though some of the affected record labels continue to offer replacements.[2]

CD bronzing is a specific variant of CD rot, a type of corrosion that affects the reflective layer of audio CDs and renders them unreadable over time. The phenomenon was first reported by John McKelvey in the September/October 1994 issue of American Record Guide.[1][2] Affected discs will show a uneven brownish discoloring that usually starts at the edge of the disc and slowly works its way towards the center. The top layer is affected before the bottom layer. The disc will become progressively darker over time; tracks at the end of the disc will show an increasing number of audio problems due to disc read errors before becoming unplayable. CD bronzing seems to occur mostly with audio CDs manufactured by Philips and Dupont Optical (PDO) at their plant in Blackburn, Lancashire, UK, between the years 1988 and 1993. Most, but not all of these discs have "Made in U.K. by PDO" etched into them (see image). Discs manufactured by PDO in other countries do not seem to be affected. A similar, if considerably less widespread problem occurred with discs manufactured by Optical Media Storage (Opti.Me.S) in Italy. PDO acknowledged that the problem was due to a manufacturing error on its part, but gave different explanations for the problem. The most widely acknowledged explanation is that the lacquer used to coat the discs was not resistant to the sulphur content of the paper in the booklets, which led to the corrosion of the aluminium layer of the disc, even though PDO later said it was because "a silver coating had been used on its discs instead of the standard gold."[3] Peter Copeland of the British Library Sound Archive confirms that silver instead of aluminium in the reflective layer of the CD would react with sulpheriferous sleeves, forming silver sulphate, which has a bronze colour.[4] A combination of the two factors seems likely because, as Barbara Hirsch of the University of California points out, the oxidation could only have occurred if the protective lacquer did not seal the metal film and substrate well enough.[2]

Those were from Wikipedia, fact is, though CD rot can be a problem, it isn't as bad as people make it out to be.

Re:CDs are still readable

[jason.sweet (1272826)]

Any storage medium you choose will degrade over time. You should plan to transfer the data every few years. Choose a storage medium that is well-supported, cheap and relatively durable - DVD or CD for now. You should also pay attention to the format of the data. If you use a video format that is rare now, chances are you won't have a way to convert it to whatever new format you need in 5 years. When it's time to copy (probably 5 years for CD & DVD, to be safe), use the same guidelines to choose the new medium.

Re:CDs are still readable

[Mr2cents (323101)]

Ok, I just went looking for the handbook I got with my photography course, there's a paragraph about optical media: The CD's with a greenish look are guaranteed up to survive for 1-3 years. The siverish CD's last about 10 years. And there are also more expensive CD's with a gold color, and a black protection layer on top, that last +/- 100 years.

It also mentions there is no durability data about DVD's yet. This seems strange to me, and it's maybe outdated.

It might be wise to get some advice at a photography store, I'm sure they get a lot of those questions.

Re:CDs are still readable

[hairyfeet (841228)]

Which is why you check the discs once a year.I have DVD backup discs going back to my first DVD burner,which was an optorite 4x I paid over $300 for. Out of the seven spindles or so of backups I have,I've only had around 3 go bad,and I was able to catch it every time.And this is with me using whatever is cheapest at the time.

I have found the best way to find disc rot is to use Emsa disk check [e-systems.ro] which is a free and fast little Windows utility. It will find when a disc is going bad LONG before it actually starts spitting errors. Anything important I burn in triplicate(1 for use,two for backups) and store in separate cool dry places. With DVD blanks being so cheap in bulk it costs hardly anything. I wouldn't use cd though simply because of the huge amount of discs you're talking about. With DVD it'll be easy to set aside a "checking day" which with Emsa goes quite quickly. Since it involves a loved one an easy way to remember checking day might be the day before or after their birthday.

Anyway this is what has worked for me and I have CDs going back to I think '97 and the DVDs start with some 1x,so whenever the first burner media came out. And so far knock on wood I haven't lost a single file to bit rot. But that is my 02c,YMMV

I have the opposite experience

[PerfectSmurf (882935)]

Your experience is very interesting because mine is the opposite. I make (at least) quarterly backups of my data and have since mid 1993 (CDs since 98, floppy before that). This spring I got bitten by the curisoity bug and started going through all my old backups looking for forgotten and interesting things. Every CD older than two years had at least one unrecovereable read error. Every CD older than five years, except for one, was completely unreadable. Between two and five years the number of read errors grew with many files being lost and several CDs being unusable. The 3-1/2 floppies were all 100% readable.

In that time period I've been through probably a dozen CD burners, both expensive varieties and cheap ones, and I've used at least as many brands of media. All the CDs have been kept stored in dark, dry, clean places. I tried reading the "unreadable" CDs on multiple computers and met some limited success accessing additional data. I didn't try any recovery software.

Fortunately for me I kept most of these backups out of habit and I didn't really care about much of the older ones outside of curiosity.

Re:CDs are still readable

[Lumpy (12016)]

Low grade garbage consumer CD-R's do that. you can buy high end CD-R's that have a gold substrate and a permanent dye that are guarenteed to last decades. I have a couple of TDK archival quality CD-R's from the very early 90's that were burned on a god-awful-expensive 1X CD burner that are still readable.

Do I store them on my car's dashboard? nope. I store them in a cool climate controlled media safe. I can still buy high end archival quality CD-R and DVD-R disks that I am sure will last a long time.

And if the 3.5" floppy drive is any example, CD and DVD drives will be around for another 10 years at least.

Recordable CD reliablity

[Lead Butthead (321013)]

Depends on the manufacturer and dye formulation. Some have failed in as short of a time as eight months while others are good after nearly ten years. For very important stuff, it is far too risky to be relying on the manufacturer. It's probably safer to make it a habit of regularly make multiple backups your data.

Re: You're looking for the wrong thing..

[RickRussellTX (755670)]

Taiko Yuden

Wow, a misspelled spelling flame. It's like digging into the ground and pulling up a USENET post from 1993. Let me introduce you to a 21st century technology that might help you in future: it's called a search engine.

It's Taiyo Yuden, according to their own web site [t-yuden.com].

Re:CDs are still readable

[SatanicPuppy (611928)]

Only professional CDs have that sort of shelf life, because they're physically stamped. The consumer grade ones use a type of photosensitive dye that DOES decompose in less than a decade.

Re:Where can I get these mythical disks?

[CastrTroy (595695)]

You may be able to copy them in Linux. Use "dd conv=noerror bs=2048 if=/dev/cdrom of=~/ImageFile.iso" . Then mount the resulting file as an IS09660 file, and you should be able to get some of the files off them. There may be some inconsistencies in the files, but most of the data should be there. I'm pretty sure i've used this method to get data off dead CDs.

Re:CDs still a pain. Keep it alive and available.

[twistedcubic (577194)]

Man, you got really upset over that RAID suggestion. With a name like 900ftJesus, I would have expected a more kinder, gentler approach.

Re:CDs still a pain. Keep it alive and available.

[RickRussellTX (755670)]

WW900FTJD?

Re:CDs are still readable

[Mister Whirly (964219)]

Back then we had to hand flatten the metal parts ourselves and burn them using a magnifying glass and the sun. And unless you had like 3 espressos, the only speed you could do it at was 1X.

Re:Stone Carvings

[Mordok-DestroyerOfWo (1000167)]

Step 1: Review video footage.
Step 2: Carve memorable/important parts into stone.
Step 3: ??? (commandments?)
Step 4: Prophet!

Re:keep copying it

[The Aethereal (1160051)]

As I understand it, holomagical disks will be loaded with so much DRM, nobody will be allowed to view the contents. Period.

Re:Tape

[fahrbot-bot (874524)]

tape is still one of the best archival storage types

Duct, Electrical, Masking or Transparent?

Re:HDDs

[DigitAl56K (805623)]

Do not buy the largest hard drives that you can find. Cutting-edge storage densities might mean bad long term reliability. Go for something a little smaller than the leading edge that has had enough test time for you to find a large number of reviews on the drive's reliability. These will also be cheaper so you can buy two and store them at separate locations.

When you store your files, write a small utility or script that runs through them all and builds redundancy data, like QuickPar, and that stores a hash, e.g. MD5, so that 20 years later you can check the data is still good bit for bit, and even if it contains errors theres a high probability that you'll be able to correct it, even from the redundancy data stored on the same drive, let alone your second copy.

One important thing: As well as all of this, on each drive store a copy of the software (e.g. codecs), as well as any registration information to make them work. Backups of your files are no good if you can't play them later. Try to avoid storing video long term in any format that requires a codec with online activation. Will that company still be around in 20 years time?

Re:HDDs

[lgw (121541)]

Oops! Dammit, preview, preview!

Re:Without meaning any offence...

[mbone (558574)]

I suspect that in a few decades, estate planning will include what to do with the family terabytes.