Ask Slashdot: It's World Backup Day; How Do You Back Up?

MrSeb writes "Today is World Backup Day! The premise is that you back up your computers on March 31, so that you're not an April Fool if your hard drive crashes tomorrow. How do Slashdot users back up? RAID? Multiple RAIDs? If you're in LA, on a fault line, do you keep a redundant copy of your data in another geographic region?"

Tahoe LAFS

[swillden]

I use a secure distributed grid. The software is an open source tool, Tahoe LAFS (http://tahoe-lafs.org [tahoe-lafs.org]). The grid is composed of ~15 servers contributed by different people all over the world. There are a half dozen servers in various locations in the US, about the same number in Europe, and the remainder in Russia and the Ukraine.

My files are AES256-encrypted on my machine, split into 13 pieces using Solomon-Reed coding, any five of which are sufficient to reconstruct my files, and then those 13 pieces are distributed to the servers in the grid. I run daily backups, but since uploads to the grid are idempotent, only the changed or new files are stored. I also run a bi-weekly "repair" operation which checks all of my files (all versions, from all backup runs) to see if any of their pieces are lost. If so, it reconstructs the missing pieces and deploys them to servers in the grid. The individual servers in the grid are fairly reliable, but problems do happen, so repair is important.

I get about 100 KBps net upload rate, so this isn't a good solution for backing up terabytes, and the occasional "surge" in my data generation (usually caused by a day of heavy photo-taking) often causes my "daily" backup to take a few days to run, but all in all it works very well.

Should my server ever die, I only need two pieces of information to get all of my data back: The grid "introducer" URL, which will allow me to set up a new node connected to the grid, and my root "dircap", which is a ~100-byte string containing the identifier and decryption key for the root directory of my archive. That directory contains the decryption keys for the files and directories it references.

Since this grid is all volunteer-based, the only cost to me for this backup solution is the hardware and bandwidth I provide to my grid (I provide 1 TB of disk and grid usage consumes a fairly small fraction of my Comcast connection), plus the time I spend administering my server and checking to see that my backup and repair processes are running. Oh, and I also contribute (a little) to the Tahoe LAFS project, but that's due to interest, not a requirement.

I'm very, very happy with this solution.

BTW, the grid could use another 20 nodes or so, if anyone is interested. There's a fair amount of trust required of new members to the grid, though, so it might take us a while to vet new members. The trust is required not because other members of the grid might have access to files that are not their own, but we need to verify that new members will behave appropriately -- providing their fair share of storage and bandwidth, and not consuming too much.

Anyone interested should check out the grid's policies and philosophy at: http://bigpig.org/twiki/bin/view/Main/WebHome [bigpig.org]. If all of that looks good, join the mailing list, introduce yourself and we'll consider allowing you to join the grid.

Re:ZFS

[TheRaven64]

I use both. Time Machine back up to a deduplicated RAID-Z volume. When Time Machine backs up a file (e.g. a VM disk image or an 8MB stripe from a sparse image) with only a few small changes, the decuplication kicks in and means it only takes up a couple of blocks.