Advice on Remote Backup Services? 30
a-freeman asks: "Faced with the prospect of doing automated weekly backups for several servers with some 200 GB of files each, I have been looking for a remote backup solution. A couple of recent articles consider backup to hard drives, although I feel this still fails the 'separate snapshot in time' aspect of good backup policy, since with many of the solutions that I have seen, you will likely lose all your backups if your array gets corrupted. However, CD-Rs and DVDs are just too damn small. Can anyone recommend a remote backup service or interesting combination of hosting service + FTP/RSync/etc., or am I stuck buying a tape drive?"
Maybe a combination? (Score:1)
Just a thought.
Remote in what way? (Score:2, Insightful)
Tape is probably the best bet so far. As far as getting a good 'image' of it, tar it and stick it on a tape. Since you don't want hard drive array, and optical is out, tape is going to be the best way, I think, unless another
Why wouldn't you want a tape drive? (Score:5, Informative)
Whats wrong with a tape drive? It is a medium that was designed for backups. If you are going to be backing up large amounts of data you need a tape-library and remote backup software. If you want the convienence of harddrives then attach the tape-library to a machine with a whole lot of disk. You can backup to the disks first and then archive whats on the backup-server to tape. Most backup software programs allow you to do this.
Re:Why wouldn't you want a tape drive? (Score:3, Insightful)
Re:Why wouldn't you want a tape drive? (Score:1, Funny)
Re:Why wouldn't you want a tape drive? (Score:1)
While it was designed for backups, its not exactly designed for restores. I've known a number of people over the years who never realized their backups were failing, and found out the hard way when they needed something in a pinch.
Other media have the advantage that you can access them directly, validate they're actually writing the data correctly, and have more random access to them. Yes, you can do these things with tapes, but its more difficult.
OpenSource Backup Solution?? (Score:1)
Re:OpenSource Backup Solution?? (Score:3, Informative)
I would be using that now except for that our company already had a license for the Veritas backupexec software for windows so I was able to just download the linux client software for free.
Re:OpenSource Backup Solution?? (Score:2, Informative)
Storix System Backup Administrator (Score:1)
rsync + remote server (Score:2, Informative)
I'm not using tape because the office I'm doing this for doesn't have a dedicated IT staff, and I'm not going there nightly/week
Rsync to large local disk array, then to tape (Score:3, Informative)
Rsyncing keeps the network traffice to a minimum, and the local tape speeds up the backup to tape.
rdiff (Score:2, Informative)
http://rdiff-backup.stanford.edu/
look at rdiff-backup if you use disks (Score:3, Informative)
for redundancy and recoverability just use it to multiple backup disks at whatever level of redundancy you need. each one will have its own full set of incrementals so if you lose one, no big deal.
Tape drives. (Score:5, Informative)
If you're talking: 1) several servers and 2) 200ish gigs per, welcome to needing a real backup solution.
One thing to keep in mind is the three 'kinds' of backups. You will need to cover (or choose not to) all three.
1) DR. Disaster recovery. A full image of ALL data, usually duplicated so you have a in house copy and a remote copy. Full system images, and a software package that can blast a full system image to a box, or full data and config backups that require an OS install before your restore. Usually this is somewhat light on tapes, since you'll only keep 2-3 weeks of them.
2) File Recovery. Someone deleted something that they shouldn't have and need it restored. Or the Database equivelent: "We dropped this table 5 weeks ago, and discovered just now this random important process that hits it every 2 months. Can we restore the DBF file so we can get that table, data and schema back?" Sometimes DR feeds into File Recovery. You just keep the tapes longer. More expensive in tapes though, and you have data you'll not use (like OS images) wasting tape space. It's easier though.
3) Archival. EG: The IRS mandates that we keep this data for N years (where N is usually greater than 7). Thankfully, this is a thin ammount of data, but it's important none the less. CD/DVD rock for this, but tapes are good too (so long as you're under 10 years. Media and reader issues will kill you after that).
Good luck. Backups are a huge pain. Be sure to test the DR portion of it at least once a year. You'll be thankful you do.
Re:Tape drives. (Score:2, Interesting)
We back up about 1TB of total data to a offsite backup over a 512kbit fractional T1, with daily rsync incremental snapshots that we keep for 30 days. Our data velocity is about 3-6GB per day
Re:Tape drives. (Score:2)
1) Data loss window: How much data can you lose if your system fails right before the backup starts? Sounds like you're initiating nightly, so you can lose twenty-four hours of data. Rsync would support reducing that (in fact, I've seen rsync solutions with three minute windows; just gotta make sure your script detects presence of active rsync's and alerts someone that the window's gotten too narrow). However, reduce
Re:Tape drives. (Score:1)
Correct, locally we run rsync with 30 minute cycles over 100bt. The script simply does a killall rsync at the beginning in case of a sudden data influx that causes an older script to run too long. Your sugesstions are good, the killall rsync was a quick and dirty way to buy a little insurance.
Sounds like you're initiating nightly, so you can lose twenty-four hours of data.
Offsites are a third level backup, we have local rsync incrementals, and also mirroring.
Re:Tape drives. (Score:2)
I've gotten to the point where I absolutely hate talking about DR though; every one I meet wants synchronous transfers and hot GSLB failover to a 100% capacity site on the other side of the country. It takes weeks to design, the price tag comes in at 150% of what they're paying now, they choke and disappear. It's been like this for years.
Expensive things are expensive. Don't know why people keep thinking that an MSP/VAR/consultant is able to buy them for less money
Rsync local and remote (Score:2, Interesting)
This system
Tapes are horrible - but still the best possible (Score:1)
It looks like the Ultrium 2's might be the curent best of the bunch capacity/cost wise - with 200GB (uncompressed) per tape. I wish they were firewire/USB2 rather than having to prat with SCSI cards.
The Maxtor MaxLine 2 hard drives do look tempting though; it will be interesting to see if the 300GB versions ever become available. (They were originally listed as 320GB!)
Try DLT tapes and Bakbone's NetVault (Score:1)
Remote Backup Solutions (Score:2, Informative)
If it's multiple remote sites as in WAN sites all over the country and you have 200GB's approx. per site to backup. Then you need to have something like a Compaq DLT Tape Library on each server and someone to rotate the tapes for offsite storage.
We have many field locations so we backup to these DLT Tape libraries and either have an outsourced company like UNISYS or Siemans go to these offices and rotate the backup tapes.
Break 3-way mirrors (Score:1)
rsync is fine when you have the bandwidth available, and when the time it takes to run the backup remains tolerable. However, unless you can write-lock the filesystem during the process, one interesting problem with a backup that takes a long time to run is that the backup copy represents a state that never really existed: the filesystem might have changed significantly while the backup was running.
My favorite solution to this is breaking 3-way mirrors; see my earlier comment [slashdot.org] about this (along with the in
Remote backup == massive cost? (Score:2)
Transferring 200Gb a day accross a 2Mb/s leased line (point to point) would be fairly fast, but then it would be idle most of the time while you're paying a monthly fee.
The only remote backup solutions I've ever heard of are remote as in fiber to the next room (or building if you're lucky). Then it goes to tape.
JJ