Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
The Internet Technology

Advice on Remote Backup Services? 30

Posted by Cliff from the when-data-storage-outstrips-local-backup-capacity dept.
a-freeman asks: "Faced with the prospect of doing automated weekly backups for several servers with some 200 GB of files each, I have been looking for a remote backup solution. A couple of recent articles consider backup to hard drives, although I feel this still fails the 'separate snapshot in time' aspect of good backup policy, since with many of the solutions that I have seen, you will likely lose all your backups if your array gets corrupted. However, CD-Rs and DVDs are just too damn small. Can anyone recommend a remote backup service or interesting combination of hosting service + FTP/RSync/etc., or am I stuck buying a tape drive?"
This discussion has been archived. No new comments can be posted.

Advice on Remote Backup Services? More

Advice on Remote Backup Services?

Comments Filter:
  • You can probably keep using tape as an easier way to get data onto a more portable backup medium that can be locked away, then use snapshot backups to a set of mirrored or RAID-5'd array, which then send the data over a decent pipe to a remote server via rsync or scp (or NFS even) from the array, but probably not as frequently due to the time it would take to transfer the snapshot data (which may compress well, but I'm not sure).

    Just a thought.

  • Remote in what way? (Score:2, Insightful)

    by toygeek ( 473120 )
    Are you looking for remote as in "in the next rack over" or "somewhere across the internet" or somewhere in between? In short, define "remote."

    Tape is probably the best bet so far. As far as getting a good 'image' of it, tar it and stick it on a tape. Since you don't want hard drive array, and optical is out, tape is going to be the best way, I think, unless another /. reader has a better idea.

  • Why wouldn't you want a tape drive? (Score:5, Informative)

    by SpaFF ( 18764 ) on Friday May 02, 2003 @01:42PM (#5863415) Homepage
    "...or am I stuck buying a tape drive?"

    Whats wrong with a tape drive? It is a medium that was designed for backups. If you are going to be backing up large amounts of data you need a tape-library and remote backup software. If you want the convienence of harddrives then attach the tape-library to a machine with a whole lot of disk. You can backup to the disks first and then archive whats on the backup-server to tape. Most backup software programs allow you to do this.

    • I agree. We have a NAS maxtor MaxAttach, which has a SCSI port on the back just for a tape drive. I think that the largest Maxtor drive right now is right around 600 gigs (ours is 300 gigs), but it would be simple enough to replace the drives with larger ones. Then just slap a LTO drive on the back and setup your backup to tape to run every night, and overwrite your backup to disk to keep the maxtor clean, and you will have a fast recent backup and slower snapshots on tapes. :)

    • Re:Why wouldn't you want a tape drive? (Score:1, Funny)

      by Anonymous Coward
      Never Underestimate the Bandwidth of an Eighteen Wheeler full of 600 gig tapes....But the Latency blows.

    • While it was designed for backups, its not exactly designed for restores. I've known a number of people over the years who never realized their backups were failing, and found out the hard way when they needed something in a pinch.

      Other media have the advantage that you can access them directly, validate they're actually writing the data correctly, and have more random access to them. Yes, you can do these things with tapes, but its more difficult.

  • Does Anyone know also if there's any good open source backup solutions for tape libraries? Don't say Amanda, it doesn't span tapes.
    • Arkeia [arkeia.com] has a nice backup suite that while not open source does have a free (as in beer) edition. When I was evaluating it, it seemed to work great with my ADIC library.

      I would be using that now except for that our company already had a license for the Veritas backupexec software for windows so I was able to just download the linux client software for free.

  • Storix makes a product that allows for backups over IP to either tape or disk. This way you could do disk backups offsite. You can also perform disaster recovery over the the Network. You can't use bootp, but insert the install floppy/CD and install from the remote image.

  • rsync + remote server (Score:2, Informative)

    by Lerxst ( 1306 )
    I'm dealing with pretty much the same issue right now as you. I've set aside a dedicated backup server (cheap K6/2 400) with a lot of disk space which uses rsync to backup the other servers in the office. Then I'm using rsync on an offsite server to keep a backup of the backup. Seems to work well. Having some sort of a raid setup on this box would be even more insurance I suppose.

    I'm not using tape because the office I'm doing this for doesn't have a dedicated IT staff, and I'm not going there nightly/week
  • I have an Rsync backup [gotdns.org] walk thru. Once everything is on a large array, you can run backups to a localy mounted tape drive.

    Rsyncing keeps the network traffice to a minimum, and the local tape speeds up the backup to tape.

  • rdiff (Score:2, Informative)

    by GigsVT ( 208848 ) *
    Rdiff backup, does incremental snapshot.

    http://rdiff-backup.stanford.edu/

  • look at rdiff-backup if you use disks (Score:3, Informative)

    by Splork ( 13498 ) on Friday May 02, 2003 @02:27PM (#5863789) Homepage
    rdiff-backup [stanford.edu] is based on rsync but allows you to keep incrementals as well as full backups. great for disk based backups while maintaining lots of history.

    for redundancy and recoverability just use it to multiple backup disks at whatever level of redundancy you need. each one will have its own full set of incrementals so if you lose one, no big deal.

  • Tape drives. (Score:5, Informative)

    by Zapman ( 2662 ) on Friday May 02, 2003 @02:27PM (#5863793)
    "It's hard to beat the bandwidth of a stationwagon filled with DAT tapes"

    If you're talking: 1) several servers and 2) 200ish gigs per, welcome to needing a real backup solution.

    One thing to keep in mind is the three 'kinds' of backups. You will need to cover (or choose not to) all three.

    1) DR. Disaster recovery. A full image of ALL data, usually duplicated so you have a in house copy and a remote copy. Full system images, and a software package that can blast a full system image to a box, or full data and config backups that require an OS install before your restore. Usually this is somewhat light on tapes, since you'll only keep 2-3 weeks of them.

    2) File Recovery. Someone deleted something that they shouldn't have and need it restored. Or the Database equivelent: "We dropped this table 5 weeks ago, and discovered just now this random important process that hits it every 2 months. Can we restore the DBF file so we can get that table, data and schema back?" Sometimes DR feeds into File Recovery. You just keep the tapes longer. More expensive in tapes though, and you have data you'll not use (like OS images) wasting tape space. It's easier though.

    3) Archival. EG: The IRS mandates that we keep this data for N years (where N is usually greater than 7). Thankfully, this is a thin ammount of data, but it's important none the less. CD/DVD rock for this, but tapes are good too (so long as you're under 10 years. Media and reader issues will kill you after that).

    Good luck. Backups are a huge pain. Be sure to test the DR portion of it at least once a year. You'll be thankful you do.

    • Re:Tape drives. (Score:2, Interesting)

      by GigsVT ( 208848 ) *
      rdiff-backup and rsync with rotating incrementals are both able to do the first two very well, with the advantage of never needing a "full" backup after you do the first one, something tape will never be able to do. This makes things like offsite backups over slow and cheap links possible that would not be otherwise.

      We back up about 1TB of total data to a offsite backup over a 512kbit fractional T1, with daily rsync incremental snapshots that we keep for 30 days. Our data velocity is about 3-6GB per day
      • Your solution brings up the other two components to consider when looking at backup though:

        1) Data loss window: How much data can you lose if your system fails right before the backup starts? Sounds like you're initiating nightly, so you can lose twenty-four hours of data. Rsync would support reducing that (in fact, I've seen rsync solutions with three minute windows; just gotta make sure your script detects presence of active rsync's and alerts someone that the window's gotten too narrow). However, reduce
        • . Rsync would support reducing that (

          Correct, locally we run rsync with 30 minute cycles over 100bt. The script simply does a killall rsync at the beginning in case of a sudden data influx that causes an older script to run too long. Your sugesstions are good, the killall rsync was a quick and dirty way to buy a little insurance.

          Sounds like you're initiating nightly, so you can lose twenty-four hours of data.

          Offsites are a third level backup, we have local rsync incrementals, and also mirroring.
          • That's pretty much how I'd do it too :-)

            I've gotten to the point where I absolutely hate talking about DR though; every one I meet wants synchronous transfers and hot GSLB failover to a 100% capacity site on the other side of the country. It takes weeks to design, the price tag comes in at 150% of what they're paying now, they choke and disappear. It's been like this for years.

            Expensive things are expensive. Don't know why people keep thinking that an MSP/VAR/consultant is able to buy them for less money

  • Rsync local and remote (Score:2, Interesting)

    by bzant ( 256795 )
    I currently have about 10G of "live" data on my office network, each night I back up using rsync, to a server at my house. This happens over a 768k DSL. On an average day I am pushing about 120MB over the wire, and it takes 10-20 minutes. Also in the office I have a 120GB drive where I have made 10 copies of the data, and each hour during the work day from 10AM to 8PM it makes an rsync copy, over 100mbit link it takes about 6 minutes to replicate the changes. I also have a nightly tape backup.

    This system
  • As far as I can tell tapes are awful, but they are still unfortunatly the best choice.
    It looks like the Ultrium 2's might be the curent best of the bunch capacity/cost wise - with 200GB (uncompressed) per tape. I wish they were firewire/USB2 rather than having to prat with SCSI cards.

    The Maxtor MaxLine 2 hard drives do look tempting though; it will be interesting to see if the 300GB versions ever become available. (They were originally listed as 320GB!)
    • I recommend NetVault's software Bakbone. It's cheap and I've used it alot. DLT tapes and drives are expensive - nearly 100 USD for a tape and over 1000 USD for a station - but very fast and stable. You could also do backup to IDE-disks with NetVault. By far NetVault is the most flexible and less expensive (1000 USD approx. for a 5 server license )product I have ever seen or worked with. See http://www.bakbone.com for details.

  • Remote Backup Solutions (Score:2, Informative)

    by Anonymous Coward
    There is an old joke. "How do I move 3TB's from NYC to LA?" Answer, FedEx or UPS.

    If it's multiple remote sites as in WAN sites all over the country and you have 200GB's approx. per site to backup. Then you need to have something like a Compaq DLT Tape Library on each server and someone to rotate the tapes for offsite storage.

    We have many field locations so we backup to these DLT Tape libraries and either have an outsourced company like UNISYS or Siemans go to these offices and rotate the backup tapes.

  • rsync is fine when you have the bandwidth available, and when the time it takes to run the backup remains tolerable. However, unless you can write-lock the filesystem during the process, one interesting problem with a backup that takes a long time to run is that the backup copy represents a state that never really existed: the filesystem might have changed significantly while the backup was running.

    My favorite solution to this is breaking 3-way mirrors; see my earlier comment [slashdot.org] about this (along with the in

  • Am I missing something, or does remote (as in off-site) backup for large amounts of data imply vast bandwidth costs?

    Transferring 200Gb a day accross a 2Mb/s leased line (point to point) would be fairly fast, but then it would be idle most of the time while you're paying a monthly fee.

    The only remote backup solutions I've ever heard of are remote as in fiber to the next room (or building if you're lucky). Then it goes to tape.

    JJ

Slashdot Top Deals

Somebody ought to cross ball point pens with coat hangers so that the pens will multiply instead of disappear.

Close