Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Hardware

Backing Up 100 Gigs in an Hour? 79

Posted by Cliff from the that's-a-lot-of-data dept.
cybrthng asks: "I am faced with finding a backup solution capable of archiving to tape about 200 gigs of a financials data in a 2 hour window. I originally looked into DLT8000 Jukeboxes with 2-4 drives but have recently discovered the new LTO drives. I am interested in knowing real world experiences with these drives as there has to be a catch. I mean there is a 3 fold performance increase in data transfers, two fold increase in tape capacity and a minimal price increase overall. With these drastic differences is there something I'm giving up with LTO over DLT or vice versa? Which backup applications are more geared to handling volume and integrate with Oracle RDBMS? Restoring speed is even more critical then backup speed so i'm curious about how these two drives compare and which applications are best geared for this much data on a nightly bases. Mind you there will also be about 500 gigs of data in an end-of-week backup as well."
This discussion has been archived. No new comments can be posted.

Backing Up 100 Gigs in an Hour? More

Backing Up 100 Gigs in an Hour?

Comments Filter:

  • Why... (Score:5, Insightful)

    by Stone Rhino ( 532581 ) <mparke@gm a i l.com> on Friday February 08, 2002 @12:17PM (#2974548) Homepage Journal
    Does the backup medium have to be tape? Hard drives are in fact more reliable than some tape, and would have a faster data transfer rate. A pair of hard drives hooked into a RAID array could backup over 200 GB of data and then be taken offsite just as easily as tape. Considering the fact that the drives would likely cost $400, tops, and could be reused many more times than tapes, I don't understand why people bother with tape anymore.

    • Easy - History. (Score:5, Informative)

      by tdyson ( 530675 ) on Friday February 08, 2002 @01:03PM (#2974864) Homepage
      Backing to a stagging array and then moving to tape is a great idea. Backing up to HD means you only get to keep the last couple of back ups. The reason to use tape is to go way back in time. With a good grand father-father-son rotation, you can keep a lot of history with a reasonable amount of tapes. I have backups dating to my first week at this company several years ago. I can raise the dead from almost any 2 week checkpoint since then. I have had requests for e-mail that has been gone for 6 months and documents that have gone for more than 18 months. It makes you look really good to just smile a friendly smile when somebody asks sheepishly, "I need a file from an employee who quit last Easter. I'm not really sure what it was called, but I think I know where it was on the network. Can you help me?"
      • Good point. Also remember though that hard drives have many more moving parts than tapes. HDDs may be more reliable than tapes sitting on a rack, but not while being carried/driven off site and back frequently.

    • Re:Why... (Score:5, Funny)

      by duffbeer703 ( 177751 ) on Friday February 08, 2002 @01:10PM (#2974906)
      Hard drives may be more reliable than tapes, but when the server room has water spewing from the AC and your controllers short out, guess what?

      Your "backups" are toast.

      Floods, tornadoes, fires, etc happen. Sometimes people fly planes into buildings. When that happens, tapes are the only thing that keeps your business in business.

      • Re:Why... (Score:5, Informative)

        by foobar104 ( 206452 ) on Friday February 08, 2002 @02:07PM (#2975244) Journal
        Floods, tornadoes, fires, etc happen. Sometimes people fly planes into buildings. When that happens, tapes are the only thing that keeps your business in business.

        I know this is completely off topic, but sometimes tape just isn't cost effective, particularly when you figure in the costs of manually storing and maintaining a library of data tapes in a vault somewhere. (Most of that cost is in head count: you have to pay somebody to do that work, and that's not a $19,000 a year job.)

        We're presently doing the cost analysis on a kind of radical idea. We're storing many terabytes of data in a data center in San Jose, California. The data center is as good as it can be, but there's still the danger (however unlikely) of earthquake or some other drastic event.

        Rather than trying to back everything up to data tape, we've gotten pricing from a telco on a dark fiber link between the San Jose data center and another data center somewhere in Colorado-- can't remember where. Since we're already putting an HDS 9960 in the San Jose data center, we can put an identical one in Colorado and use the 9960's internal "NanoCopy" software to keep them in sync.

        Believe it or not, it's working out to be more cost effective. One of the big reasons is that keeping that much tape on-line in a data center would require a StorageTek PowderHorn silo, and data center floor space is expensive. The difference in cost between the floor space and the dark fiber is so small that they cancel out.

        Like I said, I realize this is light-years away from what the poster was originally asking about, but it's kinda neat nonetheless.
        • There's some significant risk differences you may want to consider. First, you only have a single generation of backup. If someone totally screws the data, it will get synced over totally screwed, and your company may be out of business. The only way to avoid the risk is a long retention period with multiple generations on an off-site rotation. The other is the single failure point of the telecom line. If it goes down for a couple of days so does your backup. This is probably acceptable risk for most applications. Tape is still far from dead. The bigger you scale up, the lower the cost of ownership of tapes become. For one thing, you've got to power all those hard drives (and cool them), and I'm not aware of any hard drive auto changers. Tapes are also more durable for shipping. You could actually crush a tape cartridge, wind it back into another cartridge, and most likely be able to read it. Recovering data from broken hard drives is very expensive.
          • First, you only have a single generation of backup.

            The system we're talking about is itself an archive. It doesn't get backed up, per se; it just gets replicated.

            Given that our archive is going to be about 80 TB when installed, your "multiple generations on an off-site rotation" idea, besides being totally unnecessary, is frankly impractical in the extreme.

            The other is the single failure point of the telecom line.

            At about US$53,000 per month for dark fibre, the penalty clauses in the contract with the telco are such that if the connection goes down for more than one minute, we get paid so much money that we can pay off our penalty clauses and, believe it or not, actually make a small profit.

            You make a good point about tape, but it's not really relevant to our situation. For instance, we could use a StorageTek Powderhorn LSM to replicate our archive, but we'd either have to put on staff to manage thousands of 9840 tapes off line (the LSM only holds 6,000 tapes) or buy multiple LSMs. The amount of floor space that would require would cost us much more than the dark fibre.
            • You may have a special case where point in time recoverability is not necessary. Every large application I've worked with requires it. It becomes especially important anytime a new releases is being deployed. A bad query can muck up your entire database and not be noticed until weeks later.

      • Re:Why... (Score:3, Insightful)

        by sharkey ( 16670 )
        Wouldn't your tape drives be toast, too?

      • Off Site Backup (Score:3, Insightful)

        by Bryan Andersen ( 16514 )
        Hard drives may be more reliable than tapes, but when the server room has water spewing from the AC and your controllers short out, guess what?

        Your "backups" are toast.

        Floods, tornadoes, fires, etc happen. Sometimes people fly planes into buildings. When that happens, tapes are the only thing that keeps your business in business.

        No, actually it is off site backups that save your ass. All the tapes in the world won't save your ass unless you have carried backup sets off site.

        Off Site Backups can be done with harddisks too. The main advantage of tapes it they are usually less fragile than hard disks, but the costs of the tapes for some large capacity tape backup systems are higher per MB than the multi GB consumer IDE disks and they don't provide random access.

        An idea I had for backups was to have a system be a mirror for the main disks. As the day went on it would mirror all the changes to the main file server. At 6PM or so (end of busisness day plus an hour) the current DB after image file would be copied to the mirror and mirror would be broken, the disks pulled, and a set from the week before installed. The mirror would then be restarted bringing the old backups up to date. The removed set would then go home as the current offsite backup. Tape and DB backups would happen as normal. The DB backup would be written to a partition on the disk set. I would think this become infeasable if one has to backup more than 4 to 8 disks worth. At this point that could be more than one TB. There would be a set of disks for each day of the week. Weekly tape backups would be the long term archive, while the disk sets would be the offsite backup.

      • Re:Why... (Score:3, Insightful)

        by Perdo ( 151843 )
        You can move a hardrive offsite just as easy as a tape. the difference being tapes are more expencive. 200 gig tape starts at $3000. that same $3000 dollars spent on hard drives will get you TWO TERABYTES of more reliable, faster storage.

    • Re:Why... (Score:3, Informative)

      by cybrthng ( 22291 )
      Yes, medium has to be tape. We have to follow standards set upon us by governement and other standards bodies.

      The safety deposit box we use for month end tapes is just the right size for 3 rows of 12 tapes as well :)

  • is removable necessary? (Score:4, Insightful)

    by stilwebm ( 129567 ) on Friday February 08, 2002 @12:20PM (#2974566)
    You never mentioned it, so I thought I'd ask. If you don't need something easily removable, you can still have the data backed up to the other side of the data center, or even possibly they other side of the campus. With a storge array on a fibre loop you can back data up hourly, all 100GB in a full backup. Even a Gbit ethernet link could do this in under an hour, provided the link is not shared too much. Then you could run daily or twice daily tape backups off of that archive to send to you offsite safe archive location.
  • We use an Exabyte X80 with 2 mammoth2 drives (it's upgradable to 8!), and fully loaded it backs up ~1gb / sec. Not very cheap, but very fast.

  • Quite a feat (Score:2, Insightful)

    by FreeLinux ( 555387 )
    You certainly have quite a challenge here. Some of the newer LTO drives can, theoretically, achieve 55GB per hour transfer rates but, realistically you will get far less. Using an array of drives will allow you to increase this performance but definitely not as much as one would like to think. With a two drive array, most people expect to double their performance but, a 30% increase is likely as much as you will see. That said, it would require a 4 or 5 drive array in order to achieve the theoretical throughput that you desire. This all assumes that your server hardware and OS and backup software can in fact feed the array fast enough.

    I'm afraid that your only realistic options are to either get a larger window, which is probably unlikely, or perform live backups and bear the performance degradation during that time. The only other alternative that I can think of would be to mirror your data to secondary disk based storage and then backup the secondary storage off line. Any which way, I'd be amazed if you got 100 Gig an hour.

  • I'm only tackling the restores part of the question here... typically, speed or a lack thereof in restores is a function of your backup software. If I restore a library on one of my AS/400s (using IBM's OS/400 builtin backup utilities, which are not very bright), it takes a few hours to read the tape and seconds to restore it (single unit LTO). However, if I can give a tape id in addition to the library name, the restore takes seconds since the LTO is really very fast, particularly if it knows the tape segment where the file is stored.
  • First question is tape the medium you have to move the data to, or could it be staged to disk then to tape. Also what kind of system is the dat stored on now? Is it a disk array or JBOD? Who is the manufacture of the disk system? EMC, Hitachi, HP, and SUN all offer ways to create an internal point in time disk copy of production data. This copy can be mounted on a system then backed up to tape. The copy doesn't have to be mounted on the original system the data was on, just the same type for mounting purposes. The next question is where will the tape be attached? Is it going to be attached to the system via SCSI, or Fibre Channel? A single 20mb/s tape drive in a best case might be able to do 36Gb an hour. But the average I've seen has been about 80% for a substanined transfer rate do to tape, and system overhead. So your only looking at 28Gb per hour. This does not include the time it takes for the library to exchange the tape. You would need at least 4 tapes drives to even be able to back the system up in this time frame, but your recovery would suffer along with you system during backup. The only way to get a the best performance out of a tape system is to strip you data streams, and have multiple data streams. All the major backup software vedors do this (HP, Veritas, Legato). The problem is the recovery hurts as most will only recover one stream at a time. So if you backup four stream at a time, your recovery will be 4x the backup time. I guess that is basic overview of DR. I can write for hours on this...
    • EMC, Hitachi, HP, and SUN all offer ways to create an internal point in time disk copy of production data.

      Off topic (again), but FYI the HP and Sun storage solutions are OEM'd by Hitachi.

      The Sun StorEdge 9910 and StorEdge 9960 are HDS 9910 and 9960 with different skins on them. Likewise, the HP XP48 is an HDS 9910 and the XP512 is the HDS 9960, so named for the number of disks in each (48 and 512).
  • What we did to reduce backup windows was create a large IDE array. We got 13 100gb IDE drives together and we run all our backups to that. Have about 900gb of space in a raid 5 array. Once the data is to disk, we have a seperate gigabit nic with a crossover cable running to a second machine with two HP DAT40x6is which take the data off the disk and dump it to take. We have extreamly small backup windows.

    The problem with this is that to restore a file older then a week or so (once it's off the disk array) we need to restore the whole backup set which can take hours. Were useing Backupexec on Windows 2000 so I'm sure whatever software you guys are useing is more flexable.

    This might offer a cheap solution to your problem, good luck!
    • What kind of performance do you get throwing it over the gigabit network? How long does it take you to dump that much data?

      I put in a PO to have a seperate server doing the backups through a gigabit network, but not necessarily to dump the data locally. Just to offload the cpu from the production environment since in reality it is a hot backup i'm working with and the 2 hour window is the only time i have where there are not batch processes running that could really kill my backup times.

      Boy do i hate 24/7 systems :) hehe

  • AIT/2/3/4 tape libraries. (Score:2, Interesting)

    by Zurk ( 37028 )
    my high end spectralogic AIT-2 tape library can handle it easily. newer AIT-3/4 should be able to do it too. i can backup around 2.75TB with hardware compression before it runs outta space (30 catridges each with around 50GB uncompressed).
    • What type of performance do you get on AIT-2? (gig per hour)
      • I've been responsible for quite a few SpectraLogic 10000 units (2 or 4 drives, 24 or 40 slots). The hardware is high quality and the support is top-notch, put you pay for those things. 40GB/hr is about what you can expect from each drive, but that can varely widely based on the compression ratio. The AIT specs are dead-on in this regard.

        Striping tapes is a bad idea, but if the data is readily divisible (not a single 200GB dbdump) you would be fine backing up one portion per drive in a quad AIT-2 or dual AIT-3 configuration, with room to grow.

        If you can't divide the data, backing up to disk and streaming to tape later is the best option. Most client-server backup software will allow you to establish a disk-based target and subsequently copy the data to tape.

        -Bryce
      • i get around 120 gigs/hr *sustained* (that includes the time to change AIT2 tapes using the robotic arm) using 4 sony AIT-2 tape drives (i.e. saturating all four drive slots on my AIT-2 library) while writing 2.75TB in one go with hardware compression on using amanda for the backup software. peak writing capacity is well over 160Gigs per hour or more with hardware compression switched off.
        each ait-2 tape is the size of a small pack of cigarettes and really cheap. the tape library itself with 30 tapes loaded can be carried fairly easily under your arm if you want to go lugging it around as a portable device. its about as heavy as a computer monitor and about as bulky.
        get ait-3/4 which will double or quadriple the capacity per tape while retaining the same tape size if you have the budget for it. AIT3/4 should also give you around 200 gigs per hour or more.
        BTW, i use an sun E420R to whack data across to the library.

  • Mirrors and Backups (Score:2, Insightful)

    by acaird ( 530225 )
    Although this doesn't address your need for fast restores, one method of doing this that you likely already know, is to mirror all of the data, break the mirror, backup the static half of the mirror for as long as you like, bring the mirror back together, and let the RAID software worry about the sync'ing of the data (Veritas can do this, I've done it using Veritas for disk mgmt and Legato for backups.) This means your backup window, from the perspective of the application, is 0, and from the perspective of the backsups nearly as long as you need.

    However, you aren't the first person to have this problem, and I'm sure Oracle as solved this problem. If it's as important as you say, I would pay them for this knowledge.

  • Instead of trying to get a tape system that will back up in 2 hours, why not get a 'SAN' system that supports "snapshots" and then back up the "snapshot" on a tape medium that could take many more hours?

    Well, cost I suppose, but the 2 hour window will probably shrink over time, while the 200GB will turn into 250 and 300GB, so you always end up chasing the most expensive tape technologies to keep up.

    As far as the DLT vs. LTO question, I don't know much about LTO in production use, but DLT is so widely used and trusted it would seem to have more of an incumbent status as the technology of choice. I ended up using AIT 50/100GB tapes -- cheaper media, faster than DLT, smaller tapes (easier to carry lots off-site), larger 'jukebox' designs available, and higher capacity (at the time I made that decision). It's worked well and I've never regretted it.
    • We try and use RMAN for nightly backups, and it doesn't work well with the filesystem snapshot concept.

      We do this on week end/month end backups though as we can shutdown the applications and have a clean backup.
      • With standard backup software like Sun's (rebranded Legato) Solstice Backup you can also specify a filesystem to back up to as well as tape's, so you might be able to use an extra RAID for quick backup's then use the tape to back that up in a larger window. Filesystem snapshotting (Sun's Instant Image) would work fine if you can shutdown the database for a sec. I'm in the same pickle btw. Oracle Financials, Sun, tape backup, RMAN, etc. I'm going to end up doing just tape (single drive L20 DLT8K) we don't have a window nearly as small as yours though. I'm currently getting 10GB/hr via NETWORK backup, which isn't half bad. But I'm also going to have a extra RAID for Redo logs (mirrored drive) and a RAID-5 for whatever snapshotting I need.
  • The Database data is on an EMC system connected to some Sun machines (E3000's and V880's). I have BCV volumes setup to mirror the data, but the problem is the mirror is already used to establish database replication and clone the database to a development instance. Not enough time during the night to do this, backup and do nightly transactions and whot not.

    I can add 32 more drives into the EMC and setup an addition BCV pair to mirror on and then backup from the broken mirror, but the cost is just about the same as buying a 4 drive, 20 slot changer.

    This is financial data responsible for nearly a billion dollars of revenue a year so using extra disks to offload before the backups are an option, simply not using a tape solution isn't an option.

    hope that helps heh
    • Um....if you are ready have a Symm/or Clariion...(you didn't specify) why not use EMC's backup solutions, such as EDM...?

      Just a thought....
    • Did you try to get Veritas/Sun to loan you copy of either the Netbackup with snap and Oracle agent to test ? This might do the trick, however what you do with the snapshot is pretty important. Backing up off the original server will impact peformance, especially for 100 GB. You may also want Sun to ante up the Instant Image product, you can then start the backup through another server.
    • Yup, as mentioned, you should talk with your EMC sales rep about their EMC Data Manager [EDM [emc.com]] product.

      This is an Ask EMC, not an Ask Slashdot question!

    • I like the idea of using a RAID set as a intermediate step here. You need to sustain an average transfer rate of about 28 MB/sec the entire 60 minutes. Most jukeboxes I have looked at take 30sec or so to swap out a tape and set up the new one for writing and tape drives take a few seconds to ramp up to their full data rate and drop off from time to time to position etc. This is going to cost you some time and up the peak rate of that transfer number so it can average out to 28MB/sec.

      I have used drives from AMPEX corp that sustain 15MB/sec and store 300GB per tape. That was 2 years ago and they have a newer product out now that was supposed to double that rate but I have no first hand experience with those in the real world.

      Since we are talking about an Oracle database here I would also look at the plug-in modules for some of the backup software products. I know Legato NetWorker has such a module and will also allow disk storage to be used. I use NetWorker here to backup about 1TB/Wk. and it gets the job done. It should (if the module works as advertised) be able to quickly backup the database to RAID, then clone those backups to tape.
  • I'm backing up 100 desktops, a 10 HPUX workstation lab, and 4 NT server backoffice.

    The AutoLoader is SPOT ON. I have the LVD version of the library ( they make a fiberchannel ), and am running it on a 32bit pci U2W tekram scsi card.

    Actually, real world throughput ( from the spooling disks, not the network ) is 13MB/sec. The spool is two ATA100 100MB drives, striped together, each on it's own channel from a single promise controller, using the 'md' driver.

    The autoloader delivers. They are stackable, and the robot/loader is capable of inverting and passing vertically around the stack. Love HP hardware engineers.

    Here is last nights statistics: DAMNED LAMENESS FILTER!!! Heavily edited now:

    Run Time hours:min 6:02

    Dump Time hours:min 19:56 4:59 14:57
    Output Size meg 31301.4 19943.0 11358.4
    Original Size meg 55924.1 34698.9 21225.2
    Avg Compressed Size % 56.0 57.5 53.5 level:disks
    Filesystems Dumped 83 1 82 1:82
    Avg Dump Rate kb/s 446.7 1139.5 216.1
    Tape Time hours:min 0:44 0:24 0:20
    Tape Size meg 31305.4 19943.0 11362.4
    Tape Used % 30.9 19.7 11.2 level:disks
    Filesystems Taped 83 1 82 1:82
    Avg Tp Write Rate kb 12193.3 14329.8 9664.3
    That looks AWFULL now. Comeon guys, at the risk of some ascii art, and for the sake of some tables, could you allow the <pre> tags?
  • What are the real world numbers? I don't even see much info on usenet or anywhere other then typical sales bs.
    • I've installed an HP Ultrium 215 (the slow one!) at a client site, and it rocks.

      It backs up faster than data can be sent to it - there must be plenty of headroom on the faster models. The client was just using a bunch of intel servers though, no high powered SANs or anything to back up.

      No idea on reliability yet though - no probs so far (still less than a year). They get hot though, so don't cram them in a crowded server.

  • Not such a big deal... (Score:4, Insightful)

    by j.e.hahn ( 1014 ) on Friday February 08, 2002 @03:25PM (#2975779)
    It's by no means an easy feat, but the following should probably get you there.

    First, you need to consider how the data is getting to your backup server. This looks like a job for gig-e. (since you don't really want to run you DBs on the same machine as your backup server.) You should use multiple streams. (either break it into multiple smaller jobs or enable the multiple streams option in your backup software if it has one. Many do.) It's hard to flood even a 10base network with a single TCP/IP connection. (your bandwidth utilization decreases in inverse proportion with your latency. I forget the exact formula though.)

    Next there's how you're getting it to tape. I recommend running the backups to disk first if you can. This means you won't stall a network connection if you change tapes, or the like. But it does mean you need a lot of storage on the bkup server. Also, if that's a 2h from DB to tape window, this might not be useful. However, barring using a SAN and snapshots (or the like) your only other option is to go straight to tape.

    To go straight to tape you'll need at least 6 DLT drives, assuming you can keep the tape streaming and get 6Mbytes/s, and you balance them across a wide enough SCSI and PCI bus(or whatever system bus you choose) This will give you N+1 redundancy and meet your bandwidth requirement of 28Mbytes/s.

    As for the LTO/DLT trade off. We're moving to an LTO solution where I work, and it generally seems to be the way to go. It's worth evaluating, but I don't think your choice of tape either way should be your restricting factor. And there is something to be said for the reliability of DLT.


    • To go straight to tape you'll need at least 6 DLT drives, assuming you can keep the tape streaming and get 6Mbytes/s, and you balance them across a wide enough SCSI and PCI bus
      One SuperDLT drive is capable of 15MB/s (~50GB/hr) with mildly compressable data (a typical DB meets that requirement easily). A pair of MSL5026 [compaq.com] or similar, 2 drives & one SCSI adapter per library would be more than enough.

      cheers,
      mike

  • A different take on the HD idea (Score:5, Insightful)

    by sigemund ( 122744 ) on Friday February 08, 2002 @03:47PM (#2975947)
    By no means do I know a whole lot about backup technologies or any of that, but I do have a suggestion that kindof takes a different angle on the hard drive idea.

    I understand that you would want and need to keep the data off-site on tape (requirement). However, getting that transfer rate is going to be difficult. Perhaps you could do something like this:

    Use the hard drive backup (SCSI RAID perhaps?) idea to backup the data quickly and reliably. THEN, you've got it backed up in your time limit. Now, you can back up that back up with a tape, but you don't have the incredible time requirement. Get it?

    Concept:

    Original Data on Hard Drive
    --> Back it up onto a separate Hard Drive within the time limit
    --> Now, back up that hard drive that has just backed up the original. You have a backup done already, so you've met the time needed. Now, you can back it up with tape or whatever without having to do it within such a short amount of time. You can use the technology you desire to back up the hard drive copy while the original data drive keeps working.

    Then to restore, you can do it from whatever the removable media is.

    Again, I don't know a ton about this, but it's just a thought of another way to accomplish this.
  • Depending on your backup and recovery needs you may want to look at using a SAN or NAS that has the ability to do snapshots or point in time copies of your data. All you'd need to do then is pause or stop your db engine, snap the filesystem and restart the db engine. This one approch you can take to expand your backup window. This can also let you take advantage of resources that may not be available during your normal backup window. For example, we have a storage tek tape silo that is heavily used at night by our legacy systems for various production jobs. That same silo is idle much of the day time hours. We have a NAS that supports snapshotting(a netapp filer) so we can pause our db engine at 2AM, snaphot the filesystem, and restart the db engine. Then the next day, when we have lots of available time on the silo we can dump it to tape. This also gives us quick restore capability, if we need it. For example, if the DBA makes a boo-boo and needs to restore the db they can stop the db engine, rollback to the snapshot from 2AM and restart the db engine. This takes very little time compared to doing a restore from tape. But, don't just snapshot the data because if the building burns down you'd be SOL. If you really need quick restore times and tape is still too slow you can look at replicated db's or replicated file systems. If you give your friendly local network appliance, veritas, IBM, sun, hp, compaq, EMC, or auspex sales person a call they would probably be more than happy to talk with you about various products that support some form of snapshotting, rollback, replication, clustering, etc. You may also be able to cut down the amount of data that you need to backup nightly. A lot of times in large databases there is tons of static data, for example if you have a a large GIS database with lots of satellite imagery you may find that you only need to backup the imagery quarterly or yearly instead of nightly.
    • let me echo this point - get a good SAN device (emc being the best choice imho), and let it do point in time snapshots of your DBMS - then mount the PIT image as a R/O device, and you can take as long as you want to back it up.
  • This may seem ultra low tech, but I've been able to get away with backing up extremely large ammounts of data using plain-jane removable hard drives. In one case I built an entire "backup server" who's only purpose in life was to have mass ammounts of removable storage and perform backups. Instead of shuffling tapes around we just shuffled (slightly larger) hard drives around.

    We thought of using SCSI, but the cost/MB was to high. The server had a Promise UltraATA 100 card with four 40GB WD hard drives and a single 10GB WD hard drive on the on-board IDE for the op system. The case had 5 external bays, four housing the sleds for the 40GB drives and one held the CD-Rom. The 40GB drives were whipped up in RAID to create a single 120GB volume (last drive for parity)

    I realize it's not really an enterprise class sollution, but it provided the fastest backup and restores for our money.
  • Copy the data across a dedicted network connection to a large disk array. Arrange the backups so you have sequential backups - Day1,day2,day3 etc. on disk

    Back the files from the disk array to tape.

    You should come in under your window w/ the net xfer. You have all day to write tapes.

  • Use a striping RAID array for the backup, then copy to tape.
  • If your data is stowed away on high-end storage, chances are that you can easily create snapshots of the data. (eg. EMC SnapView)
    You can use snapshots to back-up your data at a "reasonable" pace.

    If near instant-restores are a necessity, consider creating an off-site BVC using FC/AL over fiber.

  • it's straight forward (Score:4, Informative)

    by tachijuan ( 557826 ) on Saturday February 09, 2002 @06:01PM (#2980285) Homepage
    Here's a copy of the email I sent to cybrthg:

    Storage is all that I do for a living. Here's a quick summary of how you could do it:

    1) First you have to make sure that the drives you have your data on are going to be able to give you the read rates that you need. I highly suggest that you go with a raid array of some sort. Preferrably one with substantial cache in front of it. The raid controller should be smart enough to do what's called "read ahead" caching. That reduces the read miss ratio and speeds up sequential (i.e. backup) applications tremendously.
    2) Get whatever tape drive you choose. Base your decision on the speed of the device - use only the native performance. LTO is either 15mb/s or 16mb/s depending on whos drive you get. It is safe to assume that you will get about 1.2x compression. So for LTO that would get you either 18mb/s or 19.2mb/s. Assuming you get the 15mb/s drive you can realistically expect to get ~64GB/hr per drive.
    3) You have to get some sort of advanced backup package to support those rates. I would suggest that you go with either Veritas NetBackup, Legato Networker, SyncSort Backup Express, or any of the enterprise class products. Don't go with cheap software - in general they do not have the performance coding necessary to move data at very high rates efficiently. This is a hard choice, but if you stick with the three I told you, you should do fine.
    4) Get a library that can handle several drives, so that you can use them in parallel.
    5) Put each tape drive on a separate scsi bus, or if you go with fibre channel put at most 3 drives on the channel. There's a ton of way of architecting this side of the house, but in general if you stick with those numbers, you should be fine.
    6) Try not to send data over the network, even with GigE, the effective rates are going to be drastically slower than those of direct attached devices. GigE also severely impacts the server - tcp/ip overhead is a bear for high throughput environments. There are ways around this, but that ouside the scope of this email :-).

    That should do it for you for the traditional backup methodology. There are other ways of doing backups - making mirrors that you can split of. Taking snapshots..... There are a ton of products that can help you on this. Some of them are software based packages that sit on the server with the data. Some of them are hardware/software devices that sit on a SAN or a NAS. Again, going into this is quite lengthy, but it can be done. I have a customer for which we are doing over 1TB/hr backups using a combination of lots of tools. You problem is quite a bit simpler.

  • Looking at what your setup is, I can say that you really have three options:

    • Straight to Tape - since you're running an EMC, I'm going to assume it's hooked up to the Suns via FiberChannel in a SAN. In order to get the throughput you need, you're really going to need about a 4-drive LTO or DLT library (since you can't stream all hour, there are going to be short windows of low-performance). And you have to leave room for growth of the data. The library should be dual-looped FC to get full performance.
    • Local Snapshot w/ offline-tape - by this, I mean you will have to add capacity to the EMC to do a local snapshot, then backup the snapshot at a later time. For this, you can get away with a 1-drive LTO/DLT, but it really still should be FC on the SAN.
    • Other Snapshot w/ offline-tape - if adding the snapshot capabilities to the EMC are too expensive (and that's a good possibility), consider adding something like a A1000 to the network (for instance, on your backup host). Backup the SAN data to the A1000, then use a locally-attached DLT/LTO drive to backup the A1000.


    Honestly, I would prefer the last option. You already need a backup host machine with FC connectivity and enterprise backup software in any of the scenarios. Adding a 100GB A1000 (or D1000) to this is only going to run $10k or so, and you don't need a massively expensive FC tape library, or the EMC snapshot software. It'll grow with your backup needs, and is relatively inexpensive. And you can keep last nights (or a couple of nights) worth of backups on the A1000 for fast recovery. Disks in an A1000 are much cheaper than for an EMC.

    I'd go with something like the Sun L20 mid-size (2 LTO drives, 20 cartridges, 2TB total) with a 200GB A1000. Sun list is about $35k for these together.

    -Erik

  • I didn't read this while scanning he other messages, so....

    1. Backup to an separate system, onto its hard drives.

    2. Tape backup the separate system.

    You get speed, and your tape requirement.

  • I see a number of posts extolling the virtues of using hard drives as backup media:

    *Fast!
    *Cheap!
    *Easy!

    Unfortunately,

    *Reliable!

    isn't something you're going to get. Also, when you consider that you may want to be able to restore something that was deleted, say 6 months or a year ago, your media costs begin to outweigh your equipment costs. Also, a hard drive is going to be physically larger than a tape holding the same amount of data, requiring more expensive off-site storage. You do take your backups offsite, don't you? What about a flood, fire, or (gasp) terrorist attack? What about a break-in?

    Another reason to not depend solely on hard drives for backup: the shelf-life of a tape is much longer than a hard drive. Fifty years from now, you'll still be able to read today's backup on tape, but the mechanics of a HDD used for backup (even if it hasn't been used) may be all goobered up rendering the drive DEAD. I won't go into handling considerations, except to tell you what you know already: hard drives are fragile.

    That said, I like the idea of setting up a mirror server which updates from the master server, then running the backup on the mirror server. This'll increase your window and reduce the load on the master server.

    Lastly, make sure you understand the difference between full, incremental, and differential backups. Use them to your advantage to balance price, speed, load, and storage (where you're gonna store your tapes) considerations.

  • ...when you get two simultaneous disk failures on a RAID-5 array. In other words, the data was toast. Happened to one of my clients around a month ago. Thankfully, they monitor their backups, making sure every one is good and taking action if a backup fails. They lost only one day's worth of work.

    I know people who slack off when it comes to backups, because they've got redundant drives, after all. They seem to believe that they never accidently delete files, and it's not that much work to recreate those quarterly reports. They don't realize just how much work they do on their computers, and how hard it is to retrieve that thought that came to them as they were typing up that letter to their congressmen.

    Just today, one of my clients had an IDE drive on a RAID-0/1 array fail. The controller (two channels, four drives) misreported which drive it was. Now, it only shows one drive in a four drive array as being a member. Here's hoping a new controller which will arrive tomorrow will allow us to rescue the data.

    No, they don't have a backup for ~100GB of data. Fools. I guess they've got a few spare months kicking around to recreate several hundred thousand pages of digital documents.

    A RAID array is great for performance gains. In a production environment, it'll guarantee uptime while a bad drive is replaced or until the system can be taken offline. Don't trust a RAID controller that puts two IDE drives on the same channel; sometimes, a failed drive will prevent the system from being able to access the "good" drive on the same channel, bringing down your system. If reliablilty and uptime are important and can be measured in $$$, don't even THINK about IDE RAID solutions. The increased support costs for IDE are more than the increased hardware costs for SCSI.
  • Well not mine, but I get to use it at work. It is an IBM Ultrium 3580. Each tape holds 100GB. The specs say it holds 200GB compressed, but all of my data was JPEG images, which I don't think can be compressed anymore. Depending on the server load, the times I have noticed are approximately 1hr 15min. per Gig. This is my best case scenario.

    The data being backed up was coming off a EXP300 external RAID array (again, IBM), conencted via Ultra160 SCSI.

    The only bad thing I can say about this drive is that we are on our second unit. A tape got jammed into the first one, and was destroyed in the process. IBM was good about getting a replacement here, however.
    • >1hr 15min. per Gig

      Umm, I hope this is a typo. We're talking about an LTO drive?

      I get about 4.5G in an hour on an OS/2 server running a DDS3 DAT drive, plus about the same for a verify pass. I don't consider this an enterprise level backup (It's for a pretty small business.)

      Using 9840 drives over FC at the data center I used to work at, we got 10M/s, best case. 9840 drives are pretty fast, but lots slower than LTO. I'd hope for at least double what the 9840 did.
      • I get about 4.5G in an hour on an OS/2 server running a DDS3 DAT drive

        It was a typo, I meant to say 100Gb in approx 1h 15min.

        My only experience with this drive has been backing up hundreds of gigs of jpegs. I haven't actually had a chance to use it to backup more compressable data (if such a thing exists?)
  • The "ask slashdot" law of answers to backup-related questions:

    Any discussion about backup solutions of ANY kind will turn into a flamewar about whether current, huge, cheap hard disks are better for backup than tapes.

  • At where I work, we deal with ridiculously large databases (SQL and Oracle), and we need to make sure we always have up-to-date backups of those databases and associated other files at all times.

    We do have an EMC, but we don't use it for backups (GASP!). Instead we use it as fast,fast,fast storage LUNS where our databases live. Once EMC's snapshot software catches up to us, we'll snapshot a LUN and then spool it out to tape.

    In the meantime, depending on the size and activity of the database being backed up, we'll either to a database export out to NAS (Maxtor 4400s) and then have our Veritas NetBack server with an ATL autloader (2 LTO drives and holds 20 tapes) back up that, or we'll do a backup directly from Oracle, SQL or the server itself.

    Backing up to the NAS units works well for our large databases, because we've had issues with NetBack aborting backup jobs when doing direct backups because it wasn't finished by the time the backup window closed.
  • Being unbiased and all, Heh. why dont you examine the NearStore box? http://www.netapp.com/news/press/2001/news_rel_200 11210a.html It's an inexpensive near-line and tape consolidation solution for existing systems, and if you had a network appliance filer you could just use snapshots which takes 3 seconds, without doubling your disk space requirements and then you can take your merry time putting it to tape. http://www.netapp.com/tech_library/3043.html
  • I have had a bit of experience with IBM Ultrium LTO drives on AIX in environments ranging from SCSI attached to Fibre Switched. They can backup in my experience 60-70GB per hour but the restore takes 4 times as long as the backup time. The problem I am told is the back-hitching that the tape drive has to do in order to keep data going to the disk drives. These things will restore much faster than the system can handle so they have to stop, go back and resend. What you might have to do is look at other backup methodologies like incremental hot backups of your database like RMAN for Oracle (I assume that this is what we are talking about here). Combine this with the LTO and you should easily make your backup window.

    Cheers,

    makath99
    • Damn, I forgot to mention that Tivoli Storage Manager (TSM) integrates with Oracle, LTO libraries and Storage management really really well. It also lets you do LAN free backups over a SAN to speed thing up even more.

Slashdot Top Deals

The Tao is like a glob pattern: used but never used up. It is like the extern void: filled with infinite possibilities.

Close