Using USB Hard Drives For Disk Images? 31
anomaly asks: "I work for a fairly large company, and we're faced with the problem
of maintaining images of hard disks for the hardware we use in our environment. We use Norton's Ghost for this process. We'd like to use a USB hard disk (because both the desktops and notebooks have USB interfaces) to store hard disk images, but Ghost requires DOS drivers in order to work. Does anyone know of a way that we could store, edit, and organize hard disk images as well as potentially installing the disk image via USB, or over a network? I suppose ideally we'd have some reliable, cheap network attached storage device (preferably in the $200-$500 range) - reliable in that we want to do 0 maintenance on these babies - so that many different hardware images could be stored and reinstalled as necessary. Any ideas?"
SCSI *IS* cheap! Even by your "analsys" ... (Score:2)
First off, I think you've made a number of incorrect assumptions. My views are based on years of corporate experience, including PC rollouts. Please read my responses below. Understand that I am the only person who gave you an useable, DOS/real-mode solution. And it's not as expense as you think.
So are mine! You think I've been fired for buying SCSI all these years? More $$$ does NOT equal quality. I go through specific products below ... (and note that NONE say "Adaptec" -- been burnt by their crap too many times).
All of the cards I use have quite stable drivers. Of course when you buy something new, you shouldn't expect it to work. You should always wait ~6 months for the bugs to clear out. But when if you'd waited 5 years for good Adaptec Linux drivers, then you'd get quite irritated.
You can easily standardize on one SCSI chipset, the TekRam TRM-S1040 [tekramusa.com]:
We cannot afford to put a Zip, Jaz, CD-R/RW and DVD-RAM/RW drive on every PC in my office. Instead, we have one or more external ones and put a $15-20 TekRam DC-315U in each system. Works great! Also great for cloning when I don't want to hit my server/network too hard (in the middle of the day), let alone transfer loads of data between systems. In Linux, I can even load/unload the TekRam S1040 driver on-the-fly, flipping drives on/off various systems without a reboot/shutdown. It's _awesome_ bay-bee!
As far as other experiences, I recently had to chuck my Adaptec AHA-2940UW (AIC-7880) in my Linux server because it is a POS (in 6 years of using Adaptec on Linux, I have yet to have a good experience thanx to their non-direct support). The sucker refused to work properly with a new, $4,000 Exabyte Mammoth2 60/150GB tape drive (talk about "betting my company's business" on a SCSI card!). I replaced it with an $60 Advansys [advansys.com] (now owned by ConnectCom [connectcom.net]) chipset-based card:
But if you need faster still, Symbios Logic [symbioslogic.com] (now owned by LSI Logic [lsilogic.com]) is always faster and more ubiquious than Adaptec. So much so that Adaptec attempted to buy Symbios out (since they were kicking Adaptec's butt in the OEM and FibreChannel market). You'll be interested in the popular 53c895/1010-series:
And when it comes to hardware RAID, Adaptec is just NT/Netware-only. As such, I prefer DPT [dpt.com] or, better yet, StrongArm ASIC-powered Mylex [mylex.com] RAID controllers with broad OS support (and better performance too).
So what brand are you blindly putting your faith in? Eh?
And those IDE drives can be put in a $20-40 enclosure and made to work at 20MBps+, right? Not! When it comes to external (isn't that what we are talking about, eh?), IDE is a joke -- with slow as molassas USB (even in 12Mbps/1.5MBps "fast" mode) being the only option (although new ATAPI-to-FireWire bridges, like this Ultra33 one from Intito [initio.com], is changing that -- although it requires OEM firmware/programming). Plus we're back to the DOS/real-mode issue (even for FireWire). Only SCSI is "ready-to-go" external.
Now you can compare GB/$ all you want. You do NOT need the latest SCSI drives. Go with a late-model 9-18GB SCSI drive. I mean, how much storage do you need? We're only talking $100-200 for the drive, another $20-40 for the enclosure and another $10-30 for cabling and termination, max. You could do it for under $150, including cables and termination, if you pinch your pennies (and buy your stuff mail-order -- use Cyberguys [cyberguys.com] for SCSI cables/terminators). Plus, you must be looking at 7,200-10,000rpm RPM drives -- don't make the mistake of comparing 5,400rpm IDE drives to obviously much faster SCSI drives.
-- Bryan "TheBS" Smith
MultiCasting ? File server ? (Score:3)
May the
Multicasting :
You could create DOS boot disks with network drivers and TCP/IP then launch Ghost and do multicasting from a Ghost server which stores the images.
Ghost server runs on NT, I don't know if there are other versions available. Basically you tell it what image file to use and a few parameters (session name, number of clients...).
There you have your network attached device which stores the images. The hard part of it is getting boot disks that work with the NIC's you may have. This can be troublesome with laptops. Either have one disk for each NIC, or tweak and do a little programming to allow a selection during the boot process (I got one such disk that Does 3Com 5x9 and 9x5).
The same applies for IP addressing : you could have a disk that asks you the IP address and subnet you want to work with (Again, I got a disk that does that, it uses a
The most important benefit of this multicasting thing : network bandwidth is not wasted sending the same data many time over the network... You can Duplicate a disk on n machines at the same time with only one multicasting session.
Another solution is to create a boot disk with TCP/IP or another NetBIOS enabled protocol (NetBEUI or IPX/SPX) and connect to a network share on a NetBIOS server (could well be a Linux/Samba box), on which you have your images. Down side : each copy requires the data to be sent from the server to the station, thus using a lot of bandwidth.
As for USB, I have no idea how to configure these things under DOS...
USB under dos (Score:3)
WE USE SCSI.... USB and even ethernet is SLOWWWW!! (Score:1)
When we need to GEN up a box from a disk image we would slap in the appropriate network-able boot disk and then load the image off the private tech network. (Two networks one for testing, one for the accountants to use) Took about 2 hours to load an average disk image.
Now most of the accountants that we support have gotten laptops, so we got two 18GB SCSI drives and two PCMCIA SCSI adaptors. Now to gen up a laptop we just boot off the floppy with PCMCIA SCSI support and GEN up from an image in 15-20 minutes... saves us time and we don't have to lug the box 1/4 mile to our new office. Go with SCSI you will be much happier. A good PCI SCSI card (Adaptec) is pretty in expensive, takes minutes to install and is wayyyyy faster and more comptible than friggin USB.
Microsoft OS's (Score:1)
< A HREF="http://freshmeat.net/projects/mkcdrec/"> http://freshmeat.net/projects/mkcdrec/</A>
this might work I have not checked.
-- Tim
2 *hours*??? (Score:1)
I've said it before ... SCSI!!! (Score:3)
I've said it before, and I'll say it again. At only $15-20 extra per machine, you should be buying systems with SCSI cards. After that, you can plug in external SCSI devices, like hard drives, that are 20x faster than USB with full DOS/real-mode compatibility.
For more information, see this previous /. thread [slashdot.org] on USB hard drives (and just how freak'n slow they are). Look for my post in the middle (search for SCSI).
Otherwise, does anyone know if IEEE1394 FireWire cards have ASPI/real-mode drivers? If so, that might be another good option. But since most PCs still do not come with FireWire, and FireWire cards are more expensive than SCSI (and the drives aren't much cheaper, although Maxtor is trying to change that), it may not be better.
Either way, avoid USB since it is a slow pig. SCSI or FireWire is a much, much better angle that is 20x faster!
-- Bryan "TheBS" Smith
Re:Specifics of Ghost vs dd? (Score:2)
Another thing is that it's possible to edit ghost images - something I don't think is possible with dd.
Mount via a loopback? Would probably have to decompress the image first, but as long as there's a supported FS in the image, shouldn't it work?
--
Why USB drives? (Score:3)
More comments... (Score:1)
Well, when I say server, it can be a laptop with a large disk (even an external disc). This is not going to be a bottleneck anyway. We often do it with an old Pentium 75 running NT server with 80 Mbs of RAM and some old and slow disk. By the way I said it's an NT soft, but I'm pretty sure it's a plain Win32 thing and it can be run on Win9x as well... (flamers, hit me if you can...)
But you sure want a fast enough network. I've always used this technique to dump images on disks in labs. Yesterday though, I dumped an image to a server on another site of our network (I guess you'd call it a MAN). I don't remember what's the speed of that link, but it was during working hours and it took 2.5 hours for about 1.5 gig. Not quite fast !
I'd sure recommend 100 Mbps where available, and possibly switched if not using the multicasting thing. I'm not a TCP/IP and ethernet wizz and I'm not sure how well these multicasted packets will react throuhg a switch or a router.
Or go fiber if you got the bucks !
The SCSI idea is not bad either if the target computers have not been ordered/paid for/installed on site yet.
USB sucks badly, I'd try avoiding that solution if I were you...
The CD solution is the easiest to set up and is perfect when all you've got to do is quickly get a workstation back to work on a remote site with a smaller infrastructure... That's the way we go with our techies on sites.
Re:I've said it before ... SCSI!!! (Score:2)
Overall SCSI wouldn't be a terrible thing either. However it's not a very standard interface in desktop class machines. You would have to go out of your way to see to it that your desktop machines were fitted with a scsi port. Your laptop machines will likely never have an external scsi port, but you could keep a PCMCIA SCSI card with your drive for use in the laptop machines.
_____________
Re:2 *hours*??? (Score:1)
Specifics of Ghost vs dd? (Score:2)
Assuming Ghost doesn't do anything really funky to the disk images, perhaps a modified tomsrtbt (2.2.18 kernel) or somesuch would be a solution. You'd also theoretically be able to access Zip (etc) drives to pull off images as well.
--
Can you read ? (Score:1)
Re:MultiCasting ? File server ? (Score:1)
backing up "deleted" files.. and compression (Score:2)
I'd like to back up every bit of several HDs I have. (I do not intend on forensic recovery... no new hardware) I THINK there is a way to use dd for this, but I'm a little new to it... I'd like to be able to grab absolutely the entire contents of the disk, including the MBR, partition table, etc., and including any "deleted" files or non-overwritten sections of partially overwritten files.
I'm also interested in preferences regarding compression. I know it will not be terribly effective (the biggest gain is taking out the fs space...) but even a small gain would be useful. I also want to avoid any compression scheme which destroys an archive due to small early errors, like gz (as I understand...)
thank you
ghost over the network (Score:1)
(I'll preface this with the fact that we're a small office, ~30 desktops, and we have part-time night operator monkeys that can do things when no one is around)
As the size of harddrives and applications has ballooned over the past few years the cheap tape drives have not kept up, so we were more or less forced to look for alternatives. We turned to Norton Ghost to make image backups because it is pretty quick, and is also usefull for setting up new machines. Ghost runs off of a boot disk and will write to any media you have DOS mode drivers for.
We first thought about getting a USB drive of some sort, since all of our machines have USB, but couldn't find DOS drivers for USB CD-RW drives. But then we remembered that all of our machines have something else, and that is a network card. Using a pair of boot disks, Ghost will image a drive from one machine, over tcp/ip to another to store the image file. Since our network is unused at night, we're not worried about traffic (and actually, 100Mbit switched ethernet, it wouldn't be that bad even during the day). We store the image on a machine with a CD-RW drive in it, and then burn the image to CD-RWs (1 or 2, depending on how much stuff is on the drive). We keep 2 sets of backups for each machine, and rotate through them. The cost of CD-RW is low, even if they do have a finite lifetime, especially when compared to tapes.
If your network is 24/7 or some such, you could always create a mobile "ghosting station" with a machine that had enough space to hold the image files (before burning to cd or wherever) and a mini-hub. Then you'd just unplug the machine from the real network, and into your ghost network in order to image.
If you don't have monkeys to run around and do the images, I believe Norton makes a Ghost server for NT that will do remote imaging, but I've never used it.
Re:I've said it before ... SCSI!!! (Score:1)
SCSI is not that cheap! Perhaps for a home system, but my company is betting it's business on the systems that we buy. That means quality, reliability, and driver issues are a big deal to us. Each change in a driver results in a different build of the OS image. If we use a no-name SCSI card, each time the support chipset changes we need to build a new image. This is very expensive for us to maintain.
SCSI hard disks are much more expensive than IDE. I just checked pricewatch, and a roughly equivalent SCSI drive was around $200 more than it's EIDE counterpart (36GB)
Thanks again for offering a suggestion
Re:Specifics of Ghost vs dd? (Score:1)
Another thing is that it's possible to edit ghost images - something I don't think is possible with dd.
I'd love to see a linux utility which would read ghost files. Ideally we'd be able to boot linux and use linux USB support for pulling the ghost image file to the hard disk.
In summary I suppose it's possible to replace the existing system with compressed dd images, but it's more complicated - and requires more training than does Ghost.
Thanks for the suggestion!
Regards,
Anomaly
Re:It's easier than this. (Score:1)
Re:I've said it before ... SCSI!!! (Score:1)
And which scsi are you talking about? I? II? III? etc etc...
But back to why I'm posting - Firewire's prices are dropping like flies. I added a Pyro board to my PC at work and at home, then picked up a firewireide interface. Now I've got an 80GB udma100 'firewire' drive that's portable and the cost was reasonable. Now I'm not lugging piles of cdr's back and forth to work with mp3's.
Incase you are curious, I went with firewire because of the previous thread on slashdot about USB drives. I was originally looking at them, but once I realized how slow it is... ugh.
I wonder if there are firewire drivers for dos?
Re:MultiCasting ? File server ? (Score:1)
I've used ghost for the past few years, but haven't messed with the multicasting server much. I generally move the images backnforth from the pc to a netware,linux(smb) or nt4 server's share (server type depended on what I was running for whatever employer I've been with since Ghost was created) with a boot floppy using tcp/ip. I have two versions of the floppy - one to write the image to a file, and the other to load it from the server. All automated with batch scripts.
This works nicely (and, for me, was the easiest to do) *if* you have a nice speedy network. I have mine 100baseT, swithched, with gigabit going switch to switch, so given the traffic load on it, I can ghost whenever I want to.
But, you may not. In that case, I'd go with ghosting images from a cd, scsi or firewire (the scsi and firewire drivers under dos will be fun tho).
Re:MultiCasting ? File server ? (Score:1)
We've been kicking around the idea of using some sort of network appliance for this. We've got a lot of locations, so we need something portable and cheap.
I'm not sure if we're ready for the network-attached storage route yet....
It's a tough nut to crack, because there are so many constraints. Most of our locations don't have servers on site.
Thanks again!
Regards,
anomaly
Drive Image Multicast (Score:1)
Many alternatives... (Score:2)
It's easier than this. (Score:3)
We use DOS network boot disks with scripts which automatically set the IP and then do a simple "net use X: \\server-name\sharename". The X: drive shows up in a list of drives in Ghost; all the stuff gets imaged right to the network drive. Don't even need a Ghost server.
- A.P.
--
* CmdrTaco is an idiot.
Network probably better option (Score:1)
CDR rules (Score:1)
--
OFF-TOPIC?!?!?! Just how much ignorance is on /.? (Score:4)
Secondly, the TekRam DC-315U is a measly $15-20, does UltraSCSI (20MBps) and interfaces nicely to just about any old/new hard drive. It is completely supported by just about any OS (TekRam even has Linux and BSD drivers -- including distro installation driver disks). I buy one for all new PCs I get -- a smart move given the number of external Zip, Jaz, burners and whatnot that fly around the office.
Third, you can get late-model SCSI drives for under $100. Most will have enough storage for several cloned images. You don't need a modern SCSI drive, just one that will give you about 15MBps performance -- that's 10x the performance of USB!!! Add $20-40 for an external case, another $20-40 for cables and terminators and you're cooking! If you use Linux, you can hot-plug the solution and modprobe the TekRam driver on-the-fly!
Fourth, I do this for cloning. Yes, I do some over the network, but I don't always like to taxi my server in the middle of the day (among other reasons). And if I need to get the system up faster, SCSI gives me better performance than the network. Either way, USB is a slow pig and anyone who has used an external USB knows what I'm talking about! God, I cannot believe people have such ignorant, "say no to SCSI" attitudes! Geez!
Fifth, I'm glad someone else mentioned (since I forgot to) that PCMCIA SCSI cards are easily swappable, so yes, I use a PCMCIA SCSI card for the few notebook systems I have. I only need one card (and I have only one card).
Six, I do NOT disagree that IEEE1394 is a better, future solution. Frankly, I'm PO'ed that Intel is stalling on making it a standard feature in the southbridge chip (it was supposed to be standard with the PIIX4 southbridge!). I think AMD/VIA will force the issue with forthcoming chipsets and that will finally force Intel to put it on-board as well. Until then, TekRam SCSI cards are half the price and almost as fast (20MBps) as the 100-400Mbps (12.5-50MBps) Firewire cards. And, again, I'm NOT sure there is DOS/real-mode support.
[ But it's still good to see companies like Maxtor coming out with IEEE1394 drives (probably using a 33MBps 1394 to ATA bridge like this one [initio.com]). ]
-- Bryan "TheBS" Smith
it wasn't a troll (Score:1)
gzip, of course, doesn't apparently meet my other needs...
but at least I have part of it solved.
ATA drives in Firewire are usually 33MBps (Score:2)
FYI, I seriously doubt you get Ultra100 on a 100-400Mbps (12.5-50MBps) Firewire interface. In fact, all the ATA to 1394 bridge controllers I've seen are Ultra33 (33MBps -- like this one [initio.com]).
Most IDE/SCSI drives burst transfer at 15-30MBps from platter to interface. The 12Mbps (1.5MBps) interface of "fast mode" USB is a major bottleneck.
Worse yet is the massive overhead (much worse than SCSI or Firewire). USB is one of the worse serial busses ever created. There were better alternatives but Intel and Microsoft designed USB so most of the effort fell on the device and drivers (the controller and OS support is minimal). Why do you think the devices lacked the basic southbridge/OS support by 2-3 years???
-- Bryan "TheBS" Smith