Terabyte Storage Solutions? 574
DeMechman asks: "As many on Slashdot may know, storage is one thing which you can never have enough of. Given the current situation with CD/DVD rot (Personally I can attest to a 10% attrition rate) hard drives in a RAID configuration seem to be a better and more economical solution. If you own more than fifty CD/DVDs, it can be a daunting task to find a file. I am wondering if anyone has found a hardware solution that can inexpensively be set up to handle 10 or more 250GB HDDs in a RAID configuration. Primarily, has any case manufacturer tackled this niche market yet?"
What's "inexpensively"? (Score:5, Informative)
http://www.apple.com/xserve/raid/ [apple.com]
Academic prices for:
1.00TB - $5399
1.75TB - $6749
3.50TB - $9899
Re:What's "inexpensively"? (Score:4, Informative)
Re:What's "inexpensively"? (Score:5, Informative)
Go buy a Lian Li case, 8 x 200gb maxtor harddrives and a 3ware raid controller.
Controller $500
Drives $150 each
Case $150
Total for 1.4TB = $1850
With 400gb drives maybe $3000 for 2.8TB
Re:What's "inexpensively"? (Score:5, Informative)
12 x 3.5 bay" Tower [lian-li.com]
3 Ware 12 drive RAID card [3ware.com]
Drives can be found at PriceWatch [pricewatch.com]
I I havn't calculated the per MB cost of all the large sizes. someone with more time please do this.
What will make this perfect is removeble drive kits (They require an external 5.1/4" bay for each 3.1/2" drive. Some even have little activity LEDs) and a server case with 12 external 5.1/4" bays.
Re:What's "inexpensively"? (Score:5, Informative)
Re:What's "inexpensively"? (Score:5, Informative)
Hopefully, they've changed this for the newer 7 series cards, but the 5 series are 'broken' this way.
Re:What's "inexpensively"? (Score:5, Informative)
Re:What's "inexpensively"? (Score:3, Informative)
Re:What's "inexpensively"? (Score:5, Funny)
Drives $150 each
Case $150
Redundant Power Supply for RAID Array.....priceless
Re:What's "inexpensively"? (Score:5, Informative)
50 CDs * 700 MB = 35 GB
50 DVDs * 4.7 GB = 235 GB
It would take 250 DVDs (all FULL!) to get you to that terabyte. But you want to put ten 250GB drives together... so you want 4 drives (for the space) and six drives for redundancy.
Expect to put down $5,000+. Or buy a 250GB drive and just store them on there. Buy two, and use the second one as a backup of the first. Total cost? $400.
If you're a home user - don't go overboard. If you're a corporate user that's just trying to cut corners (and therefore cost) then don't shortchange yourself (or your company).
Re:What's "inexpensively"? (Score:3, Interesting)
Re:What's "inexpensively"? (Score:3, Insightful)
What's wrong with this statement that it invites derisive (if informative) giggles? I don't think the author has just 50, it's just that over 50 and finding files becomes more of a task. I'm sure some good file management databases are in order, but anyway optical disc media sure seems to fail a lot more than makes it safe for easy backup.
I have 1.5 Terabytes of personally collected data and I don't understand what's wrong w
Re:What's "inexpensively"? (Score:4, Insightful)
A friend of mine once lost all the data on two drives (RAID 1) in a country with extremely reliable power (Japan; even during typhoons I never once had a power outage in 8 years) when the UPS suddenly died one day and dumped the whole battery load into the computer. The white smoke escaped from everything.
If your data is really valuable, offline storage is not a luxury, it's a necessity. Get a DLT drive (or a changer, if you can afford it). Offsite is easy. Keep at least one backup set at your office. If your house burns down, you're covered. If something so bad happens that it destroys both your house and your office, you have bigger problems than the lost data
Re:What's "inexpensively"? (Score:3, Insightful)
You may have to repeat this pro
Re: (Score:2)
Re:What's "inexpensively"? (Score:5, Informative)
Some slashdot articles on some previous attempts:
Bulk Data Storage For The Common Man? [slashdot.org]
Home-brewing a 1.2TB IDE to Firewire Monster [slashdot.org]
Books on it:
Managing RAID on Linux [slashdot.org]
Even applicable controller hardware:
LSI Megariad 150-6 [lsilogic.com]
3Ware 9000 series [3ware.com]
And soon to be applicable storage hardware:
Hitachi Announces 400GB Hard Drive [slashdot.org]
SATA Setups (Score:4, Interesting)
Re:What's "inexpensively"? (Score:2)
And a dual cpu is vastly overkill for anything you're going to put down the ethernet card
Re:What's "inexpensively"? (Score:5, Informative)
Oh, bullshit.
Linux software RAID1 is just as fast as several of the hardware RAID1 setups I've tested using Bonnie++ -- These are fucking fileservers, not renderfarms. The processor's sitting there doing jack shit anyway, and you're more than likely putting a P4 in there since you can't buy anything else with decent reliability. Throw in a decent GigE network card and your processor is STILL at 0% utilization. Make that a RAID5 with hot-standby drive and I would be very surprised if you noticed any difference in the apparent "feel" of the server compared to a hardware RAID solution.
Hardware RAID's okay but now you've got a proprietary format array with a SPOF (the RAID card(s)) -- sure you can keep spare RAID cards around but honestly unless you need every last bps on your network transfer and you've got your server so overloaded that SW RAID is impacting your performance you're just incurring extra expense. I am very happy that I can take any RAID array I have and throw it in another system should a motherboard or controller fail and I need the system up immediately. I'm very happy that LVM Just Works and works happily on top of software RAID. There's no issues and no extra question marks like there are with any hardware RAID "solution".
Want beeping? Write a script. Want email/phone/paging when something goes wrong? Write a script. Or use any of the monitoring and alerting systems you can find on Freshmeat (mon, nagios, etc.). Jesus H Christ, give your head a shake.
Oh wait, you're trying to build a performance system using an OS built for pushing pixels. Perhaps that is your biggest problem. Windows has its place, but high performance data transfer just isn't one of them. I guess if you've decided to spend a couple hundred on an OS license that gets you nothing you may as well blow another couple hundred to get hardware to go with it.
Re:What's "inexpensively"? (Score:4, Interesting)
The reason they gave is that the even a fraction of modern CPU performance still far outclasses the chips on hardware RAID cards. Also, data cached on the card still has to go over the PCI bus, but data cached in RAM... well, it's already available.
A RedHat employee who was there confirmed that RedHat has seen the same thing in their own testing. For performance go with software RAID. With anything over about a 800Mhz CPU, you would be hard pressed to notice the CPU use.
In fact, unless you are doing something that is virtually entirely computational like SETI@Home, you are going to be generating a fair amount of output. Enough that the faster disk IO actually increases your speed more than what would be gained by moving the RAID load to seperate hardware. It also lets you spread disks over a couple SATA controllers and potentially multiple PCI buses (if your MB supports it.)
It's not RAID, but ... (Score:5, Informative)
Re:It's not RAID, but ... (Score:2)
Re: (Score:2)
Warranty sucks (Score:3, Insightful)
Re:It's not RAID, but ... (Score:4, Insightful)
1 terabyte = 1,000,000,000,000 bytes
Try 1024^4 = 1,099,511,627,776.. wait, where'd my 100 gigs go?
Due to the exponential nature this little white lie hurts a bit more for every increment, here sacrificing just about 10% of the storage. I'm surprised they don't say 1000 gigs just to dodge the 10% mark.
For those who insist that tera means one trillon for bytes, I reference
Here [techtarget.com], here [sharpened.net], here [ic.ac.uk], here [thefreedictionary.com], here [wikipedia.org], and how about here [reference.com]. Now I'll admit the wikipedia entry has the trillion byte definition, but they basically said it is used in storage advertising.
Re:It's not RAID, but ... (Score:4, Interesting)
also, I beleive they don't even qualify as the badly named, raid0 as I under the impression that the disks are concatted together, not striped.
what I'd love to see is an Xraid mini as it were. something with much of the managability of the full size xraid, but not as much redundancy. so perhaps a nice desktop case (to match the g5 *of course*:) that could take 4 or 5 sata disks in hot swap caddies (maybe the same caddies as in the xraid) with a hardware raid controller on board for striping, mirroring and raid 5. a single gig ethernet on the back and then fw400 and 800 ports.
if it had the same cross platform compatibility as the the big xraid, same type of management tools etc, then it could be a big hit, and be an official filling for the big hole that is g5 storage.
sure, the xraid is great and cheap, but it's price of enhtry i still high when all you want is a terabyte or so of fast storage for one of two machines at home, ie no rack to place, no need for redundant psu's and fibre channel connectivity, that kinda thing.
HD video editors esp need something as for the data speeds they need for uncompressed hd (180MBps) thats 4 striped disks which you can't place in a g5 without using third party solutions.
just a thought, come on apple. and when you make one, I just ask for a fully loaded one for myself
dave
Re:It's not RAID, but ... (Score:3)
Basically you want an xserve raid with ethernet then in a stylin case..
Re:It's not RAID, but ... (Score:3, Insightful)
Re:It's not RAID, but ... (Score:3, Informative)
It's about 1/5th the cost of the xserve raid but not nearly as flexible:
Many have (Score:3, Informative)
See page here. [apple.com]
Re:Many have (Score:4, Informative)
Lets see, 5 200 MB drives at $120 = $600 + another $600 for the case, MB proc etc... $1200 for a terrabyte server.
I haven't looked (you can do that) but I bet there are plenty of stand alone raid units of that size for maybe twice the DIY price and that is still HALF the price of Apple.
Now THIS is informative!
Re:Many have (Score:3, Insightful)
Not exactly what you're looking for.. (Score:2)
Re:Not exactly what you're looking for.. (Score:3, Funny)
Comment removed (Score:4, Funny)
LEGO(tm)-ROM (Score:5, Funny)
Or, if you want really durable read-only storage (i.e. lasting a few hundred years without maintenance), you could use the little 1x1 LEGO blocks as bits.
Therefore, a mere eight-by-eight city block area could store a full 1 terabyte of LEGO-ROM, with no worrying about DVD rot or head crashes (although access speeds would leave something to be desired).
Re: (Score:2)
Intel SC5200 5U (Score:2, Informative)
Easy these days. (Score:5, Informative)
I have a TB here, and rather than raid, I decided to do a nightly "rsync" mirror to a "yesterday" partition.
The two advantages of the nightly rsync over RAID are
Re:Easy these days. (Score:5, Informative)
> advantages of the nightly rsync over RAID are
Instead of only keeping a "yesterday" partition, use rsync to keep EVERY daily backup.
Rsync has lots of great options to make copies as hard links if they haven't changed and only copy changed files. That allows you to make daily full backups that only use the space of daily incrementals. Do that to a backup partition, then RAID-1 the whole drive over to a mirror.
That gets you full protection from hardware failure on a drive and user failure on your files.
Google for more details [google.com]
easy to do with rackmount cases. (Score:5, Informative)
Re:easy to do with rackmount cases. (Score:2)
Performance is about on par with having the ide drive directly attached to the
Re:easy to do with rackmount cases. (Score:2)
Inexpensive, not cheap.
External Drives... (Score:2)
2nd Question - Backups (Score:2)
Re:2nd Question - Backups (Score:3, Informative)
Re:2nd Question - Backups (Score:2)
Terabyte Storage (Score:5, Informative)
The 160GB drives used to come with a Maxtor [Promise] ATA-133 card. Two of those will support eight drives. Not the most optimal arrangement because of the bus having two drives on each channel, but it doesn't seem to affect performance too much since it is striping the data across all of the drives. I'm assuming it stripes in order, so you'd want to stagger the drives such that 1 & 2, 3 & 4 are not on the same controller.
Output of df -h:
The cost to assemble something like this?
~ $600.00
8 x $70 for the 160GB drives
2 x $20 ATA-133 controllers
The biggest issue is that there is no easy way to back up the array. You could use RAID 6 and have two drives worth of parity info, but it still leaves you vulnerable to a catastrophic hardware (or building) failure.
Anyone have any ideas on how to back up 1TB in a home environment? i.e., not $3000 tape drives & $200 tapes
Re:Terabyte Storage (Score:2)
Also, 3Ware & RaidCore (now Broadcom) have 8 channel & 12 Channel SATA cards for relatively low prices. That would be a better albeit more expensive route to go.
Re:Terabyte Storage (Backup Solution) (Score:3, Interesting)
Re:Terabyte Storage (Score:5, Funny)
Ummm, yeah, it'll cost you ~$600. make another one and make a copy occasionally...
Sorry, couldn't resist...
Re:Terabyte Storage (Score:2, Interesting)
Not the most optimal arrangement because of the bus having two drives on each channel, but it doesn't seem to affect performance too much since it is striping the data across all of the drives. I'm assuming it stripes in order, so you'd want to stagger the drives such that 1 & 2, 3 & 4 are not on the same controller.
Have you worked with a 3ware card? Believe me when I say that this solutions' performance will suck compared to using a real raid solution such as a Escalade 3ware 9500s. Even on sof
Re:Terabyte Storage (Score:5, Informative)
8 Firewire drive enclosure: (i have the 4 drives version).
$600. http://www.cooldrives.com/fi80013oc5fi.html
$136
$700 = Hardware for a Linux machine as correct file server
= $2930 for 2TB of raw space, 1.5TB Of raid 5 with an hot spare, or 1.75TB of raid five with no hot spare.
You got yourself a nice fileserver for home usage... install that with mythtv and you're up for hours of video....
Re:Terabyte Storage (Score:3, Insightful)
Can anyone give me a rough formula of wattage/# of devices?
Re:Terabyte Storage (Score:4, Informative)
According to Western Digital's site, a 250GB SATA drive pulls 12.8 watts when reading/writing and 9.5 watts on standby. I figure for 6 drives that's about 100 watts of a _good_ power supply's rating.
3ware Controllers + Drive Friendly Case (Score:2, Informative)
Re:3ware Controllers + Drive Friendly Case (Score:5, Informative)
Our 48 Node beowulf has a
Anyway, that kind of load brought that head node (dual proc 1700+ MP) to its knees until we decided to rebuild it. Moving from the hardware controlled raid to linux's software raid completely resolved that problem.
Re:3ware Controllers + Drive Friendly Case (Score:4, Interesting)
The reason why you're getting better RAID 5 results from software RAID vs. hardware RAID is because of the parity calculations involved with writing to a RAID 5 volume. On a hardware RAID setup, these are calculated on the RAID card itself, which probably has a 200 or 400 mhz. chip that does these calculations. Back when CPUs were only 400 mhz, this was great, because there was no load put on the CPU, and the RAID controller worked just as fast or faster than a software RAID setup. Now that CPUs are 3 ghz. +, there's no way a dedicated hardware RAID card can keep up, and unless you're running a huge load on the server, youv'e probably got 1 ghz. or so of free CPU bandwidth to burn for software RAID...
Want to see the performance really increase? Give up RAID 5 and go with a real RAID solution like RAID 1 or RAID 1+0.
Re:3ware Controllers + Drive Friendly Case (Score:3, Interesting)
How on Earth is RAID 5 "less real" than RAID 1 or 10 ?
Wow! (Score:2)
Sorry, I couldn't help myself...
We did it and have a couple now ... (Score:3, Interesting)
Seagate (Score:2)
What I did... (Score:5, Interesting)
Unfortunatly, once you have all this space, you WILL find a way to use it all and need more. I put this system together about 10 months ago, and it's at 85% capacity now. I'm preparing to build a new server with 12 250GB drives, to have just over 4TB between the 2 systems.
Sometimes commercial is nice (Score:3, Interesting)
Re:What I did... (Score:5, Informative)
CD/DVD rot (Score:2)
I'm a bit concerned by this phenomenon and think surprisingly little is said about it, when you consider how common these media area. Has studies been made with comparisons from different brands? I'm not sure a study of unknown brands are very helpful since there could be great differences between different manufacturers, or?
I would never buy a DVD from, say, Princo or other budget brands, and rea
Re:CD/DVD rot (Score:2)
I can't think of anyone (Score:2)
Nexsan ATABoy (Score:2)
www.nexsan.com
relatively cheap raid boxes... (Score:2, Informative)
Cheap Fast Reliable (Score:2)
Cases are difficult but.... (Score:2)
Ever Try External Hard Disk Enclosures? (Score:3, Informative)
Just built one... (Score:5, Informative)
I went with Serial ATA for a couple reasons:
1) It's cheaper and has more capacity than SCSI;
2) Cabling is not a mess as it is with regular IDE (if you've never seen serial ATA cables, the first thing you will notice is that they are small!);
3) It can hotswap, unlike regular IDE;
4) It's not that much more expensive than regular IDE.
I custom-built a 3U server from InterProMicro. [interpromicro.com] They are a small (local if you are in the Bay Area) SuperMicro reseller that does great work. (If you need something, call and ask for Andy. Tell him Erica from Simpli sent you!)
The machine I specced out was as follows:
* 3U case with 8 hot-swap SATA drive bays;
* 8-port 3Ware 8506-8 SATA RAID controller;
* 5x250GB SATA drives in a RAID-5 array;
* Dual Xeon processors.
The 5 drives give you 1TB of storage, and expanding up to 8 gives you 1.75TB. I would also recommend a separate mirrored SATA 10KRPM array for the OS if you want really fast speeds.
This whole solution (Xeons; 5 drives; 3U case) cost just over $3000... which is pretty reasonable for 1TB of network-accessible storage. Interpro has solutions that go up to 24 SATA drives [interpromicro.com], which at 250GB each gives you an ungodly amount of space (5.75TB, if my calculations are correct.)
My suggestion is to go with a niche server builder like InterproMicro over Dell or Compaq or any of those guys. You can get the same high quality from a custom manufacturer without paying the steep brand name price from a larger manufacturer. As for the drives, any time the goal is "as much space as possible", SATA should be your first choice.
Good luck!
How cheap are you trying to get this done? (Score:2)
Probably your best bet is to get a full tower case and add some drive bay capacity to it with sheet metal. You'll need to add several new fans, and you will want probably a good 500W of power supply capacity if not more. If you can get drives with a spinup delay which you can specify, then you can probably get away with less. Two or more cheap power supplies should do the job, I bought a couple of 250W power supplies for $7 each a while back, should be easy enough to do that still.
Adding drive capacity
Similar question (Score:2)
Maybe LVM (Score:2)
3Ware (Score:2)
The Escalade 8506-12 has 12 x SATA ports onboard. Full hardware implementation; appears as a SCSI host adapter to the OS. Drivers and management utilities for MS-Windows, Linux, and FreeBSD. It will even email you if you have a disk failure.
3Ware was one of the first ATA RAID vendors to put a driver in the Linux kernel, and it was a fully-supported, GPL driver from day one. Rock solid stuff. Good tech support, too.
Highly recommended.
Yes and No... (Score:2)
More recently I've purchased 500Mb and 1TB LaCie Big Disk and Bigger Disk storage devices. They work extremely well and have almost no latency for their size. I use them for testing things and have been quite
In RAID, IDE has the disadvantage... (Score:3, Insightful)
I'm not referring to performance, reliability, etc. (although those are serious issues), but about price.
If you have a master & a slave, then you reduce performance... That can be a very serious if you have a RAID configuration. So, if you want to put 7200RPM hard drives together, you start to need a 6 or more channel RAID card (whereas a single channel SCSI RAID card would work fine). And guess what? Decent quality 6+ channel RAID cards are very expensive, perhaps even negating the savings from using IDE drives rather than SCSI in the first place.
Remember, that's based on price-only... I haven't even begun talking about how much worse the performance would be, or reliability issues with using inexpensive IDE drives.
Re:In RAID, IDE has the disadvantage... (Score:3, Interesting)
IDE RAID hit mainstream over five years ago, when Adaptec released an IDE RAID card. This card happened to have four separate IDE controllers chips on it, and four cable connectors. I installed a solution using this card with four 73GB IDE drives from IBM (as big as they came in 1999, I think) in an 0+1 configuration. Mirrored striped sets, total usable capacity of 130GB, I think. (Not bad considering I had replaced mirrored 9GB SCSI drives.)
Wouldn't you know it, but
Several options (Score:2)
You can find RAID cards that will support up to 8 drives, but few that will support more, and often those that support multiple drives cost more than the drives themselves.
Your best bet, I suspect, is to make a d
Re:Several options (Score:2)
-Adam
Distributed RAID (Score:2)
Good solutions still cost a reasonable amount (Score:5, Informative)
-What RAID level you want (5 usually requires better hardware)
-Whether you want hardware RAID (I strongly recommend this) or soft RAID
-How much redundancy you need (Battery backup cache? Redundant controllers? Hardware environmental controls?)
If you are looking for good pci cards, I would strongly suggest a card from 3ware [3ware.com], and a card from a place such a Seagate [seagate.com]. Getting a super-duper cheap card when terabytes of data are on the line is just fundamentally stupid. You can save some bucks now, but be ready with your next Ask Slashdot: "How do I recover data from my dead RAID?" Seagate now has a nice 5 year warranty, which match well with good quality and reasonably cheap drives. Look at some of the SATA drives like the Barracuda. However, any decent quality drive maker can work. If you have even more money, you can look at some of the things offered by places like StorCase [storcase.com]. A larger initial investment can become cheaper as you scale up the cheap harddrive count, and it can be a good thing in the long run. Obviously, the more time you are willing to invest doing things yourself, the cheaper you can get to some extent vs premade items. However, no support as well.
Do read up on some of the fundamentals of RAID: Everything you need to know (and lots you don't) is probably at least mentioned in the PC Guide [pcguide.com] on RAID. Look through that. Things like hot swap and hot spares are important to understand. Finally, you should remember to check compatability. Unfortunately, I for instance have not been able to find much of anything in the way of controller cards that is compatable with OS X (except the obvious, the XServe RAID). So I have something set up on a BSD box in my server closet that I then link to, more like a storage appliance. Happily, the 3ware cards and many others are now compatable with a wide variety of *nix and BSD flavors along Windows, but do check to make sure.
Last but not least, remember this!: RAID is *not* a backup solution, but an highly redundant onsite storage system. Have another form of backups, even if it is just a RAID 1 off site, or DVD-Rs, or something. If a disaster happens (thieves, fire, nuclear destruction, John Ashcroft) on site storage won't save you.
Off the shelf or build yourself? (Score:3, Informative)
If I were going to do it I'd build it my own by combining a nice case [rackmountpro.com] and a 12 port 3Ware controller [rackmountpro.com] with whatever server configuration and SATA drives I wanted to get.
All about the Apple Xserv Storage Arrays (Score:2)
NAS or SAN (Score:2)
Can't go wrong with a Skyhawk IPC-5101 (Score:2)
Only $150 at CableMart, Inc, with free shipping. [cmicomputer.com]
We've built two 2-TB NASs with this Skyhawk case and are working on a third right now. The case is a 5u, 10 bay, industrial strength one that's damn sturdy.
Throw in a decent power supply, a 3ware 8506 8 or 12 port SATA RAID card (or the equiv. for standard IDE), 8 data and 2 system drives (7200 rpm SATA or IDE), some Kingwin BK-81 [newegg.com] drive bays, an inexpensive motherboard and chip (Biostar M7VIZ w/ an Athlon XP 2800), and a gig or two of value RAM [ms4me.com] (make sure
Eh... I'm measly (Score:2)
It's running at RAID 5 via Mac OS X and software called RAID Toolkit from FWB Software. Great stuff, never failed and was fairly cheap..
Enclosures = 3x$40
Hard Drives = 3x$250
Software = $99
$3.00 per raw GB... but RAID 5 I only realize about 550GB so that's about $5.30ish I guess.
we made LOTS of 1.3 TB boxes at about $2000 each (Score:5, Informative)
Rip all your CDs as FLAC [sourceforge.net] so that (1) you never have to rip them again (it's lossless), but (2) it's half the size of saving WAV files
At least that's what we've done with our 68,000 [cdbaby.com] CDs we have here.
Re:we made LOTS of 1.3 TB boxes at about $2000 eac (Score:3, Interesting)
Re:we made LOTS of 1.3 TB boxes at about $2000 eac (Score:3, Interesting)
All you need is 1-2TB of cheap disks (Score:3, Interesting)
I use DVArchive with DVD or satellite to ReplayTV for video capture and play back, DVA is great for managing multiple volumes and dynamically discovers vidoes if I want to move them to another drive. It also supports copy/move between the two systems (I use a 1Gb switch between systems). CPU performance is not key for play back though it is critical for transcoding (I use a dual processor system for transcoding and it smokes my single CPU system).
I have a LARGE MP3 collection (forgive me for not publically admitting to its size) and I find the same systems/drives are ample for supporitng a digital audio library. I switched to iTunes for managing music (MusicMatch melts down when the number of files gets large) and stream it with SlimServer to squeezebox devices for high quality playback on home theater and other receivers.
My recommendation is to go with generic disk drives - brand names, 7200 RPM with 1-3 year warranties --I get them locally on sale for under $150, sometimes $130/250GB, thats 52 cents per GB, a little more per GB than a DVD-R disk but more reliable and infinitely more flexible. I can recreate a DVD off of the disk image if needed.
I am more concerned with heat and power consumption (it adds up) than disk performance, someone will need to explain to me why I'd need to mess with RAID for this...
I understand how CD rot works (Score:4, Interesting)
My usual solution... (Score:3, Informative)
* SuperMicro motherboard (any of the newer ones, depend on your choice of architecture). Be sure to get one with PCI 133/64 and gigabit onboard.
* 3Ware RAID board(s).
* Chembro rackmount cases (they have a very nice one with 16 SATA hotplug slots with backplane and all)
* Don't go cheap on the power supply. You'll need at least 600W. I always go for redundant ones.
* 16 SATA disks of your choice (250, 120 or 80GB)
* Linux!!! (Be careful with fedora core2, it doesnt support nativelly the 3Ware cards - you'll need to compile your own)
Of course you could save about $1000 by using a cheap motherboard, chassis and PS. But it really pays off using the good brands on those.
By the way, you should always get an extra hard drive (or two). They will fail (sooner or later) and you don't want to be left hanging.
Off the shelf, not home-made (Score:4, Informative)
Disclaimer: I work for a storage integrator, both are brands we sell.
Hardware RAID (Score:3, Interesting)
http://www.areca.us/IDERAID.htm [areca.us]
It takes up 3 external 5.25" bays and allows you to connect 5 3.5" drives. It provides expandable RAID 5, all internally with it's hardware and simply looks like an ATA or SATA device to the computer.
Has anyone here actually used one?
kiwi
--
System Architecture
Toshiba TMPR4927ATB 200MHz 64-bit RISC processor
64MB on-board cache memory with ECC protection
Areca 5 channels IDE controller (ARC600-66) with enhanced H/W XOR engine
NVRAM for RAID configuration & transaction log
Write-through or write-back cache support
Firmware in Flash ROM for easy upgrades
RAID Features
RAID level 0, 1 (0+1), 3, 5 and JBOD
Multiple RAID selection
Array roaming
Online RAID level/ stripe size migration
Online RAID capacity expansion and RAID level migration simultaneously
Automatically and transparently rebuilds hot spare drives
Hot swap new drives without taking the system down
Instant availability and background initialization
Automatic drive insertion / removal detection and rebuilding
Disk Bus Interface
Ultra ATA/133 compatible
5 channels, operating in parallel
5 hot-swap drive trays
48-bit LBA support allows disk exceeding 137GB
Staggering the Spin-Up of Individual Disk to Solve the Power-on Surge
Host Bus Interface
ARC-5010
Dual ATA interface-Ultra ATA/133 & Serial ATA 1.0
Ultra ATA/133 compatible Transfer rate up to 133MB/sec
Serial ATA 1.0 - 1.5Gbps(150 MB/sec)
ARC-6010
Ultra 160-Wide LVD SCSI; Transfer rate up to 160MB/sec
Tagged Command Queuing
Concurrent I/O commands
Somewhat off-topic, but... (Score:3, Informative)
Um... ever consider the mind-bogglingly simple solution of:
ls -R> ~/dvd.index/<disc_label> for each dvd
grep "<whatever_youre_looking_for>" ~/dvd.index/*
You're doing it wrong (Score:3, Informative)
Two or three fairly normal PCs with STANDARD drive controllers, PSUs and HVAC.
Look, we're talking file servers here. 128MB RAM is gobs if you aren't running any other service on 'em. Pick and OS, any OS: 2000 gets you dfs, *nix gives you NFS. Both give you a homogenous networked file system.
So...
Standard case/PSU/cheapo CPU (Athlon mobile or Via or P3, for lower power consumption)/RAM - That's $250, maybe. Add another $20 for a gigabit NIC or two per machine.
4x 200GB drives @ $110 apiece (pricewatch shows $96 as the low price, but I'll go $110 for a little wiggle room)
So... something around $700 gets you
Buy three machines. $2100 gets you tons of storage and scads of redundancy no matter how you look at it.
This is the philosophy I use in setting up my file servers (now serving 6.5TB!). Over time I've added 3ware cards, upgraded PSUs and added gobs of RAM, but my basic starting point is a very modestly-appointed system.