Hardware RAID 5 Performance Configurations? 62
gandy909 asks: "I am facing the need to replace a major server in the next few months due to both EOL status and disk I/O bottleneck issues with the array containing the data. The server is configured with a 2 channel array controller. It is a RAID 5 array and has 4 drives, 2 per channel (2 data, 1 parity, and 1 hot spare). Obvious performance benefits in replacing the server are quadrupling CPU speed and doubling the memory. Other side benefits I will gain with the new drives, that I think should help my performance, are moving from u160 to u320, and going from 7200 RPM to either 10k or 15k RPM." How would you configure a larger array to best increase its performance?
"Having googled around, the consensus is that increasing the number of drives is the preferred way to attack the I/O bottleneck. What I don't find much help on is determining the configuration of the larger array. Assuming I am going to be using a 12 drive array I have come up with the following possible configurations:
1 2 channel controller, 6 drives per channel
1 4 channel controller, 3 drives per channel
2 2 channel controllers, 3 drives per channel
3 2 channel controllers, 2 drives per channel
Would any one of those configurations provide better performance than the others, or would they all even out considering other factors?"
Get away from RAID 5 (Score:5, Insightful)
But generally, I don't recommend RAID 5 for performance critical situations. It's great for data warehouse, but if you lose one drive their goes your performance. Also, realize that, often, the place where you can really boost performance is in the database, not the hardware. How's your query optimization? Do you have appropriate indexes? Is the code accessing the database efficient?
Re:Get away from RAID 5 (Score:2)
Re: (Score:2)
Re:Get away from RAID 5 (Score:2, Informative)
Are you kidding? Have you ever used a Hardware array controller? do you even know the difference between a 0+1 and a RAID 5?
First of all any controller made in the last decade has a fast enough proccessor on it that it is going to be faster than software RAID, second, a 0+1 is WAY slower than a RAID 5. For example lets say I got 4 disks if I setup a 0+1 that would be 2 mirror sets and the data is even split across the 2 arrays so I ask for a file and I have just 2 drive seeking the info, and retreiving th
Re:Get away from RAID 5 (Score:3, Informative)
RAID-10/01 gives you a mirrored stripe; mirrors improve read performance by letting you balance reads across drives either to increase STR or TPS, stripes improve read performance again and a
Re:Get away from RAID 5 (Score:2)
This one statement shows that you do not know much about RAID 5.
RAID 5 only requires that the parity stripe and the changed data stripe be updated. It's really quite simple, you read the old data and parity stripe, XOR the new data stripe with the old one, XOR that output to the parity stripe, and write out the new parity and data stripe.
Re:Get away from RAID 5 (Score:1)
I withdraw my "no matter how much" remark. RAID-10 is still simpler to implement and an excellent performer, which given how crap most hardware controllers/drivers seem to be, can only be a good thing
Re:Get away from RAID 5 (Score:2)
Re:Get away from RAID 5 (Score:2)
The performance hit with RAID5 doesn't come from the parity calculation, it comes from the additional I/O operations that RAID5 requires.
For example lets say I got 4 disks if I setup a 0+1 that would be 2 mirror sets and the data is even split across the 2 arrays so I ask for a file and I have just 2 drive seeking the info, and retreiving that info, with a 4 disk R
Re:Get away from RAID 5 (Score:2)
(Answer appears to be that your controller sucks [pcguide.com]).
Well, knowing 0 about your business requirements. (Score:5, Informative)
i am not an expert but... (Score:3, Informative)
#1 can be fixed by adding more controllers
if you're
Depends on usage patterns (Score:5, Interesting)
Second, RAID-5 is great for read speeds, but less great for write speeds. A good caching controller will help hide this, but a small write operation requires a read from each disk in the set before the write can be completed (in order to recompute parity for the stripe). If this is mostly reading, or if most writes are large (not small and random), RAID-5 will work fine (data storage, data mining, etc). If writes are frequent (transactions), RAID-5 is painful. RAID-10 might be better.
Re:Depends on usage patterns (Score:2)
I meant to say larger.
Re:Depends on usage patterns (Score:4, Insightful)
Fast drives can sustain about 60 MB/s nowadays, so don't put more than (max channel rate)/(max drive rate) = (320)/(60) = around 5 drives on a single channel.
As for multiple controllers, if they aren't present for redundancy purposes, you're probably just as well off using just one.
But then again, I'm just a Slashdotter who hasn't kept up with the latest specs, so I'm pretty much just making stuff up. Free advice is worth every penny.
If the server has dual PCI-X buses... (Score:2)
Re:Depends on usage patterns (Score:4, Informative)
Re:Depends on usage patterns (Score:2, Informative)
a small write operation requires a read from each disk in the set before the write can be completed (in order to recompute parity for the stripe)
Not entirely. If your parity is a simple XOR of all the chunks, then you only need to read the chunk to be rewritten and the appropriate parity chunk. XOR the old chunk with the parity, XOR that with the new chunk and rewrite the parity chunk, and finally write the new chunk. However, this still necessitates a read from the drive where the write is to occur and
Re:Depends on usage patterns (Score:1)
Why not forego raid5? (Score:2, Interesting)
Re:Why not forego raid5? (Score:2)
Because in anything where you are worried about performance a simple RAID-1 is going to suck completely? Sure you get some advantages on reads with a decent controller but that's it. For performance critical systems a large RAID5 with a good controller or a RAID-10 across one or more controllers is definitly the way to go. For database type operation with a heavy write load (the most common load where this level of optimization is necessary) prese
Why RAID5? (Score:2, Interesting)
What are the applications? Would you be needing more drive space or adding drives in the near future?
If it's a single application and you know you are not going to need to expand soon or add/change apps - drives are BIG compared to what app needs AND performance is _THE_ issue. Then I'd suggest optimizing around the application.
For exampl
Transaction Logs and Such (Score:2)
What seemed to be of value was to put WAL onto a separate array simply out of the knowledge that those disks would get burned through faster, and we'd know they would have a higher replacement incidence.
What I'd like to do to fix that is to have a few GB of SSD, where heavy writes wouldn't hurt any disks.
But on
Re:Transaction Logs and Such (Score:2)
Maybe someone should have a "HDD" that's battery backed 1-2GB RAM for WALs or stuff like that. Rechargeable lead-acid gel batteries (2+ year lifespan).
Would 1-2GB be good enough for such things?
Re:Transaction Logs and Such (Score:2)
The slick part isn't having a huge battery, but rather having a form factor that's similar enough to plain old disks.
What I'd love to see is the combination of:
Long battery life isn't necessary; you just need long enough to write it all
More more more (Score:1)
I'll just assume it's an ERP, cause I like ERPs. They do a ton of reading and infrequent sustained transfers. If it is an ERP, your top priority should be more main system memory. Then use more smaller, fast drives. The more cards the merrier because your double or quadruple the buffers. At worst a buffer offers no advantage. Hopefuly it's a dynamic optimization supporting what the database is already optimizing in main memory. Littl
External an Option (Score:2)
Re:External an Option (Score:2)
google, which most of us would consider "enterprise," they're ability to keep orkut online not withstanding, uses IDE.
if you have the ability to hot swap in a drive, the failure of a single one is inconsequential; yes with RAID5 you are going to take a time hit to rebuild the set, but as many people have already pointed out using RAID5 with the number of discs he is proposing is likely not his best move.
Also glad to know you found that
Concider software RAID5 (Score:1)
If you are planning to go the hardware route make sure you get a controller that can keep up. There's nothing worse than havin
No we will not do your homework for you. (Score:3, Informative)
It looks to me that if you're upgrading from a computer that was 1/4 the speed of today's servers, then if you get a modern server machine with raid SATA you'll be fine.
An incomplete question... (Score:2)
More Information... (Score:3, Informative)
The app is a legacy non-SQL type db that is not, nor ever will be, anywhere near normalized by any stretch of the term. The largest of the data files is just over 1 gig at this point. The OS file size limit is 2gb. Due to this, and the other reasons we will likely be moving to a completely different system in the 5 year range.
Hardwarewise, the box as I inherited it, is a Dell 6400 rackmount server with 4 700mhz P3 Zeons (only 1 activated...don't ask), 1g mem, a PERC2(AMI MegaRAID) dual channel controller, and a split(4+4) backplane. It holds 8 9 gig drives in 2 arrays. Even with these small drives there is over 50% and 70% free space on the arrays.
My budget limit is $10k to replace it. One of the options I was looking at was a Dell 2650 with a PERC3-QC controller and one of the Storcase 10 bay Infostations they offer on the Dell site to hold the rest of the drives. The way the app is so 'interconvoluted' together I don't think I gained anything by separating the data into 2 arrays and will likely just use a single array on the upgrade.
I hope this helps...
Recommendations (Score:2)
Second, w
Re:Recommendations (Score:2)
The system in question is one I inherited. The only reason the OS hasn't changed to Linux is the fact that Synergy wants in the neighborhood of $20K for a 'transfer license' and the beancounters won't approve it.
Yes, I have a SCO system. Whew! There, I said it. I'm not proud of it, but I do have to maintain it. What follows is the
Re:More Information... (Score:2)
Oh, you should be able to get this machine for half your budget -- don't order from the Dell website -- talk to a sales rep. You can get some very good deals on servers n
Re:More Information... (Score:1)
Re:More Information... (Score:2)
PERC3 cards are all pieces of shit.
Read the archives at linux.dell.com on the linux-poweredge mailing lists.
The PERC4 are a great step up but you can still do better with your own cards.
Also, don't ever buy drives from dell. Buy them from Dell "parts and accessories" (the phone number that routes you over VOIP to India...)
You'll get the SAME, BRAND NEW drives from them, still under your normal warranty, still from Dell at about 1/2 or 1/3 of the price. Buy ten drives and they will really start to dro
Re:More Information... (Score:2)
Re:More Information... (Score:1)
+ Dual P4 Xeons
+ 800MHz FSB
+ upto 6GB RAM
+ six hotswap drive bays (optional split backplane 2+4)
Go for..
Dual 3.2GHz (if u have money to throw 3.6s for $1100 extra)
4GB RAM (more is questionable benefit on x86's 32bit arch, some apps it may be worth it)
six 36GB 15K drives
addon PERC4/DC RAID controller
All for $9012
A RAID 10 config would be the best..
+ highest redundancy and
+ best performance.
RAID5 is faster for sustained continuous reads/write but slower for random/small read/writes.
HTH
Re:More Information... (Score:2)
RAID5 != Performance (Score:2)
If you're looking for performance, ditch RAID5 and go with RAID0+1 (mirror pairs of disks, then stripe the mirrors). It costs more in terms of dollars per gigabyte, but the performance and reliability increases are substantial over RAID5. The only justification for RAID5, really, is "we can't afford to buy enough disks to do something else" (which is a valid argument for many people). The other thigns you touched on still apply in a stripe/mirror world as far as splitting things over controllers and wha
3 disk RAID 5 sucks (Score:2)
also in RAID 5 the parity info is spread all the disks, otherwise it's a RAID 3 set, so loss os single disk doesn't mean major performance problems.
seriously (Score:1)
things to consider 1) hardrave sustained transfer speed, IBM made some bad ass 10k ultra 160 drives but they could only push 40 MB/s so a 4 disk RAID 5 would saturate the channel on the controller. also 2)consider the card slot 32 bit slot on a 66 MHz bus is 1053 Mb/s or 132MB/s in theory, so you need a faster bus like new chips or a 64 bit bus, most the last server hardware I worked on only wnet to 66 MHz bus speed with 64 bit busses, so your mileage will vary, so basicly get an U
Re: (Score:2, Informative)
Re:One thing 3ware recommends is... (Score:1)
I disagree on the rounded cables statement. Round cables (esp. in >4-drive arrays) allows more airflow, which keeps the disks cool. For 10K and 15K rpm disks, this is critical.
Now, if you'd said "don't try to make round cables yourself," I might agree -- even though I've done exactly that in the past AND once managed to cut one of the wires in the process -- but there is nothing inherently wrong with round IDE cables. You might claim crosstalk is an issue, but that's largely what the extra ground lin
Re: (Score:1)
Re:One thing 3ware recommends is... (Score:2)
3 2 channel controllers, 2 drives per channel (Score:2)
*BUT* YMMV depending on how your application actually uses the I/O
I know of a widely used RDBMS that really sucks on RAID5 due to its underlying data storage.
You'll need to test with all the combinations and also for RAID 10 (stripped mirror) and RAID 01 (mirrored stripe) to see which gives best performance for your application
Totally Unscientific Example (Score:1)
All systems dell sc1600's 2 x 2.4 xeon's 1 gb ram, perc/3 dual channel raid card
for the highest usage server we did 5 x 36 raid 5 array 2 on one channel 3 on the other for ~ 100gig space + hot spare.
for the others we did 2 73's mirrored one on each channel.
All drives were same family from maxtor mark IV's i believe 10k's
In my rudimentary testing using hdparm -tT the mirrored drives per
use a disk performance monitor (Score:1)
Add RAM (Score:1)
Re:Add RAM (Score:2)
They're really nice boxes [hp.com]
Re:Add RAM (Score:1)
Easy use raid 1+0 (Score:2)
and you will get muchbetter preformance. even just 1+0 is faster than raid 5.
My lowly MS SQL server has 4 raid 1+0 channels.
1 for the OS and swap, and 3 for databases.
--Tim
Re:useful but not fancy configuration (Score:1)
It should be possible to get 147GB drives for the data mirror and faster 73GB drives for the others.
I don't know how the Perc controllers are but when I was speaking to EMC about this they mentioned that they put a lot more effort into optimising RAID-5 performance in their SAN products than other RAID types as this is the most commonly used.