Selecting Computing Hardware for CFD Research? 14
bapya asks: "I am a
PhD student working on CFD ? at National Chemical Laboratory, India.
We mainly use commercial CFD code (Fluent from
Fluent Inc.) and some inhouse
codes. Currently we use SGI machines (Origin 200 and O2 family running
IRIX), single and dual processor NT machines, DEC alpha and dual
processor Linux (running RH 6.2) machines. Now we are planning to
upgrade/expand our computational facilities. For the same amount of
money (about 25000$), we can buy either a decent unix(IRIX) server
(origin 200 dual cpu) or build a much better performing Intel system
(either NT or Linux). My professor is in favor of IRIX system. What
I would like to ask is what are the criteria for selecting hardware
for such a job? and How can one justify the high prices of
branded UNIX systems when other systems of same performance level
can be built at much lesser costs?"
You need hardware your software will run on. (Score:1)
- What kind of hardware do you need to run your software,
- How much money can you save by going with the home-grown solution, and
- How much time and risk do you take by avoiding the COTS solution? (This depends in part on your own expertise.)
There have got to be people who've tried to do things with their own neato system and failed, but we don't hear as much about the failures as we do about the successes. This does not mean that your own situation will be better, or be swept under the rug more quietly, if you try to build your own and fail. 'Syour life.Re:You need hardware your software will run on. (Score:1)
Application toss-up, I'd use Linux (Score:1)
That said, it appears to me that the applications available for more traditional Unix boxes and cheap Linux boxes are about the same. Such being the case, I would certainly go with a Linux box, just for the price factor... it seems as if you can get more raw computing power with Linux on a mid-range PC than on other platforms. Of course, this is just from anecdotal experience from within my company, but why else would you be asking
IRIX (Score:2, Interesting)
SGI/IRIX in my experince is great for crunching numbers, we have an Origin 200 dual CPU with a gig and a half of ram supporting the load of 3 full time 3d designers. Granted SGI has the advantage in the graphics arena, but in our situation the rendering aspect is handed off to x86 workstations (but thats a whole other ball of wax)
One other thing to consider is SGI's wonderful support. We recently had a failure that crashed the server, SGIs ESP program notified them of the crash, and within a few minutes a tech had called us to schedule a service guy to come out and replace our motherboard/ram/cpu with all brand new equipment. Since we're an out of the way town, the soonest we could get is the next day.
On a side note, from what our service tech was saying most of @Home's customer proxy servers are(used to be) Origin 200's.
If you do choose to go the IRIX route, you might look into the Origin 300 line, from what I hear its supposed to be the next step for the 200 series.
If you are still considering, and need any real-life experince feedback or have any questions, drop me an email.
Re:Fluent Runs on Linux??? (Score:2, Informative)
Parallelize or paralyse. It's your choice. (Score:2, Interesting)
Seriously though, it is easy to take a good parallel code and murder it by going for a Beowulf option. I assume that your CFD code is a heavilly parallel beast. From what I have seen from the links available here [beowulf.org] one must be very careful about the relative importance of the node communication bandwidth and the node latency on the code in question.
In my opinion this decision boils down to the following:
How interdependent is the parallel code? If the code is trivially parallel, as many of mine are, then a cluster of the cheapest ix86 machines running Linux is an excellent bet.
Are you certain that the code will work on the platforms in question? Perhaps invest a couple of kiloquid (thousand pounds) or so in a few ix86 machines for a feasibility study, if it doesn't then you have cut your losses and have a few machines to play Quake on ;-). For the ultra cheap option, if your Prof is unhappy about gambing any dosh (money) on buying new stuff, stick Linux on the NT machines and try those.
How far through your PhD are you? Quite seriously, if you're anything like me you may find that you end up spending most of your time tweaking computers rather than doing physics. Building your own supercomputer could easilly suck your entire PhD down the time toilet.
I expect you have done this already but ask the people at Fluent Inc. what experience they have on parallelizing their codes.
Finally, I would say while the support you may get from commercial *NIX vendors is generally good it can also be very expensive. While this is not necessarilly a problem in a commercial datacentre environment, in academia it can be crippling. If you and your co-workers are willing to expend time and effort into trying to get the code working on a Beowulf style cluster it could well be a good long term investment.
Good luck!
P.S. When I checked this post, www.beowulf.org was down so you may want to check the google cache [google.com].
-edRe:Parallelize or paralyse. It's your choice. (Score:1)
Maybe they should have used a beowulf cluster of webservers with load balancing and fault tolerance...(configured of course by natalie portman with a pants-load of hot grits, and her penis bird assistant).
Sorry - didnt have the will power to resist.
Re: (Score:2)
It's not just computing power... (Score:1)
I'm assuming with the Intel solution, you're planning on going with a cluster of some sort. You run into the problem of the already lower memory-CPU bandwidth of the Intel architecture, and making it worse by making it transfer things over the network interconnect. If it's an easily divisible problem (once again - not an expert), this may not be a big deal. Or it could be a huge one.
Another thing: SGIs are used for this for a reason. As some other people have said, ask your software vendor what they would recommend. Sure, the Intel may give you more computation-for-price, but there are things other than raw CPU power.
Finally, it comes to level of mathematical precision. If you need 64-bit precision, a 32-bit processor isn't going to do it as quickly as something with a 64-bit native integer type. (Although, I may be wrong about the exact architecture of the MIPS - it may just be a 64-bit address bus.)
Anyway, in closing - talk to your vendor. They'd know best.
Look at the US National Labs experience (Score:1)
We have traditionally gone with IBM hardware (RS6000) running AIX. I personally hate AIX for its crappy development environment (shared libraries in particular are completely broken). However, IBM has done a decent job with inter-processor communication and IO.
We also have some 32-128 processor SGI Onyx (Origin 2k?) systems running IRIX. I personally love them, but we do *not* typically use them for CFD, but for visualization of results (yes, in parallel). Part of their beauty is that they are SMP machines, so codes which don't use MPI but do use threads work great on them. The IBMs, by contrast, are 4-way (or more recently 16-way) parallel per node, but you are forced to so cross-box communication a la MPI for more than that.
However, we are looking into a 60-teraflop machine right now. It could be a big IBM, but at the current hardware speeds, that's about 50000 processors (!). One realistic option is a massive Intel/Linux (Lintel?
The current problem is that no P4 systems are more than 2-way SMP, so the interprocessor communication will bog you down much faster. You can do something like 4-way PIII's cheaper at the sacrifice of single-processor speed. It depends on how much the code you use depends on the interprocessor communication vs pure processor speed.
You can use something like a Quadrics interconnect to get really good inter-box communication speed, but those only scale to 128 nodes without extra work, and even then I only know they support 256 nodes.
This sounds like you are doing something on a much smaller scale for sure, but if your Fluent code is very parallizable, go with the cheap Linux cluster. If you need interprocessor speed, you can do the same as long as you get a good interconnect. Except for the problems of the more do-it yourself style support, it will cost you far less for the same number of FLOPS.
If it is possible to get a larger SMP machine for your money and Fluent is only threaded, go the SGI route, or maybe a single node 16-way IBM, to get the most processors within a node that you can.
Selecting Computing Hardware for CFD Research? (Score:1)
Simple (Score:2)
Second, high-end systems have x86 beat to hell when it comes to expandability. I have a 3-4 year old DEC Alpha 500MHz system here, which originally only had about 64 MB of RAM (Large at the time) but will support up to 1.5GB total, and this from what originally was a Low-End Alpha Workstation.
Reliability should be a big concern as well. I've used x86 systems from just about every major manufacturer, and the quality, reliability, and stability just can't compare with _real_ hardware.
Personally, I prefer Alphas when performance is important, but I'd definately take SGI over an Intel setup. Of course, Apple makes some great hardware (I personally don't care for any version of Mac OS) and most multimedia software is designed for Mac OS anyhow.
Summary, I miss DEC, and SGI/MIPS is better than Intel even though the numbers might seem to indicate otherwise.