How Well Does Windows Cluster? 665
cascadefx asks: "I work for a mid-sized mid-western university. One of our departments has started up a small Beowulf cluster research project that he hopes to grow over time. At the moment, the thing is incredibly weak... but it is running on old hardware and is basically used for dog and pony shows to get more funding and hopefully donations of higher-end systems. It runs Linux and works, it is just not anything to write home about. Here's the problem: my understanding is that an MS rep asked what it would take to get them to switch to a Microsoft cluster. Is this possible? Are there MS clusters that do what Beowulf clusters are capable of? I thought MS clusters were for load balancing, not computation... which is the hoped-for goal of this project. Can the Slashdot crowd offer some advice? If there are MS clusters, comparisons of the capabilities would be welcome." One has to only go as far as Microsoft's site to see its current attempt at clustering, but what is the real story. Have any of you had a chance to pit a Linux Beowulf cluster against one from Microsoft? How did they compare?
Point? (Score:5, Interesting)
1. Not having to buy an licence for each machine
2. Having an infinitely configurable system (meaning that you can load as much or as little of the OS and libraries as you want/need)
3. The use of high quality, low/no cost development tools.
It seems to me as though running a cluster on 2k would possibly be easier (point and click) but less efficient.
I believe you're correct... (Score:3, Interesting)
Clustering Exchange (Score:1, Interesting)
We run in what's called an active/active cluster. But for the most part, the machines are just sharing responsibility. In most windows clusters the other server is just sitting there waiting for the first to fail. They share two drives (on seperate hardware either fibre channel or SCSI attached) and when it fails, the other server picks up those drives. Windows writes data to those drives so when the other server picks it up it can import that data and pick up where the other server left off.
Clustering is PERFECT for fault tolerance. It is useless for most intents and purposes for load balancing. If you want load balancing you can use NLBS, but it just plain sucks, and never works right.
Beowulf (Score:5, Interesting)
http://www.windowsclusters.org/projects.htm gives a list of current Windows clusters.
Finally, are you out of your tiny little mind? I wonder why M$ is so keen to help. There is no such thing as a free lunch, espically from M$.
Re:BSOD!! (Score:2, Interesting)
when they Blue Screen?????
A Cluster Fuck?
(if you diden't know what it ment, then you woulden't be offended)
MS AppCenter server (Score:2, Interesting)
However there is a server solution I saw demoed at a MS DPS I attended called Application Center [microsoft.com]. It allows you to manage your cluster and distributes workloads throughout the cluster.
Now, I'm not sure if you NEED this to take advantage of Windows 2000 clustering. The last time I worked with a MS cluster was under NT 4 and it was failover only. The load balancing was "faked" by a router that would just alternate which server the request was sent to.
(insert "yeah but MS is evil" comment here)
(insert "yeah but Linux Beowulf clusters cost less" comment here)
(insert "yeah but who wants to have to reboot your cluster all the time" comment here)
(insert "I wish the sigs were longer because that's a really good quote by Richard Feynman" comment here)
Re:Licensing/Reliablity (Score:5, Interesting)
I can only hope MS's poor performance will make them switch.
Windows Clustering (Score:3, Interesting)
Haven't seen the reported "bsod round table" where one machine crashes, shortly followed by another and another. The problems we have seen is a single machine bsods, and the other machines in the cluster don't realize it's down.
If your already in the MS camp, it will work, it look at other solutions. I think they will be more cost effective.
Seen in list of software included... (Score:2, Interesting)
* PLAPACK package (open source software)
heh.
-JT
Well, with Condor... (Score:2, Interesting)
I've been looking at this a lot myself now, as I'm also building a cluster for use in a computational bio lab at Florida State. It certainly seems that Linux is the only way to go right now. In case anyone cares, my cluster right now is 16 nodes of:
Tyan S2460 with 2 Athlon MP1800+ processors per node
1 gig PC2100 RAM per node
20 gig 7200 RPM Maxtor HD
3Com Gigabit over copper Ethernet
low-end cheapass video and floppy, etc.
All in these really nice rack cases, with a big black 2001 monolith-esque rolling rack to shove it all around in. It cost just about $26,000 to build so far, but the plans are to expand it to as many as 512 nodes within the next year or so. Whee!
Clustered MPEG encoding with TMPGenc? (Score:2, Interesting)
That said, with the three computers I have at my place (a p3 desktop, a celeron I use as a low grade server, and my p3 notebook) I'd love to be able to set up a cluster for encoding. Such operations will be the killer app for clustered systems IMHO.
Limits seem to be the key (Score:3, Interesting)
While I haven't been near a Microsoft Cluster in a while, I do remember a couple of things that really stand out about them:
The number of systems able to be part of the cluster is severely limited. At the time, it was limited to 2, but I'm pretty sure that has increased to a somewhat larger single digit number.
The number of applications available to run on the cluster is just as severely limited. Again at the time, there were exactly zero applications, but I know that there is at least one (Exchange) now.
Given the limitations of what uses you can put an MS cluster to, I wouldn't bother with it in the first place.
Windows clustering a la Microsoft (Score:2, Interesting)
I followed the link to Microsoft's clustering solution. Another link took me to a free evaluation page - the package includes:
All for $7.95 shipping and handling!
May be a cheap way to get a few Win2K licenses?
why not macintosh cluster? (Score:2, Interesting)
ACME (Score:2, Interesting)
Re:Beowulf (Score:4, Interesting)
Why do they do this? Simple: it's a long-term marketing trick (and a cheap writeoff.) Train the students with Windows, Office, Visual Studio, MSSQL Server, IIS, et bloody cetera, and that's what they'll know when they get out into the working world. Companies that already use M$ shit will have an easier time hiring new people. Companies that are deciding on new systems will have people in their IT dept. who say, "Well, I don't know anything about Linux/Solaris/gcc/Apache/whatever, but I know all about NT and VC++ and IIS," and may well make multimillion-dollar purchasing decisions on that basis. It's not hard to figure out.
Re:You're running on old hardware right? (Score:3, Interesting)
/Brian
Here's what to tell them. (Score:5, Interesting)
So, take the MS reps through the operation, tell them the capabilities. Ask them if they can meet or exceed them. If they say "Yes", you're either not using the real capabilities of your Linux machines, or they're lying.
steve
Re:That's easy/ Wish I could. (Score:3, Interesting)
AC3 - High Performance Computing (Score:2, Interesting)
There are numerous machines, such as the 256 CPU Veclocity 1, that run MPI-Pro over MyraNet(?), that was one of the 500 fastest computers [cornell.edu].
Windows is a very viable and high performance solution for running scientific parallel application, and you should order the $8.00 evaluation kit [microsoft.com] from MS and check it out for yourself.
I've developed for some of these systems, and have been very impressed. I've worked with Linux clusters too, but only on older, weaker machines, so it would not be fair to compare the two.
(Btw. all opinions here are my own, and in no way should be construed as those of Cornell or the TC).
One thing you might want to consider is administration time, scientists, who are already annoyed that programming destracts them from their real work, might not want to devote the time and effort to learn to and administrate all those linux boxes.
Anyway, if the MS rep is very eager, he might offer you some great deals. MS is very eager to be taken seriously as an HPC option.
Re:Well, with Condor... (Score:2, Interesting)
For REAL scientific stuff, they use Linux.
Re:Licensing (Score:2, Interesting)
Re:Take a look at (Score:2, Interesting)
The whois lists the University of Southampton as the owners, but the whole page smacks of Microsoft PR.
Re:first post - no way (Score:2, Interesting)
However, with Windows and windows software you often do not have that option. Is the management of processes entirely handled at the OS level? It seems like that might be somewhat inefficent, as opposed to having the program handle at least part of the management. If not, are there ANY aplications that are designed for a Microsoft clustering environment?
Simply... (Score:5, Interesting)
Then, point out the scads of Beowulf clusters and Linux/Unix based systems.
Finally, inform the rep and your management that you've chosen to use the more cost effective, higher performance and standardized choice...Unix.
If management resists further, do a cost analysis. That'll convince them.
299,792,458 m/s...not just a good idea, its the law!
I can always count on Ask Slashdot... (Score:5, Interesting)
Right now a coworker and I are looking at pricing and configuring a fault-tolerant cluster for a client who runs Windows 2000 and Exchange 2000. They're a bit paranoid, so they've decided they want a cluster. We've tried to educate them on exactly what a Microsoft cluster can and can't do, so it's difficult to understand exactly what they want (basically an entire network exactly like Microsoft's own, but for $1000).
Pricing on a two system cluster is around $50,000. Buying two copies of Exchange and Windows Advanced Server will total $20,000. Then there's the hardware costs. For our client, they've specifically requested this, so they're ready to pay.
My question to Whamo is are they really taking the Microsoft rep seriously? If they have to pay software costs for their new cluster that's going to mean two things: either buying less CPUs to add to the cluster, or not doing the project at all, because just the software will put them over budget. With Advanced Server running somewhere around $4000 that's a lot per machine when Linux costs at most $5 to burn a CD after downloading it via the university's T1/T3/etc. Whamo says "it is running on old hardware and is basically used for dog and pony shows to get more funding and hopefully donations of higher-end systems" and to me that is your answer. If you can't afford the hardware you can't afford to buy Microsoft's software...
Also, there's MOSIX [mosix.com] as well, but I don't have much experience with MOSIX and thus cannot comment on it.
Cheap Way To Get VisualC++ Std (Score:2, Interesting)
If you can cope without the optimising compiler of the professional edition (which appears to make little difference which optimisation method you choose), $7.95 (+$1.95 to get it to the UK) for Visual C++ seems like a bargain.
Steve.
Re:Point? (Score:5, Interesting)
Here's the problem: my understanding is that an MS rep asked what it would take to get them to switch to a Microsoft cluster. Is this possible?
What would it take to switch? Well, you go to the rep and ask for X P4 1.8 GHz desktops, X licenses of Win2k, Y trips to Wicrosoft Clustering Training Junket in sunny Bermuda.
What does MS get in return? A department of trained CS students and professors who, when someone asks about distributed computations, will respond "Microsoft" instead of "Linux". And when those students enter the real world and the PHB (who wants MS anyway) asks about clustering the answer will be "Microsoft".
Remember, Linux earns mindshare, but Microsoft buys it... and it is almost always easier to buy someones loyalty than to earn it.
Re:money, for one thing (Score:2, Interesting)
1 scenario is to use the built in clustering technology in windows advanced server/datacenter. You must license each machine in the cluster, and it's not meant for distributing computing, just to provide a hot standby. Academic pricing is pretty aggressive, but the clustering only works so-so in the environments I have seen it deployed in.
the second scenario requires that you still buy windows server licenses (but not datacenter, which is much more expensive than plain server), and then use a third party clustering app like veritas cluster server, or Stonesoft's products.
I still don't think you're getting the same type of functionality as you get from beowolf, but these might be alternatives. The original poster didn't describe what kinds of apps he wants to eventually use.
Re:money, for one thing (Score:3, Interesting)
Re:first post - no way (Score:4, Interesting)
The AC3 folks at cornell [cornell.edu] have done quite a bit with these windows clusters. I guess the parallel Matlab is pretty nifty, but there's no reason any of this stuff couldn't be done on a more mature platform.
Personally, my biggest turnoff is the fact that you need KVM switches wired up to each node...well that and the overhead of running the bloatware that is win2k. Compared to a 256 node headless linux cluster we built this just sucks. Hard.
Re:Ask for modifiable code and no injurous NDAs (Score:3, Interesting)
2. This is a given, except when closed source would be revealed publicly (which is also a given).
3. The very idea is ridiculous.
4. Pretty much a given as well (free, that is).
Re:first post - no way (VMS vs NT) (Score:2, Interesting)
My experience is with Windows NT 4 Server Enterprise Edition. MS chose to use a "shared nothing" implementation - which, IMHO, means they don't do clustering. There is no cluster-wide locking, software runs on one node at a time, there was a limit of two nodes, and it required a shared disk.
ask for references (Score:1, Interesting)
If someone else has 'successfully' implemented one, you can find out for how large a cluster, what classes of problems it's solving, and how long it stays up (oh, and how many times they've had someone break in). Presumably administrators of an existing installation at a university won't egregiously lie to you
Then find three success stories at universities of Beowulf clusters and similar information for them. Side-by-siding these for your management should make the point verifiably clear. Ask the Microsoft sales rep for these contacts (don't settle for grossly postprocessed 'success stories') and maybe he'll disappear.
I think the point must be made to management that there is no better proof of concept than a working implementation that matches or comes close to your needs.
Not so much a wolf "pack" but wolf... (Score:2, Interesting)
"It's not really accurate to refer to this arrangement of servers as a wolf "pack". A pack is an organized group with a leader working towards a defined goal using a plan with a visible, known structure. These servers just sort of hang out together. It would be more accurate to call them "wolf buddies"."
Microsoft didn't get to put their software on the solution but I did tend to put more credence in what that particular engineer would tell me about the capablities of Microsoft products.