What Do You Look For in a Big Iron Review? 262
ValourX writes "We're starting to write more reviews of enterprise-class hardware and software and although we've done pretty well with our reviews, the high-end products are a lot trickier when it comes to testing and evaluation. Obviously it is not possible to build an enterprise-grade 'your neck is on the line' production environment just for writing reviews, but maybe we can do something smaller, just for testing purposes. What do you as an IT professional want to read in a review for a server OS or a high-speed switch, or a big iron server or proprietary workstation? What tests should we run? What results and feature comparisons are going to be most meaningful to you?"
Real numbers (Score:2, Insightful)
I would like to see what they get in a regular user's hands.
Phil
Load Test (Score:3, Insightful)
Who paid for it (Score:5, Insightful)
Who asked for it and more importantly did anyone pay for it either directly or indirectly.
Cost Analysis (Score:2, Insightful)
If the cost of this "new" server is 5X more expensive (as a package) than another system that gives you the same functionality and comparable performance then knowing that this alternative exists and what the performance / price difference is would be valuable.
Re:Mostly... (Score:2, Insightful)
No shit?
Not to drag this even further OT than it already is, but I already knew that. However, when moderating, you should moderate what you think the thing is. If it's funny, moderate it funny. If it's off topic, moderate it off topic. You shouldn't be moderating just to add karma to a poster.
If you are that worried about somebody else's Karma, you obviously don't have enough to do. Try to turn the computer off once in a while and maybe take a nice walk in the woods.
Trust in the vendor is our #1 concern. (Score:3, Insightful)
Nothing is more annoying than if you buy a big frame and then you find out that a silly little piece of software is no longer maintained. Or like HP announced today, that they are once again changing their HP-UX roadmap and once again proved that they can't be taken seriously if they predict anything further out than 3 months.
It all comes down to the simple fact that in the end, almost all of the big boxes are the same to the application. Sure, some have hard and some softpartitioning. Sure, you have different cpus, memory latencies and whatever - to the app it is just a bunch of system calls. But in the end, if you can't run your app on it, its useless, no matter how fast, redundant or whatever it is. We have completely moved away from selecting the box by its hardware properties. They are all sufficiently redundant and whatever. We go purely by how well the software we need to run is supported on the OS and if they have a roadmap that can be trusted.
Peter.
sorry to be blunt, but... (Score:5, Insightful)
This is going to be harsh, but you need to hear it.
Obviously it is not possible to build an enterprise-grade 'your neck is on the line' production environment just for writing reviews
In order for the review to be accurate, that's how it has to be tested. Evaluating enterprise equipment in a non-enterprise environment with people who have no enterprise experience is pretty much worthless...and you're not going to fool anyone.
There's also no market for this sort of thing. Equipment on that level is bought because of high level executive briefings, price negotiations, migration options, and politics. Why? Because the market is so cutthroat and all the features that matter are there. The decisions are not made on whether or not a power cord was included, it was easy to unpack, the manuals were clear, how well built it looks, and how it did on SysMark SuperServerSimulator 2005...which is about the only thing all you 2-guys-with-a-webserver "hardware review" sites know how to do.
Further- often when a hardware vendor wants to get a contract, they provide a unit for evaluation.
On top of that, the major analyst firms already fill what little niche there is, and they have really big names 90% of the important people with Nice Shoes will recognize, which means even if that analyst is wrong, the decision to go with their recommendation is justifiable and won't get the Nice Shoes person fired. You'd be lucky if .01% recognized your name, much less trusted it. "Jones! Why does our website keep crashing?" "Well, we're having a lot of hardware problems." "Why did we go with ABC for our servers?" "Oh, XYZhardware.com said they were the best." "Jones, clean out your desk."
So...sorry, there's no market for what you're trying to do, and you don't have the means to do it.
Does the company support linux on it (Score:2, Insightful)
to big dog servers and I just bought two more today.
1. Does the hardware vendor support linux or just pay lip service to it.
2. Can I get it without a os loaded or can I get it
preloaded. If it so much as comes with a oem windows
cd in the box I will ship it back.
3. Have they pissed off the community lately.
I just bought over 40k in servers today and guess who did not see a penny of that? DELL
Reviews are insignificant in this setting. (Score:3, Insightful)
1. Requirements solicitation - figure out what needs we need to fill, be it wifi net access, a file server, etc
2. Vendor research - contact the usual suspects in the field (networking, big iron servers, etc) and arrange for consultation and formal bids to be made. NOTE: this step is skipped ENTIRELY if the company/institution already has a corporate account with a vendor that provides the appropriate services that you require.
3. Formal bidding process - pit the vendors against eachother, it's fun when you get them onsite to demo their gear. Generally vendors will lower prices to sweeten their bid.
4. Award the contract to one of the vendors, or (more likely) have funding denied to you by the beancounters and end up doing a half-assed implementation of what any of the vendors was going to do.
Individual machine or software reviews are a *tiny* part of the process for securing enterprise level hardware/software services.
Re:Not Speed (Score:5, Insightful)
Not a bad idea, but all I see is the manufacturer lowering the maximum specs to any tests will show it 'overachieving'.
What I'd like rated is the support side. My AS/400s self detect hardware problems, phone IBM to report the problem, and a tech is dispatched. The IBM support centre phones me to tell me the system detected a problem, and that a tech is on the way. Usually the tech shows up with parts in hand inside an hour. Before the hardware has caused any downtime! I've never had a catastrophic failure on an AS/400.
Good support, redundant and hot swappable hardware, like RAM, makes for the best big iron. Low to no downtime are just as important as throuput and storage.
Re:Not Speed (Score:3, Insightful)
For Load They should max out the system slightly above the recommended specs and see how well it handles it.
Nah, push it until it falls over and see how it degrades. Stick a line on the graph where the rated capacity is.
Re:Big Iron - Devaluing the Brand (Score:2, Insightful)
My domicile is therefore "big iron" according to your definition.
Maybe I'll set up a time-sharing bureau.
Re:Most important feature (Score:4, Insightful)
slashdot load test
I don't look for reviews, period. (Score:2, Insightful)
Vendors will bend over backwards to get you to buy their big-ticket items. They will generally give you test machines and allow your engineers to hammer away. Those making the purchasing decisions will talk to their engineers, and value their opinions much higher than those of a magazine.
At least thats how it should work, and that is how it does work in top companies who rely on these machines for their entire business.
Reliability (Score:4, Insightful)
Try to install faulty components, see what happens (Experience: even if the manufacturer claims failure tolerance, this is seldom the case).
Check if the iron really runs in the manufacturer's reported maximum temperature and what happens at the temperature plus couple degrees (Experience: Sun boxes keep running, HP/UX boxes immediately shut down).
Check if the system runs itself down gracefully when UPS reports power is out. Cut power entirely, see what happens.
Check if you can administer everything without touching the iron, including shutting the box down and starting it (Lights Out Management).
Re:"WebServer"? (Score:3, Insightful)
VMS is nearly as good; some argue it's better.
Re:Not Speed (Score:3, Insightful)
- How well does the system react to overheating?
- How well does the system react to unexpected shutdown?
- How well does the system react in general?
While big iron often goes into controlled environments (generators, air conditioners, etc.), those systems sometimes fails. Whether their failure causes an inconvenience or a catastrophic problem is a significant question.
For example- does the system detect the heat and shut itself down, or does it overheat and fry itself?
Notification systems- Is the user notified when a fan fails? (Is it via a proprietary protocol or something standard like SNMP?)
User replaceable parts- Can I keep a stock of extra fans in the back room, or does a tech have to come out to replace things that *do go wrong*?
In short, a review should come from the perspective of running a production data center and the issues one faces in this environment:
- What is necessary for high uptimes?
- What is necessary for long lifes?
- What is necessary for solid performance?
-jbn
Just how far from reality (Score:2, Insightful)
Product cross comparison of specifications using iedntical test suite rather than manufacturers 'tuned' suites.
Real world test comparison. How well does the box do it's job when it's doing everything it will do in deployment at once.
Clear breakdown of cost so that all the 'gotchas' like proprietary cards or code that is not included, warranty, spare parts turnaround, ease of diagnosis, actual electric consumption, etc.
Re:Not Speed (Score:3, Insightful)
If you have 100 servers you pay 5 servers/year in maintenance money.
If you just buy new servers and junk the failed ones, you can afford a 5% yearly failure rate (or even higher, if you sell the failed servers for parts). Plus you constantly keep upgrading without keeping outdated CPUs...
Host /. on it and measure marketoid sweat (Score:2, Insightful)
Re:Big Iron - Devaluing the Brand (Score:5, Insightful)
Once you work for an organisation that has had an IT department longer than the mid-1990s (say, a bank where it goes back 40+ years), you'll realise this article about "Big Iron" is a joke.
Where I work, we have several designations for breaking down inventory. Destkop/Wintel (including stand-alone WinOS servers), Mid-Range (Unix, cluster windows, linux), and Transactional ("Big Iron").
E15ks are big, powerful, and excellent enterprise servers. But...even then they're just servers. Their workload is managed differently (e.g. no batch) and their failure/recovery modes are completely different.
Aside from DEC, I mean Compaq, I mean HP Tandems, MVS (Z/OS, aka OS/390) would be the only thing we consider "Big Iron". Calling an E10k "Big Iron" is equivalent to noob-speak in senior IT corporate circles. If we had Crays, that would be in scope of this definition too.