How Would You Benchmark an IT/IS Department? 144
ferretworks asks: "Our IS/IT department has been asked by our CEO to find a way to benchmark ourselves against IS/IT departments from other companies with similar technologies (none specific). This sounds like an innocent enough request, but diving into it has made me realize that this is, not necessarily undiscovered country, but a desolate one and rarely visited. So, my poll to the community is: In your Opinion, what is best way to benchmark an IS/IT department and what categories/sub categories would you base your judgment and ratings on?"
Talk about internal benefits first (Score:5, Insightful)
Downtime (Score:5, Insightful)
"Email was down about 25 times for up to a day over the last year. This is because we don't have to budget to buy a redundant system. If you give us more money, we can increase uptime." etc etc.
Survey the users (Score:4, Insightful)
my 2 cents... (Score:2, Insightful)
Having worked on both sides of the IT vs Business user / programmer fence, I think any measure of the success of IT needs to include some form of customer satisfaction. This is important because IT's customers tend to be internal and most customer satisfaction queries are focused on external customers.
I've seen battle lines drawn between IT and everyone else, and nobody wins because energies are focused on battling with the other side instead of finding ways to help the company make money. While I understand that IT's responsibility is to the infrastructure, it should be done in a way that makes it easy for their customers (the rest of the company) to do their jobs quickly and efficiently.
Get Ready to be Outsourced (Score:2, Insightful)
Been there, done that, don't have the t-shirt.
Seriously, they are looking at cheaping labor costs in India/China/Godknowswhat3rdworldhellhole to send your jobs to. If they are asking for a lot of procedures on your work and maps on how it gets done, leave faster.
How do you benchmark other Operations departments? (Score:4, Insightful)
Here's your benchmark... (Score:5, Insightful)
(This method of managerial interaction is derived from the groundbreaking book "Shut the fuck up" by Dr Denis Leary)
Seriously, it's not an "innocent" request. It never is. First, they just want to know how you compare. Then, they want to know why you're not the absolute bestest in the whole universe. Then they compare you against departments with fewer people. "Hmm, seems we could get comparable results with 2 fewer bodies". Then each employee is evaluating their specific performance trying to justify their job. Or you're filling out "Time code sheets" to tell them what you do in each 6 minute block of time. Pretty soon you're hearing about their buddy Steve, who has his entire office and web presence run by one part time summer intern who happily works for $6/hour plus tips.
Evaluate your performance based on internal expectations. You want great uptime on the web server? Why, we were only down for 2 hours last month. Compared to previous months, that's great! You want quick response to employee problems? Our average response time for properly filed tickets was down %4 compared to last quarter. That's the way you evaluate. If you start giving these fuckers examples of how other companies are doing, they'll cherry-pick and start challenging you to match. "Why, there's a place in Wisconsin that only has 1 tech covering 400 people! I don't know what this 'Wyse terminal' thing is, but surely if just one guy can cover all those people, we're overstaffed here!"
I know it seems like it could be good. "Why, we're outperforming most other companies, surely they'll see this and use it as criteria to give us better raises/more benefits/better perks". NO. NO, they won't. Just NO. Come review time, you could be leading 90% of the field, and they'll go on about the top 10% and how those guys earn less than you, so they shouldn't give you a raise. Or they'll go on about how other companies have their helpdesk techs do server support too, so they don't need your $80k server admin. If you're underpaid, and everyone else makes more than you, you can show them that too, and they'll nod in an interested fashion, they'll hum and haw, and they'll end up giving you some bullshit excuse about budget constraints. If you make more than other people, expect a pay freeze or cut. Even if fall in the middle of the bell curve, expect them to find a reason to lump you with the folks on the bottom of the curve.
Management doesn't understand you or what you do. They're scared of you. They want to control you and marginalize you so they can eliminate you. Don't let them, don't help them, don't give them any excuse. No matter how much you train them, or explain to them, or draw them little fucking flowcharts in Visio with pretty colours, what you do is still voodoo to them. You push keys on the magic box, typing 10x faster than they can, and you get it to do things they couldn't even dream of, and you do it really easily, and that makes them feel dumb. And just like stupid bullies on the playground, they want to get back at you, subconciously perhaps, but they still do. They want to sit there across from you, at their big fancy desks in their spotless shiny offices, because that's their one time to feel superior to you, to be in control of you, unlike all the other times when you're the magician who just makes miracles happen.
Don't fall for it. Rephrase their request, turn it around on them, avoid it at all costs. Does your accounting department have to compare themselves to other businesses? Do your managers have to go around finding out how other managers in other companies are doing? Fuck no. Don't let them treat you any different than any other department. They're just doing this because they don't understand what you do. Don't explain it to them, just give them numbers based on internal statistics so they can sit back and go "phew, IT is performing well, I'm happy".
Just remember folks, any time anyone in management opens their mouth, echo these words in your head:
"It's a trap!"
Quality of services (Score:1, Insightful)
So, for quality of services, lots of downtime is bad.
But just about as bad are setups that are clunky to use. I've seen some custom apps that are pretty, but if the person has to enter forms, not being able to tab is bad. Cancel buttons that make the app pop up an error box instead of cancelling, and worst of all, do what you have to do to make the app fast. I use some clunky custom cash register app where I work, the database it uses is crap so it's steadily gotten slower over time. Soemtimes now it'll take over 15 seconds to add an item to the receipt, other random times it's seemingly instaneous.
Security. An IT dept. isn't top notch if it's left SSNs or credit card numbers up on a web site even if they're really nice otherwise 8-). Also, I don't think it's so good if there's systems failuers due to viruses etc. Beyond that, it might be hard to tell anything, I suppose most companies are tight-lipped with security statistics. If security goes beyond being secure to being a nuisance, that's a problem, which is why I didn't list security first.
Cost. If several departments have happy users, and good security, then the one that does it cheapest wins. If two departments provide similar service services, but one took 5x as much hardware (per user) to keep everything snappy, then the other department should get props for making things efficent rather than throwing more computers at the problem. This of course also saves electricity and cooling costs. Spending $ billions on an app should put them right out of the running, even if the app ends up being extra nice.
Another typical "not what you asked" response (Score:1, Insightful)
Worst. Idea. Ever. (Score:3, Insightful)
Why would you do such a thing to yourself? With those metrics, the absolute very best you could do is, 'we didn't cost the company anything...other than salaries and resources.'
Why not use metrics/goals that actually help justify your job? Why not use metrics to show how much IT is adding to the bottom line? Seriously.
BTW, before you go out and waste resources on another system you don't understand, how about learning how to handle what you have? (Hint: It's not downtown; it's 'scheduled maintenance.')
Re:Here's your benchmark... (Score:3, Insightful)
When I first started working at my company there were no solid metrics in place. We had a backup system not being tested. We had a NAS system growing out of control. What happens when we started placing metrics in place? Things begin to change.
I recall someone wondering why the director would never approve a project until I went to the director with numbers and history (DATA!) and a 6 month fight turned into a quick chat with the result in a decision.
Performance and time spent has to be measured because this is a cost to the company and yourself.
When the IT department performs terribly on it's help desk support you can end up alienating your customers - the company - and always make it feel that contacting help desk is a waste of time.
If some wise guy exec decides to make the quick decision to replace the entire department (although you outperform 70-90% of IT departments in the country) with outsourcing. They'll never find out the impact money/time wise until is too late. Now, you're out of a job yet you were the best.
Metrics can be your friend.
Re:Here's your benchmark... (Score:3, Insightful)
In my experience, all functional areas are compared to other businesses. It's what benchmarking is all about and has been all the rage for at least twenty years now. IT is no exception. Sometimes it can even be useful! If you're doing well, then it can help demonstrate you're doing well. If not, then it can act as an incentive to improve things.
Sure, sometimes it can be used for political, even malicious, purposes; but in those instances the corporate culture is often dysfunctional enough that that's just an excuse to screw you. If it's not benchmarking, it'll be something else.
If you make more than other people, then surely you'd be able to justify that? Maybe by benchmarking your IT department's performance against others?
In my experience if you fall in the middle of the bell curve, they'll ask the question "Why aren't we ahead of the curve?" The answer can lead to a productive discussion about what exactly the business needs to invest in in order to make IT performance even better.
And this embodies the attitude that wants to maintain the priesthood at all cost - keep IT mysterious and scary, rather than enlightening the business
And update your CV while you're doing that, cause you'll be needing it soon.
Time to get your resume in order (Score:3, Insightful)
1: Management is clueless as to what the IT staff actually does. 2: Management is looking for ways to reduce headcount.
When you see this nightmare raise its head at your place of employment - be worried. Especially if you're at the top of the pay scale - you'll go first.
If you see this at the same time as there's handwringing about the cost of health benefits, don't just get your resume in order, start looking for another job. Even if your job survives the reorg that's just around the corner, you'll be working twice as hard.
What a load of crock. (Score:3, Insightful)
Look at it this way:
If you are better than the rest, you should make sure everyone knows about it and ask for a payrise.
If you are doing worse than others, then you better improve.
It's the latter situation that most people such as the parent end up fearing (and hence ranting about). Simply, they are scared to change their current work practices - they are content doing a mediocre job. They are the sort who spend enormous effort trying to maintain the status quo whilst the rest of the world improves around them
Uptime is assumed. Get strategic. (Score:4, Insightful)
Either you're reliable or you're not. But reliability, while important, is only part of the picture.
For struggling, barely competent IT guys, maybe reliability is all they can hope to achieve, but let's assume that's not you and yours. Let's assume you've got reliability down pat. Why do I make this blithe assumption? Because we can purchase reliability off the shelf today for not very much money. Windows XP boxes on programmers desks regularly go 2-3 weeks without requiring a reboot. Servers can run even longer if properly configured, fed and cared for, even running Windows.
So, where do we go from 99.999% reliable with an IT department? We start consulting with our business managers. We work with them as a partner to help select and adopt technology that will make the business more efficient and/or more effective. And we benchmark these efforts by measuring the productivity of the business function before and after the new technology/system is deployed. This is where we show the strategic value of IT. Not in uptime.
And, we take into account cost. If we can get 99.999% reliable and your competitors IT group can get 99.999% reliable, the budget becomes the tiebreaker and the winner is the one that does it for less money.
For instance, if a business is indexing large document sets for the litigation support market and we can figure out how to move that function from Dallas to India thus cutting labor costs by 60% without a loss in quality or responsiveness by cleverly deploying networking technology and low-end PC's into Madras, we have provided an easily quantifiable cost structure improvement which can give our company an important competitive advantage. This is an example of the big payout from IT. The other big one is quality improvement. And the really rare one is opening up a whole new way of doing business...disruptive technology.
Warning - opinionated opinion below. (Score:5, Insightful)
Number one thing I'd like to get out of the way is that the people who qualify this request as a waste of time are idiots who shouldn't be in an IT department. I've been to too many places where no one can tell me what they're running, where the servers are, what patch levels they are at, which machine is up or down and what they need to keep pace with growing demand (your business IS growing, right?). IT departments like these are the root cause for bloated IT initiatives, downed web servers and crappy customer service.
Number two is the realization that the IT department is in the business of the keeping the rest of the business up and running. This means two things: your responsibility is to keep the rest of the company happy and productive, and you get to charge them (system doesn't matter, as long as you keep track of what it means to service 200 requests a day for lost passwords) when you perform work to keep someones servers up, the mail flowing and the network lit.
Once you understand the place of IT in a business, metrics are easy to come by:
- how well you're doing is measured by how available your servers and your apps are.
- how efficient you are is measured by how much money you save the rest of the business in avoided downtimes and increased productivity.
- Bonus if you keep track of how long it takes to service requests and what their resolution was.
- Extra Bonus if you can do performance forecasting and figure out when you need to expand your infrastructure.
The performance of an IT department is NOT measured by any of the following:
- budgets of IT departments in other companies
- how many new products you've rolled out
- how many people you fired (or hired)
Anybody who suggets those needs to be smacked around and told not to speak in public anymore.
With that in mind, there are a million products to track your statistics. My company will be happy to quote you anything from a 4-figure to a 10-figure deal for this. There are open-source tools that will do similar things for free (though in my opinion, you get what you pay for). There are in-house and hosted answers to any of these questions. But the one thing you need to remember is that you need to know the answers to the CEOs questions about what the status of your machines and your people is. If you can tell him that your hardware uptime runs at 99.99% for mission-critical servers (the Oracle RAC that holds your financial information, for example) and your app availability is 99.7% during business hours, that it takes you 4 hours to respond to a priority 1 request and 3 days to a priority 3 request, and that based on current response time trends, you need to double the processing power and database space in 6 months, you're golden.
If you can't tell your CEO these things, you will be replaced by someone who can. Or even worse, your job will be outsourced to my company, and I get to work out these things while you're flipping burgers. Yeah, I'm being a jerk. That's because I'm flabbergasted at some of the comments (and their moderation) and how their authors still have a job. IT metrics are simple, and don't have to start with complex SLA measurements and other crap. You can start with a basic ping monitor of your critical servers, and go from there. But for heaven's sake, do something. You'll be a hero if you play it right.
Oh, and just to repeat - do not benchmark yourself against other companies. You don't have access to valid data, and you won't find an identical business. Instead, find out what your other departments need from you, and benchmark from there.
Re:Here's your benchmark... (Score:2, Insightful)
Re:Survey the users (Score:2, Insightful)
It can't hurt to have your report include a count of preventative maintenance issues and a provisional costs saved, including parts, downtime (possibly scored as a proportion of quartely working hours / quaterly revenue) and the cost of data loss. This last one will require that you know what's going on within the company (go and chat about what each project lead is doing: flatter them by listening), who's tendering contracts or running an important project, so you can weight the value of their data in your backup system on top of the standard value of e-mail and filestore.
Re:Here's your benchmark... (Score:2, Insightful)
I got a benchmark for ya. "Are things running well?" Yes? Yes they are? Then shut the fuck up and stop asking me to waste time comparing myself to other people.
Exactly. Because an organization should never strive for better.
Seriously, it's not an "innocent" request. It never is. First, they just want to know how you compare. Then, they want to know why you're not the absolute bestest in the whole universe. Then they compare you against departments with fewer people. "Hmm, seems we could get comparable results with 2 fewer bodies". Then each employee is evaluating their specific performance trying to justify their job. Or you're filling out "Time code sheets" to tell them what you do in each 6 minute block of time. Pretty soon you're hearing about their buddy Steve, who has his entire office and web presence run by one part time summer intern who happily works for $6/hour plus tips.
There no doubt are places like this, and you would do well to exit such an organization sooner than later. Trying to avoid accountability or fighting audits only make you appear aware of and afraid of your own incompetence. If you are providing no benefit above what Steve and his summer intern can do for a tenth of your salary, then management needs to act. If you can't explain why going with Steve and his summer intern is a bad idea, you have left it to management to decide with the information they have. If management makes a stupid decision in selecting Steve and his summer intern, the organization will eventually pay for it and you can find a better job elsewhere.
Evaluate your performance based on internal expectations. You want great uptime on the web server? Why, we were only down for 2 hours last month. Compared to previous months, that's great! You want quick response to employee problems? Our average response time for properly filed tickets was down %4 compared to last quarter. That's the way you evaluate.
This is pretty common in IT, but it often leads to a bloated and stagnant IT structure and itself leads to outsourcing. A well-managed IT department in an organization with competent leadership rarely will be outsourced, especially in the smaller to mid-size environments (yes, bits and pieces will be as it makes sense to do so, but I'm talking about the whole shooting match). I can't count the number of companies that I've seen outsource IT simply as a clean-slate housecleaning move, because someone finally realized that IT was grotesquely oversized, wrapped in red-tape, and lagging behind the industry in technology.
If you start giving these fuckers examples of how other companies are doing, they'll cherry-pick and start challenging you to match. "Why, there's a place in Wisconsin that only has 1 tech covering 400 people! I don't know what this 'Wyse terminal' thing is, but surely if just one guy can cover all those people, we're overstaffed here!"
And you ought to be able to explain why that example isn't relevant to your place of employment or why it would be a bad idea.
I know it seems like it could be good. "Why, we're outperforming most other companies, surely they'll see this and use it as criteria to give us better raises/more benefits/better perks". NO. NO, they won't. Just NO. Come review time, you could be leading 90% of the field, and they'll go on about the top 10% and how those guys earn less than you, so they shouldn't give you a raise. Or they'll go on about how other companies have their helpdesk techs do server support too, so they don't need your $80k server admin. If you're underpaid, and everyone else makes more than you, you can show them that too, and they'll nod in an interested fashion, they'll hum and haw, and they'll end up giving you some bullshit excuse about budget constraint