Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Security

How to Measure Security ROI? 64

Posted by Cliff from the difficult-metrics dept.
UM_Maverick asks: "Does anybody out there have any experience measuring Return on Investment for security-related expenditures? For example, if management says that there's $1 million left in the budget, and you can either implement a new customer tracking system that is projected to save $300k per year, or implement a new security technology or process, how do you measure the return on the security spend, and convince them that it's at least worth considering? Googling for 'Measuring Security ROI' seems to just produce a list of articles that say 'Measuring security ROI is difficult.' Does anybody have some more direct experience or information?"
This discussion has been archived. No new comments can be posted.

How to Measure Security ROI? More

How to Measure Security ROI?

Comments Filter:
  • Why not grow it within your infrastructure?

    If you buy a "1 million$" security infrastructure, you WILL miss something. Instead, build the security from the ground up, paired with each node.

    If you have to "pay for it now", you're already too late.

    • If you buy a "1 million$" security infrastructure, you WILL miss something. Instead, build the security from the ground up, paired with each node.

      If you have to "pay for it now", you're already too late.

      My optometrist told me a lesson he once learned as a brand new army lieutenant, from a seasoned seargent: If you do not spend your budget, it will get cut. If you do not ask for more next year, you will likely not get last year's level of funding.

      Basically, submitter has to make do with the resources they a

      • Re: (Score:2)

        by mgblst ( 80109 )
        This, together with managers egos, is one of the major problems with big beurocracies, especially government departments. Since each department has this theolisophy, spending will never decrease, and waste happens on a huge scale. Departments need to be rewarded for reducing spending, and complete control needs to be taken away.
    • Instead, build the security from the ground up, paired with each node.

      When you infrastructure already exists (and might date back 10 or more years in parts), building from the ground up is not an option. And I'd bet the poster isn't planning on going out and buying the "Securalizer 5000", but rather talking about an investment in updated firewalls, spam filters, SSL gateways, network infrastructure, etc. In some shops $1 million might buy port level authentication in a new chassis Gigabit infrastructure w

  • Potential Damage (Score:3, Insightful)

    by frieza79 ( 947618 ) on Wednesday January 31, 2007 @04:49PM (#17832472)
    I would start with figuring out what it would cost to fix broken systems, downtime, etc.

    Then you can at least put a price on not being secure, and let management make a somewhat informed decision.

    • I would start with figuring out what it would cost to fix broken systems, downtime, etc.

      Right on!

      This is not a situation that can be analyzed in terms of ROI; ROI is the wrong tool for this work. Writer of TFA should check out "Risk Management" [wikipedia.org] for a start. That is what you want to be doing: providing the corporate officers with a report that says "Here are the risks measured in dollars of potential loss; here are the odds we face on each of the risks; here are some strategies we could use to mitigate these risks; and here are the costs of adopting each of the strategies".

      If I was thinki

  • Proving a negative (Score:4, Insightful)

    by mlts ( 1038732 ) on Wednesday January 31, 2007 @04:51PM (#17832492)
    Measuring security ROI is proving a negative. Because stuff is not being broken into and information is not being stolen, the company is "saving" money by not losing money and gaining bad press.

    Your benchmarks are what type of security issues you do encounter and how they are handled. For example, if a security package catches would-be intruders, that can be shown as a sort of ROI (as the package prevented X dollars of loss.) Another example is the cost of whole disk encryption. Having a laptop that is protected by WDE get lost, one could state that the encryption software (assuming its properly deployed, proper password and/or security token policies set, etc.) saved the company the loss of the data on the laptop.

    Probably the best bet in proving ROI is how many, what type of, and the cost of, the breaches and incidents one had before a policy/software/infrastructure went into place versus afterwards.
    • And since you can't prove the negative...

      I'd take a lazy approach to security. As fun as it is being paranoid about my own server security, it's my time and money that gets spent on that project. If you're in an industry that has specific security requirements (e.g. VISA, SOX), then there's an obvious cost: the ongoing cost of paying the fines of non-compliance. In the case of VISA, you may even be forced to stop accepting credit cards if you were to fail an audit.

      Is security an ongoing problem or a theoret

  • Risk math (Score:5, Informative)

    by theonetruekeebler ( 60888 ) on Wednesday January 31, 2007 @04:53PM (#17832544) Homepage Journal
    Here's a gross oversimplification:

    The cost of a security breach is measured as the probability of an incident multiplied by the cost of the incident. Both numbers can be calculated surprisingly well, or at least made to sound plausible. Security software will reduce the probability of an incident. Calculate the difference. If it exceeds the cost implementing security, it's a good thing.

    This is a basic formula used for all types of data security, including backup and disaster planning.

    • Re: (Score:2)

      by Thansal ( 999464 )
      shouldn't that also include any recovery soloutions that are part of a security soloution? (Forinstance the time to respond to a DDOS or how long it takes to pull your servers out of the slag heap they were turned into)

    • Re: (Score:2, Informative)

      by KerberosKing ( 801657 )
      There is a decent book on this from the Cisco Press: The Business Case for Network Security: Advocacy, Governance, and ROI by Catherine Paquet and Warren Saxe [amazon.com]. Not only does it help put this in terms the execs and bean counters can understand, but the appendix shows you the equations to compute ROI for preventing security breaches. If you've never taken a business administration or accounting class and feel lost when the PHB asks for this stuff in a power-point deck, this book can help.
      • But you get smelly fingers. You can't calculate the probability of a breach because you can't enumerate the threats or the vulnerabilities. How many unpublished zero-days are there for the stuff in your environment? How many hours of unplanned outages will you have this year? Consequently you are just pulling a number out of your ass. I agree you can get some good numbers for the cost of a breach. Not the probability. So you are evaluating a cost times a guess.

        There is no security ROI. It is loss-av
  • At night come into the office and take out the server and steal any other info ....lock it up in some office where the boss wont look. When everyone arrives for work the next day and cant work due to the fact there is a missing server, and the police are being involved talking about taking all sorts of
    equipment for forensics evidence, then pipe up and say that THIS WAS A DRILL...and let everyone go back on about their business. Once you are faced by the boss to explain your actions....just say that had this
    • As amusing as this scenario is, I am not certain it represents the specific manner of security investment the OP is inquiring about. While we've heard a lot about laptops, backup media, and so on going missing (more due to negligence than anything else) how common, really is server theft...?
      • I think the GP is just trying to make the point "what if a server was taken offline?" It doesn't matter if it was stolen, hacked or little green men ate the hard drive, it will still need to be replaced.

      • Re: (Score:2)

        by Thansal ( 999464 )
        And it is people like you that let people like me keep on stealing servers!

        muahahahahaha!

        Actualy, the idea the gpp put up is a horrible one. An imporptu 'drill' (espcialy one that gets the cops involved) will get you fired, and possibly in legal problems.

        The better way of doing this is to write up a GOOD report, and explain the probability of such things happening.

        Ofcourse, as the PP pointed out, when most techies talk about security, they are not reffering to physical security, they are reffering to every
      • As I am answering this and all other sub comments aimed at my initial comment....yes the ramifications would be very unpleasant, especially including the police, but THIS IS THE ONLY WAY TO GET THE TRUTH....otherwise, you have people saying well that number is off, and this number is off, you may include the upper management in the drill, but in the end if you are paid to be a security expert, and your job condones such actions, you have nothing to fear. Next off, how often do servers disappear...we are de

    • Re:One way of doing things (Score:4, Insightful)

      by MarcoAtWork ( 28889 ) on Wednesday January 31, 2007 @05:13PM (#17832940)

      possibly cost you a week suspension
      I don't like judging people by their posts, but what you write makes me wonder if you're still in high-school: in the real world something like the above could net you either a written warning or, more likely, a pink slip, if not being sued for the amount of money that was lost during your 'drill' (which, if this was a financial institution, could be quite large).

      In any case, if you worked for me and pulled a stunt like that I'd be starting to look for your replacement asap: I pay you to do your job, not to prevent other people from doing theirs.

      • Re: (Score:2)

        by Trogre ( 513942 )
        But what if part of his job is information security, and his PHB hasn't given him the budget to do it because he doesn't believe it's all that important?

    • Re: (Score:1)

      by mlts ( 1038732 )
      I'm not sure if deliberatly sabotaging a production server crucial to a company is a good idea. In most companies, that would mean the loss of a job. If management was really ticked, they could file a criminal mischief complaint, the value of which would be all those people's times, loss of income (for a lot of online stores, this could be sizable), the cost of getting the police to come for a false alarm, and other things. If this gets high enough, this could be into felony-hard territory.

      A determined D
      • This post is to comment on the many posts previous to this one....
        I just don't have the time to respond to each reader that doesn't grasp the concept laid here.

        Please don't take this as a post directly to your post, although some of it might hold true
        to answer some daunting comments.

        I am guessing that with the many posts I got concerning my job, as it is what I do for a living,
        I am stupefied as to how many people can't read in todays society. Someone even accused me of being in high school.....yet he finish
  • count the time is takes to deal with forcing people to use passwords with a lot rules and making them change them a lot. As they will right them down / forget them a lot.
  • Until there is a major security breach, only a thorough security audit will give the organization an idea of how much a security problem can cost. If an audit demonstrates terrible flaws in security it should become obvious money needs to be spent on it. If the audit shows security is already reasonably tight then it's a tough argument to spend a lot more money on improving it.

  • Simple (Score:2, Informative)

    by Stormcrow309 ( 590240 )

    (([Total Cost of Intrustion] * [Percentage Chance of Intrustion]) / [Costs of Security Measures]) - 1 = [ROI]

    (($5,000,000 * .10) / $100,000) - 1 = 4

    • Re: (Score:1)

      by jofny ( 540291 )
      Total Cost of Intrusion: Some monetary value, largely intangible Chance of Intrusion: Impossible to model realistically Cost of Security Measures: Ok, yeah, you can figure this one out in numbers. If someone wants a formal real ROI on security, they won't get one. It doesn't work unless you make up numbers that you absolutely cannot know. This equation should only be used for marketing and illustration purposes. It's not useful for real ROI purposes.

    • Re: (Score:3, Informative)

      by SatanicPuppy ( 611928 ) *
      Informative? Informative would be explaining how he came up with accurate numbers for [Total Cost of Intrusion] and [Percentage Chance of Intrusion].

      That's where the problem is in this whole issue. How much will it cost if we get owned, and how likely is it that we will get owned? If you can calculate those two data points accurately, then yes, it's easy as pie to figure out your ROI, but the problem is that figuring out the former, requires the services of a mind reader, and the latter requires the knowled
      • What he said. There are several reasons the situation is this bad.

        Insurance companies can tell you how likely a fire is and how much it costs to clean up and rebuild after one. They have the numbers to justify "loss prevention programs" and to justify giving you discounts for alarm systems. Finance people know all about this.

        For security incidents those numbers simply aren't available. It's hard to cover up a fire, but lots of places hush up security events. The costs are partly intangible (how do you put a
        • I think you are right - you can get an estimate of costs. But the chance of incurring those costs are not calculable. You simply have to guess. You can probably say one risk is higher than another, but you can't enumerate unpublished zero-days, nor assess which threats have them ready to use against you.

          ROI is a badly broken way to look at security.

          • Re: (Score:3, Interesting)

            by Stormcrow309 ( 590240 )

            There is a way to get a concept of the chance of a successful intrusion. There are actuaries that do create this data. Garner may be able provide a good benchmark, as can some industry associations. Heck, insurance companies probably are collecting good data to get a predictor.

            I paid garner for a research paper to justify the purchase of one SAN solution over another. The second solution went TU a year later. I have met the guys who write the reports. They are pretty smart guys.

  • Security is a Vague Term (Score:4, Informative)

    by 99BottlesOfBeerInMyF ( 813746 ) on Wednesday January 31, 2007 @05:04PM (#17832798)

    Spending money on "security" can mean a whole lot of different things. What type of security? What are you trying to prevent? I work at a company that produces certain security products, some of which have other applications as well. When you hand the CEO a nice graph of the DDoS attack that you got your ISP to filter for you when you subscribed to their service, show how many hours of downtime it prevented, and how much money went through the online store during that time, proving ROI is fairly easy. Other kinds of security are fuzzier. Stopping worms within your network saved IT X hours of rebuilding PCs and prevented those machines from being down this many hours times the average worker's hourly rate would have been unable to work during that time etc. and you can provide some estimates.

    Before you get to that stage, however, you need to have specific security measures in mind designed to address specific security threats to your business. Some of these measures are easy to justify (need certification to do business with government agency Foo) and some are hard (better passwords make it harder for insiders to steal our customer database and sell it to Russian hackers who then use it causing a publicity problem and resulting lost customers).

  • It's the only way to be sure.

  • Potential cost of breech (Score:3, Interesting)

    by JoeCommodore ( 567479 ) <larry@portcommodore.com> on Wednesday January 31, 2007 @05:09PM (#17832878) Homepage
    I guess I would give the PHB a potential cost of what breaches could happen and an analysis of your situation and what measures need to be done to prevent it.

    i.e. If you are running a business that keeps SSNs, bank data or some other sensitive data you would factor in the cost of how many customers times how much it would cost if thier personal information were compromised. If you are in design/manufacturing, you could factor in R&D/loss of contract costs if designs were taken, etc. (not to mention press coverage and effects on future customers and the stock market for public companies.)

    Also get any stories of breeches to a similar IT installation to show example that there is an issue.

    It's not really an 'investment' as much as a reduction of liability, if the potential liability is less than the cost of the security it is a hard sell. But most likely it will be a fraction of the potential liability without it and even if you do get a breech after the security update it looks a whole lot better to clinets, the public and the press if you show a track record for keeping your security up to date.

    • FYI: you seem to be alternating your spelling of "breach" and "breech." You want the former. The latter applies to the rear-end of your pants. Note, I'm not normally a spelling Nazi and feel free to ignore this comment as it is certainly off topic.

      • I know how to spell it (as well as the many other words and syntax faux-pahs I create) just that my fingers don't like to type them and I don't notice till it's too late.

        I guess it's "no mod points for me!" :-D

  • For example, if management says that there's $1 million left in the budget, and you can either implement a new customer tracking system that is projected to save $300k per year, or implement a new security technology or process

    Security isn't an add-on in this way, and it will (currently) always be bad advice to "invest extra $X in security". Security ROI only really becomes useful when you have decisions like: "We need X security, what is the best ROI solution".

    Also consider that there is a large fuzz

  • Security ROI (Score:2, Informative)

    by Atrivis42 ( 997334 )
    Security should be something that is considered from the beginning of design. Having said that, I know from experience that it isn't and that management tends to want to plug the hole after the boat sinks. That is, once something bad happens, you get all the money you want and all you have to say is "security". In order to get management to fund security efforts on their data networks, you have to have a good idea of what could happen to your network/data. The first step is to identify all the vulnerabi
    • Sure, it would be nice to live in that world, but what do you say to your bosses after some accounting weenie loses a laptop with an entire period's accounting data including customer banking account data, because he wanted to work in the coffee shop and didn't take the laptop with him when he went to take a piss.

      For every piece of perfect planning, there will be an idiot who opens a hole in your security that you could never have forseen in your preplanning. It's better to have a system that is simple and
  • Being in INFOSEC, and coming from both sides (security vendor, large enterprise) their is no easy solution. A malicous attack can be a loss of information, which can be shown by the value that information is to the company. If its higher than the cost of implementing a protective measure, then you can see the difference easily. The hard one is if the malicous attack takes down your network or e-commerce sight or email. DoS attacks have far reaching effects and cost burdens depending on the attack. What is
  • IMHO, you can't try to get any useable ROI figure for security features. Whatever security feature you add, it's supposed to handle an unlikely event. Should that event never happen, whatever money invested would prove totally worthless. However, _if_ you encounter an issue, then you'll be happy to have spent enough money into security.

    AFAIK, you consider security the same way you consider insurance (or as an insurance complement): How much your business continuity's worth? Should you be hacked/DDoSed, ho

  • Economists have long had a method of measuring "expected value" which is the sum over all outcomes of the probability of that outcome times it's value.

    So in this case the value of the security software is:
          (1 - Pb) * 0 + Pb * VA

    Where:
          Pb = probability that it saves you from getting broken into
          0 = value if you don't get broken into
          VA = value of your ass
  • I would rank and identify the projects you feel are most needed in the security area and then do some research and bring the ones you can make a realistic case for. Management likes numbers but keep it concise and honest. They also like to think they, being excessively smart, hired excessively smart people so cover all your bases beforehand.

    If you can say "We could buy this system which severely decreases the chances of X happening. When X happened to Bob, Inc. they lost eleventy-billion dollars in re
  • TJX (TJ Maxx, Marshalls) will have recent data.
    • Sure, you can ask, but you'll never get them to tell you what it really cost, or what they did to prevent that from happening in the future.

  • Better search term... (Score:3, Interesting)

    by RudeDude ( 672 ) * on Wednesday January 31, 2007 @06:34PM (#17834394) Homepage Journal
    "Risk analysis" is a formal approach to what you are talking about.
    To a lesser extent "Decision Science" and "Influence Diagram" are also attempts at tackling this type of problem.

    Google scholar will turn up many papers in this area and I know that my school (University of Virginia in the Systems and Information Engineering department) has some active research in "Cyber Security" and related security planning.
    http://www.sys.virginia.edu/risk/ [virginia.edu]

  • Proper configuration, proper coding, logging, and timely patches cost hardly anything.
    Antivirus software attempts to substitute for user education, and sometimes slows down systems, reducing productivity. But some users never learn.
    IDS software warns you about threats that should have been blocked by proper configuration. Except that it's nice to find out when an employee brings their virus infected laptop in and connects to your network, maps network shares, etc. I always figured Snort was the best IDS out
  • Cost of a case vs. probability of it happening.

    Unfortunately (or thankfully), a lot of companies don't have experience with a case actually happening, so they can't easily figure out the probability. The threat of viruses/trojans is actually more important for many companies these days than an actual targeted attack (unless they guard some important business secrets in their servers). The chance of this happening can be gotten fairly easily from a security company, they usually have the numbers. The cost pe
  • You are never going to get money back from security investments, you are limiting losses.
    That puts you into Risk Management analays, not Return on investment.
    Think of it like going without insurance, worker injury prevention, or other loss prevention/mitigation.
  • For a new deployment, you have to take into account what is in place, what are the weaknesses, and how they are being
    addressed by the new thingum. google around for household names with breaches like so:

    http://www.itworldcanada.com/a/Enterprise-Infrastr ucture/33200565-b133-4eed-8c05-c6f35f8f60b6.html [itworldcanada.com]

    That article talks about basic things like establishing a perimeter. IF your company does not have a decent DMZ defined,and proper
    safeguards wrt Intrusion detection, and properly walling off remote services

  • Hire an actuary.

    (Though I have no idea where you'd find an actuary who would be able to answer your question.)

  • There are specific methodologies for modeling risks / threats and estimating their impact, that are used for justifying
    Information Security budgeting.

    Principles of Information Security [amazon.com] is one book that I'm familiar with that has quite a bit of coverage of this topic. We used this for my course in Information Security a couple of years ago, and I found it pretty useful, FWIW.

    Additionally, check this OWASP Page [owasp.org] for some good stuff.

    And finally, try googling for terms like Security Risk Analysis [google.com], Security Risk Assessment [google.com], and / or Security Threat Modeling [google.com].

    • In Canada, the RCMP's Threat and Risk Assessment (TRA) for Information Technology is a popular approach. It categorizes threats according to impact (grave, serious, less serious) and likelihood (high, medium, low) and prioritizes the threats from 1 to 9. Here's the link to the guide:

      http://www.rcmp-grc.gc.ca/tsb/pubs/it_sec/index_e. htm [rcmp-grc.gc.ca]

      This approach is process oriented and not focused on specific technologies. The date on the guide is 1994 and it is still in common use, so it has stood the test of time.

      I

  • ROI. Wonderful buzzword, usually meaning "I don't have a clue what I'm talking about but I want to sound impressive".

    Put simply you can't earn a return on a cost, only on an asset. Investment doesn't mean "put money into it", it means capital expenditure to acquire an asset. You invest on the basis of an expected return at a given level of risk associated with the asset. ROI is one of the measures that can be used to assess the attractiveness of the investment.

    These are definitions. You can't subst

  • ROI is essentially a ratio measuring a payback period, which can lead to distortions. Say you have two projects. The first has an investment of $1,000,000 and saves you $100,000 per month. The second has an investment of $100,000 and saves $10,000 a month. Both have a payback period of 10 months (100,000/10,000) and both have an ROI of 100% (100,000/10,000). Which project do you do? Assuming that you can afford to both project, which do you do? Based on this information, you would do both. The missing elem

Slashdot Top Deals

The use of money is all the advantage there is to having money. -- B. Franklin

Close