[Edit -- fixed my mispelling of company name.  D'oh!]

Overview

Application Security Debt is based on the concept of  “technical debt” proposed by Ward Cunningham (a programmer who developed the first wiki program): describes it like this:

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.

Chris adds:

The cost of technical debt is the time and money it will take to rewrite the poor code after you ship and bring it back to the quality required to maintain the software over the long haul.

Here is Chris’ summary of Application Security Debt:

Security debt is similar to technical debt. Both debts are design and implementation constructions that have negative aspects that aggregate over time and the code must be re-worked to get out of debt. Security debt is based on the latent vulnerabilities within an application. Application interest rates are the real world factors outside of the control of the software development team that lead to vulnerabilities having real cost. These factors include the cost of a security breach and attacker motivation to discover and exploit the latent vulnerabilities.

Chris’ second post describes a financial model that estimates the cost of Application Security Debt.  Framing the metric in financial terms will presumably help managers compare the cost of the “debt” to the cost of developing more secure software or costs of fixing the vulnerabilities.  (Note: Veracode provides a range of application security testing services, so they have an interest in economically justifying their services.  This isn’t a criticism of Veracode, Chris, or his proposal.  Just a reality.)

Chris’ model is focused on the simplest case where the application developer and application user is the same organization, so that it bears the costs of development, maintenance, and also any security breaches that result.  Starting with the simplest case is a great idea when proposing a new method.  So far so good.

Chris defines his financial model this way:

The basic financial model for security debt is monetary risk that can be expressed as expected loss. The formula for expected loss is event likelihood X impact in dollars. Event likelihood is based on the makeup of vulnerabilities in the application and the likelihood that the vulnerabilities will be discovered and exploited. The impact is the cost of a security breach based on an exploit of one of those vulnerabilities.  [Emphasis in original]

This is, of course, a version of the bottom-up Annualized Loss Expectancy (ALE) formula for individual risk elements:

• ALE = Single Loss Expectancy X Annual Rate of Occurrence

(Mike Rothman recently crapped on all “risk metrics” by lumping them all into the ALE formula.  I’ll critique ALE and Mike’s post in a separate blog post.)

ALE issues aside, I think Chris is making mistakes in his definition of Application Security Debt that will lead to serious confusion.

Debt = Expected Principal + Interest Costs

Chris made a mistake when he defines monetary value of the Application Security Debt as expected loss due to security breaches.    Instead, the ‘Principal’ part of the debt formula is the cost of fixing security problems beyond what is budgeted. Chris had it right in his summary in the first article:

The cost of technical debt is the time and money it will take to rewrite the poor code after you ship and bring it back to the quality required to maintain the software over the long haul.

Expected losses are in the category of “Interest Costs” as Chris said in his summary:

Application interest rates are the real world factors outside of the control of the software development team that lead to vulnerabilities having real cost.

Putting this together in simple language:

“Application Security Debt is a ‘loan’ with variable principal which could range from 0% to 100% of your original project costs. The ‘principal’ is what you’ll eventually have to pay to fix security bugs or rewrite the code.  It also has varying and uncertain ‘interest costs’, which are the costs of security breaches due to these vulnerabilities. This includes the possibility of the mother-of-all balloon payments (i.e. a huge loss event).”

The good news is that Expected Principal is relatively easy to estimate with good accuracy and without a lot of outside data.  The not-so-good-news is that Interest Cost is a bear to estimate.

Estimating ‘Expected Principal’

For simplicity, let’s assume that cost of fixing code (above the budgeted costs) occurs in discrete increments, F:

1. Zero  (i.e. your debt is ‘forgiven’)
2. Minor fixes and patches (‘Principal’ = 10% increase in project cost)
3. Major fixes and patches  (‘Principal’ = 25% increase in project cost)
4. Substantial rewrite (‘Principal’ = 50% increase in project cost)
5. Total rewrite   (‘Principal’ = 100% increase in project cost, or more)

Thus, the best case is that you owe no principal and the worst case is that you owe principal equal to the entire cost of the project.  You could include other factors such as external costs of schedule delays, costs of rehiring your programmers after you fire them all , or what ever.  My point is that these costs are not open-ended, but are a multiplier on your original development costs.

The Expected Principal (EP) is equal to each of these cost scenarios multiplied by their probability of management choosing that option:

For example, if the original cost of the application development project is $1 million, and there is 5% chance of Zero costs, 80% of Minor code fix costs, and 15% chance of Substantial rewrite costs, then the Expected Principal would be $155,000, or 16% of the original cost.

This is important: Expected Principal is ultimately determined by management decisions and ‘threshold of pain’.  This means that the value of p(F), above, is a subjective probability.  It would be an ideal metric to estimate using prediction markets (PMs).   (PMs have been used successfully in software development to estimate shipment dates and defect rates, for example.)

Another implication: you don’t need to accurately forecast future loss events or their economic impact to get a decent estimate of Expected Principal.  Instead, you only need to estimate the Interest Costs very roughly to determine which code fix scenario is most likely.    You could even estimate p(F) by setting thresholds for the number and severity of vulnerabilities discovered by certain levels of effort.  Better, you could combine these methods to ‘triangulate’ on estimates of p(F).

To calibrate these subjective probability estimates, it would be very helpful to collect historical data on the % of applications that have some level of rewrite or schedule delay due to security problems.  (Hint hint!)

Estimating ‘Interest Costs’ on the Debt will be Hard

The second part of the Application Security Debt formula is ‘Interest Costs’.  This is where things get hairy.   All the members of the ALE family of risk calculations have a similar flaws: 1) prodigious data requirements and 2) propagation of uncertainty through the calculations.  Furthermore, some suffer by using only mean values and ignoring extreme values (i.e. the “tails” of the probability distribution curves).

Chris acknowledges these issues, at least the requirement for more and better data:

Now you are probably thinking that this is getting a little tenuous and it is. We need better data on likelihood type and likelihood of an application breach by industry and other factors like company size.

Data issues aside, I think there are flaws in his use of ALE and calculation methods.  Here’s one thought experiment to show how it could lead to the wrong conclusions, in my opinion.

Let’s use Chris’ ‘baseline expected loss’ table, where he calculates the expected loss for each type of vulnerability.  Imagine that we are comparing two similar applications, A and B.  Assume that each project is expected to have the same number of vulnerabilities, five each.  Let’s say the development cost of each project is $1 million.  Application A has five SQL injection vulnerabilities while application B has one SQL Injection vulnerability and four Remote File Inclusion vulnerabilities.  Doing the calculations:

• A’s expected losses = $19,220,000
• B’s expected losses = $5,074,080

Does project A really have four times more risk than project B? Probably not.  From what I know, the number of vulnerabilities in an application is not proportional to the likelihood that the application will be breached.  Instead, I’d guess that the likelihood of being breached is a function of where the application is in the IT architecture, how accessible it is, how important it is to attackers, etc.

Also, there’s the ‘weakest link’ effect: “given enough random attackers or one persistent attacker, it only takes one vulnerability to lead to a breach”.  Assuming all SQL Injection vulnerabilities are equally discoverable and equally exploitable, then we should estimate that application B with one SQL Injection vulnerability is just as likely to get breached as application A with five, all other things being equal.

(I confess I’m not an expert in application security or vulnerability analysis, so these comments are my interpretation of what others have written or said.)

Even if my logic here is flawed somewhat, my main point is that the relation between number of vulnerabilities and likelihood of being breached is non-linear and it may even be indeterminate if contextual factors dominate.

This example also hints at another severe weakness in the ALE method – it ignores correlation and dependence between risk elements and factors.  We know from forensic analysis and the DBIR that severe security breaches involve a sequence of exploits and attacks.  This means that the likelihood of breach in one application is dependent on the likelihood of breach in other applications and systems.  An application might appear unimportant, but it might be a stepping-stone to other applications, databases, and networks.

It’s hard to account for all these factors and influences together without some sort of over-arching model for enterprise-level information security and risk.   Basically, you are looking for the ‘risk contribution’ of those specific application vulnerabilities to total costs, now and in the uncertain future.    Formally, the ‘Interest Cost’ for any given set of application vulnerabilities is the difference between the Total Cost of Security (TCoS) in two possible worlds: World 1) application A has X vulnerabilities, vs. World 2) application A does not have X vulnerabilities (or if application A is not deployed at all).

What we really need are some short-cut approximations for this that doesn’t require a complete data set and risk estimates for the whole enterprise.  One approach I’m interested is in using modern AI methods (data mining, machine learning, inference methods).  This is on-going research.

Summary

I’m glad Chris proposed his Application Security Debt metric.  I hope my post has been helpful in correcting some of the errors, as I see them.  The good news is that the “Expected Principal” component of the metric looks like it can be estimated fairly easily and with good accuracy.  On the other hand, the “Interest Cost” component needs a lot of work.  I’m happy to collaborate with Chris or anyone else who wants to work on this.

Here’s the website: www.nortoncybercrimeindex.com (all in FLASH)

Here’s a typical article:

Norton Cybercrime Index, unveiled today, rates the current state of cybercrime in a single, simple number and indicates whether the danger level is going up or down. Interested visitors can drill down for almost any level of detail. [...]

The index is open-ended, like the Dow Jones Industrial Average. Symantec’s proprietary algorithm draws on many sources to produce the index, among them the Symantec Global Intelligence Network, Norton Safe Web and the millions of customers using Norton 360 Version 4.0, Norton AntiVirus 2011, and Norton Internet Security 2011. To ensure the validity of the algorithm Symantec had it analyzed by experts at the University of Texas’s Institute for Cyber Security; the experts approved.

What’s the goal?  From the FAQ (embedded in FLASH):

Symantec created the Norton Cybercrime Index to show people that cybercrime is real, it can happen to anybody, and there is something you can do to protect yourself.

How is it calculated?

…using a statistical model and algorithm, which assigns values to the number of online threats observed each day.  Threats include malware, fraud, identity theft, spam, phishing, and social engineering trickery.  Once threats are quantified and processes through an algorithm, the Norton Cybercrime Index number is generated.  The algorithm has been endorsed by the University of Texas San Antonio as a valid measurement reflecting the risk of cybercrime.”

My initial judgement

It looks like it is purely a product of Symantec’s marketing department.  There’s a massive PR effort underway via blogs, twitter, public places (e.g. London, Times Square), and probably at the RSA Conference, now underway in San Francisco. The web advertising firm Fine Design Group created the FLASH UI, and tweeted about it first.

It will be interesting to probe their methods and data, assuming that Symantec will be transparent about the “proprietary algorithm” used to compute the index.  If they really want to establish credibility, it would be irrational to treat this as proprietary, confidential, and closed, for all the obvious reasons.  ID Analytics is listed as a data provider, but there’s no evidence that their ‘advanced analytics’ are used by Symantec, only their summary data regarding personal identity theft in the US.

I’d be very surprised if any of Symantec’s metrics experts are behind it.  I don’t know of anyone in the security metrics community who has been contacted or involved as an outside expert.  They certainly haven’t presented it for peer review at last Monday’s Mini-metricon (why not?) or to the securitymetrics.org email list (why not?) or any academic conference or journal (why not?).  Searching the University of Texas at San Antonio, Institute of Cyber Security’s web site, I couldn’t find any mention of their work on this project, nor any presentation or report.  A search of Google Scholar for “cyber crime index” produced a few results, but not related to this and not from anyone at UT-SA.

Q: Who did have an early look at this?  A: Angus Kidman, a blogger from Gizmodo.  And what did he learn from his demo?  From his blog post:

“On the day of the demo, these were the top search terms being targeted for poisoning:

• Invisible
• Camel toe
• Wifetube”

Right.  How very useful.  I’ll now modify my search patterns so I avoid those words today.

I don’t have  a good feeling about this

It smells like FUD in a spiffy FLASH interface. Sure, there probably is real data behind it, but it’s aggregated into an index that is supposed to mean something.  A daily index!  The FUD label fits because this presentation gives the illusion of scientific validity, precision, reliable aggregation, and meaningful signals, when that none of these are present (it appears). Using fancy words like “statistical method” and “algorithm” gives it the air of scientific validity without really saying anything.  Worse, those words hide the assumptions, judgments, fudge factors, and who-knows-what that make the index work.

My intuition about this is that Symantec marketing manager wanted to create a “daily itch” to get average people to read what ever news blips were available that day about ‘cybercrime’, which would increase the chances that they would move from ‘awareness’ to ‘action’ (= buy more Symantec stuff).  By getting this out as a daily index, any up or down moves each day will trigger some people to click the buttons to find out ‘why?’.   But this will take them to news items, but not any credible justification of why they might be at greater risk on that day, compared to the day before.

As a thought experiment, imagine a similar ‘Risk Index’ that is powered by astrology readings, numerological interpretations of Nostradamus’ texts, or some other daily signal source.  With the appropriate shroud of credibility, some number of people are going to start following it, and when ever it changes, they will seek information as to ‘what does this mean for me?’  It would serve have exactly the same function as their current design.  This doesn’t prove anything, but establishes in my mind some plausibility.

What’s the harm?

Some might argue that this is harmless or even mildly beneficial if it prompts people to be more aware of security problems and to fix their security problems.  But I think it’s harmful because it promotes a false signal and a false method for doing information security metrics — for consumers or for anyone else.

Maybe I’m wrong and this may be an important advance, or at least a step forward.   At very the least, it shows that one  major security product/service vendor spent money to define a method, collect data, and make public the results.  Prior to this, no major vendor was even spending money on it.

What to do now

Is there any way this Index could be redirected to be a more valuable and extensible project?  I hope so.  But for that to happen, those of us how care about the New School approach to security need to apply the full-court press on Symantec to open up their method and data.

Your action — contact Symantec, preferably in-person at RSA Conference, and demand they open up and also engage in the security metrics community in a serious way.  The burden of proof is on them, and if they can’t back it up then they should be shamed.

As a business guy, I was somewhat surprised that much of the discussion and opinions centered on MBA as a credential rather than what knowledge or skills someone would learn in an MBA program.  None of us at New School are a fan of credentials as such, so my interest in this question is on the educational value compared to alternative investments in education

Also following the New School philosophy, I thought I would look for data and evidence rather than just offering my opinion.

To my delight, I found a fairly comprehensive study: THE CHIEF INFORMATION SECURITY OFFICER: AN ANALYSIS OF THE SKILLS REQUIRED FOR SUCCESS, by Dwayne Whitten of Texas A&M University . The paper is worth reading because it gives a good overview of the conflicting values and forces that are affecting CISO hiring, evaluation, and effectiveness.

Specifically, he finds a gap between how CISOs define success and the job duty descriptions. Quoting from his conclusion:

Based on a thorough review of the literature, interviews with security executives, and an analysis of job listings, a comprehensive list of duties and background/experience requirements were found related to the CISO position (see Table 3). The most interesting issue that arose from this research is that business strategy did not make the list of most included job duties. Given the high level of importance given to this by the literature and the executives, it is surprising that it was not listed on the job listings surveyed. Thus, it appears that many of the organizations searching for new CISOs during the research period did not fully understand the importance of including the CISO in the business strategy formulation.  [emphasis added]

This dichotomy seems to relate to how CISOs are viewed.  From one point of view, CISO is equivalent to “Most Senior Information Security Manager”.  That is, they contribute to the organization in exactly the same way as do other information security managers, but only on larger scope.  It is this perspective that is most closely aligned with the opinion that an MBA education would not be helpful.  Instead, it would be more valuable to get deeper education in InfoSec technical aspects — engineering, forensics, incident response — plus regulations, compliance, etc.

Another point of view is that a CISO is an executive officer of the organization, and thus has fiduciary duties to stakeholders regarding the organization’s overall performance, and also has teamwork responsibilities with the other executive officers regarding crucial strategic decisions.

Maybe this is rare in practice, and maybe the “Chief Information Security Officer” title is just another example rampant job title inflation.  But if a CISO in some organizations are expected to perform in this role, then it is not the case that they are not “just another information security manager, only  bigger”.  Their job is qualitatively different and the knowledge gained at a good quality B-school might be just what they need.

To respond to Eric, who said “And I’ve yet to see a course on security risk management in traditional MBA programs”, I offer two examples: 1) James Madison University offers an MBA in Information Security.  2) Worcester Polytechnic Institute offers an MBA concentration in Information Security Management. The WPI MBA course catalog list quite a few that would be directly valuable to a CISO (e.g. “INFORMATION SECURITY MANAGEMENT”, “OPERATIONS RISK MANAGEMENT”, and “E-BUSINESS APPLICATIONS”), plus many that would be indirectly valuable (statistics, change management, negotiations).   (Disclosure: I got my undergraduate degree from WPI.  Their MBA program is very good, esp. for technical managers.)

I’ll close with a comprehension test for CISOs.  Read this workshop report: Embedding Information Security Risk Management into the Extended Enterprise.  It’s the output of 18 CISO discussing the most challenging issues facing them regarding information security across their enterprise and across their supply chain.

I think you’ll see that most of the problems involve analysis and methods go well beyond the typical education and experience of information security managers.  Instead, they require knowledge and skills that are more typically covered in MBA programs — business strategy, economics, finance, organization behavior and change management, organization performance management and incentives, plus business law and public policy.

Conclusion: if a CISO is expected to be an executive officer (esp. for a large, complex technology- or information-centered organization), then he/she will need the knowledge and skill exemplified by the comprehension exercise, above.  MBA is one path to getting those skills, at least if you are thoughtful and selective about the school you choose.  Other paths are available, so it’s not just about an MBA credential.

Otherwise, if a CISO is essentially the Most Senior Information Security Manager, then MBA education wouldn’t be of much value.

A noble effort, but…

Before I start chopping it up, let me say that I think their annual survey is a good effort and it’s positive that they can get sponsorship and also readership for the results.  I think they have good intentions and try to give a fair, balanced, and reasonable estimate.  Our field would be better off if there were similar data gathering efforts in other areas of InfoSec.  I also don’t believe that any of the errors are due to intentions to ‘spin’ or mislead.  It looks like they didn’t have sufficient expertise on their team in business finance, marketing analysis, and economics.

But I see some serious problems with their methods.  This is a big deal since ‘indirect costs’ make up a majority (68%) of their estimate of total costs.

Problem #1: A fog of buzz words

If their data and analysis were bulletproof, then maybe we could forgive sloppy use of terms.  But it isn’t bulletproof and their use of terms is actually misleading because it gives the impression that the method is well established and well executed when it really isn’t.  Furthermore, it’s a sign that whoever is doing this part of the analysis doesn’t know what they are talking about.  Examples:

• “The survey design relied upon a shadow costing method used in applied economic research.” (p. 36) — There is no such method as ‘shadow costing’.  Do a web search if you doubt me.  The only examples of ‘shadow costing’ are economic studies that use ‘shadow prices‘ multiplied by input quantities to derive ‘shadow costs’ for certain manufacturing or service process.  Having just completed a Mathematical Economics class last semester, I can assure you that the Ponemon method has nothing to do with shadow prices or shadow costs.
• “Utilizing activity-based costing…” (p. 3) and “The diagram below illustrates the activity-based costing schema…” (p. 36) — They do not use activity-based costing.  Activity-based costing (ABC) is a way of allocating overhead costs by measuring some ‘activity’ in operations that are thought to drive those overhead costs.   The ratio of ‘activity’ for each business unit to the total is used to allocate the overhead cost to that business unit (or product line or customer segment or what ever).  You can read more about ABC here and here.   How big a flub is this?  Big.  It’s like labeling a signature-based AV software as an ‘expert system’.  Anyone who uttered such a statement would immediately be dismissed by security experts.  What ever the Ponemon method is, it’s not ABC.  Just because costs are related to activities does not mean you are doing activity-based costing.
• “…most companies experience opportunity costs associated with a breach incident...” — No, these aren’t ‘opportunity costs’.  The term ‘opportunity cost‘ has very specific meaning in microeconomics.  Basically, an opportunity cost is the cost of giving up your next-best alternative when you make a decision.  They misuse the term here to refer to costs that expected in the future, i.e. beyond the historical frame of the breach incident and post-incident remediation.  Their use of the term here is just sloppy and could easily mislead someone who doesn’t know economics.

Problem #2: Mixing accounting costs with economic costs

This is a subtle but fundamental problem, and it’s why people get degrees in accounting and economics.  Accounting costs are those that appear in a financial statement somewhere and follow specific costing rules, e.g. GAAP.  They have already occurred (historical costs) or they are forecasted to occur (pro forma costs).  In the Ponemon method, they list four categories of accounting costs:

1. Detection or discovery
2. Escalation
3. Notification
4. Ex-post response

Now it’s probably true that most organizations do not have explicit accounts for these costs so they have to be derived from other accounting costs.  But it’s pretty easy to slice and dice accounting data (i.e. general ledger entries) to get decent estimates of these costs.  It’s also possible estimate costs by using per-resource costs (labor cost per hour) multiplied by the usage of those resources (hours to resolve an incident).  In the Ponemon survey, they ask their point-of-contact for their estimate of these costs.  That’s probably OK, given that the point-of-contact is a privacy/security person directly involved in the incident.

But then they mix in future economic costs (what they mislabel as ‘opportunity costs’):

1. Turnover intentions of existing customers
2. Diminished new customer acquisition

(Leave aside for a moment that they are asking about “intentions” of customers to defect.  Adam discussed this in his post.)

These are both economic costs.  (See this slide deck, slide #2.  The wikipedia article on this topic is not good.)  Basically, economic costs are all in the future.  There is no such thing as ‘historical economic costs’.   Only cash flows count in economic costs — no ‘intangibles’, no depreciation, no ‘good will’.  Those can only be included in the form of future cash flows discounted for time and risk (and uncertainty).  Economic costs include opportunity costs (see above), which are the cash flows associated with the next-best alternative.  However, opportunity costs will never appear on a financial statement, now or in any future.  Economic costs are incurred when the commitment is made, not when they are recognized in the accounting system.

Most important: All cash flows are discounted for the time value of money and the riskiness of the cash flow. This feature is essential for rational deicsion-making over time and over risky alternatives, but it also guarantees that no estimate of economic costs will ever equal the corresponding accounting costs because accounting systems to not adjust for the time value of money or risk.  Finally, a full estimate of economic costs includes the present value of ‘real options’ and  should be adjusted for risk (i.e. derating by using the costs of insuring against unexpected/extreme events, cost of lowered credit rating, etc.).

The element of their method that specifically invokes economic cost is ‘lifetime value of customers’ (LTV).  In their method, the cost associated with lost customers is estimated by multiplying the % of breached customer records that will defect (‘abnormal customer churn rate’) multiplied by  LTV.  (LTV originated in direct marketing in the 1980s.  Wikipedia has a decent article that explains it. Here’s a good demonstration.)   LTV is a net present value, discounted by the cost of capital associated with the riskiness of the cash flow.  It’s an economic profit, not an accounting profit.

Putting this all together, either the costing method should use only accounting costs (historical and/or pro forma) or it should only use economic costs (prospective discounted cash flows, risk-adjusted).  Otherwise, the numbers don’t add up, literally.

Problem #3 : Decision polices matter

(i.e cheap short-sighted bastards can have lower costs than prudent socially responsible managers)
Here’s another problem with mixing accounting costs with economic costs.  Let me illustrate this with a story.  There are two companies — Cheap Bastard, Inc. (CBI) and Nice Guys R Us (NGRS).  CBI has decision policies to spend as little as possible on InfoSec, especially in incident detection and incident response.  They push all liability onto their customers, suppliers, and contractors.  They systematically downplay evidence of breaches, and downplay the severity or costs of breaches.  They avoid forensic analysis if they can get away with it.  And so on.

In contrast, NGRS puts a lot of attention on pro-active security and detection and goes out of it’s way to mitigate the costs of insecurity on it’s ecosystem.  They are especially eager to spend money post-breach to restore public trust and to learn from the event to get at root causes.

How would CBI and NGRS show up in the Ponemon survey?  My guess is that NGRS would have cost per record of 2X or 3X greater than CBI, primarily because CBI will have much lower accounting costs (as covered by the survey) by decision policy. It’s also likely because CBI can ‘safely’ ignore the probable future costs of their rapacious behavior (i.e. class action lawsuits, regulatory penalties, even larger security breaches).   I put ‘safely’ in quotes because such corporate behavior is only safe until you get caught or get screwed.

I don’t see a way around this if you only use accounting costs.  If you use a sufficiently broad framework for economic costs, then you stand a chance of understanding the ‘total costs’ that exposes the riskiness of CBI’s decision policy.

Problem #4 : Do respondents really know anything about customer LTV or ‘churn’ intentions?

I’m surprised that no one has brought this up before.  As someone who has calculated and published LTV metrics for a business unit, I can say with some confidence that almost no one who didn’t read those reports would have been able to guess the LTV of customers, including accounting people who knew about the cost and revenue categories but never put them together into LTV.  SWAGs (as in def. 5) were could be off by an order of magnitude.

My opinion is that asking a privacy/security/incident response specialist to estimate LTV is fundamentally flawed unless that person has access to their own company’s management reports that include LTV.  It might be possible to elicit useful estimates from them after their estimates are calibrated through exercises, including exercises that estimate the weighted average cost of capital, average lifetime of a customer, acquisition costs, retention costs, etc.

Same goes for ‘churn’ rate (percentage of customers who leave because their records were breached).  To estimate ‘abnormal churn’ due to the breach, the point-of-contact would need to know something about ‘normal churn rate’ and, as Adam says in his post, the variability of churn rate.  If churn rate varies widely from year to year, then a small increase in churn due to a data breach would be washed out by the other factors driving variability.

It would be much more useful to find out if the company increased their marketing budget as a direct consequence of a given data breach.  If they did, then this would be credible evidence that the number and value of lost customers was great enough for the company to change it’s spending decisions.

Problem #5: Leaving out significant cost categories

This problem may be bigger than all the others combined.  If they left out major categories of cost, then their estimate of cost per breach could be off by 50% or more.

To answer this question, you first need to decide between estimating accounting costs vs. estimating economic costs.  Every economist and every B-school professor will advise you to estimate economic costs.  It may be useful to analyze historical accounting costs as a way to estimate future economic costs, but that is a separate exercise.

As an economic cost analysis, it might be best to frame the decisions this way:

• Given a breach of customer data of size X records (same size as historical breach), how would the firm’s economic costs change vs. no breach?

I’ll point out two categories of cost that this analysis would include that are currently excluded in the Ponemon survey.

Cost of additional spending on security
If a firm incurs incremental spending on security due to a breach, shouldn’t those costs be included in the ‘cost of a data breach’?  This goes back to my story about fictional companies CBI and NGRU, above.  If CBI is likely to spend more to fix their crappy security in the future if they experience a large breach today, then they will be forced to ‘pay the piper’ in economic terms and their decision policy of spending as little as possible won’t help them avoid the full costs of data breaches.  This will also capture the cost of half-measures, since trying to get off cheap on the security upgrade will still show up as higher expected costs for future breaches.

Of course, this raises sensitive political issues with respondents to the survey.  They may be reluctant to answer questions about actual spending on improvements to security or, even more, to speculate about possible future costs.  For example, what if a company’s outsourcing strategy is hopelessly insecure and the firm is forced to reverse those decisions and insource those processes.  What if a company is forced to exit a line of business because the security risks and costs are too high?  What if a data breach leads to process changes that diminish or eliminate their key competitive advantage?

Factoring these costs could increase the cost of a breach by 0.5X to 10X.  I would make it much harder to do cross-company and cross-industry comparisons.  But wouldn’t it make the true economic costs of data breaches more relevant to management decision-making?

Social costs
The other ‘elephant in the room’ is the cost to consumers or employees for having their private data breached.   Add these all up and you get ‘social cost’, or appropriately adjusted,  ’social welfare‘.  I understand that the Ponemon survey is estimating only costs that are incurred by a single organization that experiences the breach, not by any other stakeholders in that firm’s ecosystem.

There are plenty of studies on the direct and indirect costs of identity theft and the perceived costs of breaches of privacy.  Drawing on these studies might make it possible to estimate the social cost of a breach.  Then the estimation question is “What portion of social cost will the firm have to bear?”

The answer to this question depends on firm policy (see CBI and NGRU story, above), the legal system, the regulatory system, and also the legislative system.  Basically, if a firm or collection of firms consistently and egregiously impose large costs on their customers or employees, then one or more of these other social/political mechanisms might kick in to impose an ‘equity remedy’.

The most immediate remedy, from the American firm’s point of view, is a class action lawsuit.  Of course, estimating the likelihood of getting sued, the damages sought, and the likelihood of losing such a suit is risky business .  But just because it’s difficult to estimate with precision should it be excluded?

Again, including this cost category might increase the cost per breach by 2X to 10X in some cases.  But it might also shift management attention to crucial questions such as “What is our role in our value network regarding information security and risk?”

Problem #6: Unsupportable inferences

Given that their survey method is not statistically robust (see p. 33), they do not have sufficient confidence to make the inferences summarized on p. 28.   I won’t go through these one by one, but anyone who has done statistical sampling and inference knows how sample size and variability affect confidence intervals.  If the difference in question does not exceed the confidence interval, then you cannot support the inference from the data.  The best they can do is say, “we say X% of companies report Y, vs. A% of companies reporting B.  This suggests that…”.  All such suggestions would then need to be subjected to additional tests.

Problem #7: Is ‘Cost per Record’ the best measure?

It appears that only a few costs truly vary by the number of records breached.  These include costs of ‘notification’ and some of ‘ex-post costs’.  But ‘discovery’, ‘escalation’, and ‘indirect costs’ are mostly independent of size of breach measured by number of records.  Some might be fixed costs that are independent of the size of breach.  Some might be increasing functions, perhaps relative to some threshold of that defines ‘big’ or ‘material’ (to use the accountant’s term).

This problem may not be significant compared to the others.  I just think it needs to be justified by comparing it to alternative formulations.

Summary

The summary result ($204 per record) reported in the Ponemon survey is not reliable.  No one should rely on the absolute value of this measure.  Some of the relative measures might be informative, especially the direct costs that the point-of-contact respondents are qualified to answer.  Trend analysis might be somewhat informative.  None of the recommendations reported (i.e. the value of hiring outside IT security consultants) can be supported by statistically significant inferences.

To get a reliable measure of Cost of a Data Breach will require substantial revision to the survey instrument, sampling method, analysis methods, and reliability controls.  I’m guessing that this is beyond the appetite of PGP, the sponsor of the survey.

Call to action

What would it take to launch a Version 2.0 of this study with more robust methods and a stronger team of experts to execute it and analyze the results?  There’s no mystery about how to do Version 2.0.  The only obstical is resources and commitment.

<Addendum:  For a related discussion, see my previous post: “Cost of a Near-miss Data Breach“>

Real dashboards

Dashboards are appealing because they are macho.  The metaphor is derived from the instrument panels of airplanes and cars, and especially fighter planes and racecars.  Managers like to think they are daring and brave fighter pilots and auto racers, so the dashboard as visual metaphor is superficially appealing.

To the right are three examples of instrument panels:  a vintage fighter plane, a modern sports car, and a modern helicopter.  The basic form is an array of individual meters, mostly analog meters on a continuous numeric scale.  Some have thresholds or warning levels (e.g. ‘red line’ on the RPM gauge or the ‘over temp’ on the temperature gauge).

Below are two examples of security dashboards that make obvious the visual metaphor: 1) a compliance dashboard offered by ISACA and 2) Seculert’s executive dashboard.  Of course, the ISACA dashboard is simplistic and stylized, but it does make the most obvious use of the visual metaphor.  (I’m not talking about ‘dashboards’ that are merely assemblies of charts, graphs, and tables.  They don’t really make use of the dashboard visual metaphor.)

ISACA compliance dashboard

Securlert dashboard

Dashboards work when controllers match the meters

Going back to the original context of instrument panels for airplanes and automobiles, we can describe their use case in simple terms as a feedback loop:

1. Look at the instruments
2. Adjust the associated controllers (throttle, joystick, etc.)
3. Repeat (1) to see the effect of (2), relative to goals or thresholds

The key to usability is the association between appropriate controllers and the individual meters.  In a car, the controllers are the steering wheel, the gas pedal, the brake pedal, the ignition switch, and the gearshift, primarily.   Generally, there are one or two controllers associated with each meter and the action of each controller is usually proportional to the metric that appears on the meter (e.g. Gas pedal and brake pedal control speed; gas pedal and gear shift control RPM, etc.). There are more controllers on a plane, but the same relationships hold between controllers and meters, at least for older planes.

Information security management is different

The dashboard metaphor doesn’t work for information security management because the ‘controllers’ – i.e. the actions and decisions of InfoSec management – are not in close correspondence to the output of the ‘meters’.  At a CISO level and above, managers take action through budgets, spending priorities, architectures, staffing levels and resource allocation, performance evaluations, security policy management, security awareness programs, compliance audits, vendor relations and contracts, security development lifecycles and methods, incident response policies, and (hopefully) collaboration with business executives on information security implications of business decisions.  In addition, there is a management function that is under-recognized in my opinion: InfoSec risk intelligence and organization learning/agility related to information security.

Even if your list of InfoSec management actions/decisions is different, I believe you’ll agree that almost none of these ‘controllers’ is in one-to-one or few-to-one correspondence with any of the ‘meters’ in the security dashboards.  This means that there is no simple way to see the effect of adjusting any ‘controller’ by monitoring a ‘meter’, which translates to usability problems.    Managers may still go through the motions of looking at dashboards, but their actual decisions will be guided by other information.

What visual metaphor would be better?

The right answer to this question might lead to a great business venture.  I don’t have the answer, but I can offer some suggestions on fruitful directions.

There’s a lot of research that shows that people think about risk in terms of stories.  Any visual metaphor that conveys a stylized story about risk and the ‘controllers’ associated with risk could be very useful.  One approach is ‘rich pictures’ associated with Soft Systems Methodology.  Here’s another example.  Of course, these rich pictures are static unless you watched them being created.  A brilliant designer could probably turn them into dynamic animations, maybe using video game techniques.

Speaking of video games, there are many possible ways to adapt the way they visualize their virtual worlds, which often entail threats, risks, and uncertainties.  One example is the campaign map from the Total War series for Windows PC, a personal favorite of mine.  It is somewhat analogous to the rich picture example given above. One interesting feature of this map is the areas of ‘light’ vs. ‘dark’.  The dark areas are where you, as a leader, have little or no intelligence.  Only when you send a unit or agent into the dark regions do you find out what is really there.   I really wish that security metric displays had a similar way of conveying uncertainty, ambiguity, and ignorance.

While not a solution in it self, another interesting visualization method for the social aspects of information security and risk is to use facial expressions to represent the risk perceptions or emotional disposition of various groups – users, adversaries, supply chain partners, regulators, etc.  I experimented with this with good results in the 1980s when I built a prototype of Michael Porter’s competitive analysis method in Hypercard (!).

Automated facial expression animations could be useful to express risk perceptions by stakeholders

Summary

Dashboards work when the user has proportional controllers or switches that correspond to each of the ‘meters’ and the user can observe the effect of using those controllers and switches in real time by observing the ‘meters’.

Dashboards don’t work when there is a loose or ambiguous connection between the information conveyed in the ‘meters’ and the actions that users might take.  Other visual metaphors should work better.

For more information on visual metaphors in design, check out these two seminal books: Donald Norman’s Design of Everyday Things and Edward Tufte’s The Visual Display of Quantitative Information.

Background: I’m currently working on a small research project to model the innovation arms race between attackers and defenders.  For simplicity and availability of data, I’ve chosen to focus on spam and anti-spam (filtering).  One of my goals is to model how innovation by one side triggers complementary innovation on the other side, and under what conditions would either side “escalate” the innovation arms race to pre-empt the other side.

To do this, I need to understand paths of innovation and the perceived costs and benefits of each path at any point in time.  In particular, I need to model the dependencies between various components and technologies as they evolve.  In the world of email spam, one such technology is the encoding and formatting of the email message (plain text, HTML, CSS, etc.) that is used by the spammer to “trick” the filter and the end user.

Data: One interesting source of data is John Graham-Cumming’s “Spammers’ Compendium“, where he tracks spammer “tricks” found in the wild, and codes them by method and by technology.  One graph shows the number of “tricks” by technology type vs. time, shown here:

Here’s how you read this chart.  The vertical axis is “number of tricks, cumulative”, and the horizontal axis is time, by quarter.  In this time period (2003-2007), you can see that HTML has been the most prolific vehicle for spammer “tricks”, and continues to be popular in spite of more sophisticated technologies like PDF and Flash that have recently appeared.

My Analysis: I defined a small set of inferences regarding this data to derive other metrics.  Namely, I’m interested in estimating the investment required to master each technology (roughly, the level of skill and effort) and also the “affordance” potential (i.e. how fruitful the technology is in allowing spammers to invent new tricks). I used inference rules to produce the following diagram.  (The inference rules just look for certain patterns in the data, then draws conclusions regarding the two dimensions and also the evolution paths.)

Here’s how you read this chart.  ”Effort” (=spammer investment) is on the horizontal axis, ranging from “low” to “high”.  (It’s probably a log scale, but I haven’t quantified it yet.)  ”Affordance” is on the vertical scale, again ranging from “low” to “high”.  Roughly speaking, this scale could be quantified by the following test: “How many spam tricks could be invented by a reasonably skilled spam team, given a fixed amount of money and time?”  But, again, I haven’t formally quantified this scale.

For example, what I’m asserting with this diagram is that Plain Text is easier for a spammer to master than HTML, but not by much, and that HTML offers much greater affordance for inventing new spammer tricks than Plain Text.  Similar assertions can be made by the placement of the other technologies relative to the two axes.  The arrows between the boxes indicate the evolution path for innovations:  (roughly) they have to master innovating in Plain Text as a prerequisite to innovating in HTML, which is then a prerequisite for innovating in Javascript, CSS, Image spam, and others.

(Of course, I’m not literally saying that spammers have to develop these capabilities in this order.  It’s more about logical dependence relations.  For example, any toolkit that supports HTML spamming will also have some Plain Text spam trick capabilities, and so on.)

The dashed grey lines represent constant Return on Investment (ROI), where the investment is the spammers time, energy, and money spent learning the technology, mastering and configuring tools for automation, then then tuning the operation for effective mass production.  (Embedded in the “return” element of ROI is the capability of spam filters at that point in time, but I’m modeling that separately.)

(FYI: other sources of data, including longitudinal case studies of spammers, will be used, so I’m not solely dependent on this one source.  But it does seem especially good to focus on the rate of innovation on the side of spammers, and how innovation branches out.)

Questions for you:

1. Do you think this analysis is credible?  What are the holes, if any?
2. Do you agree with my ordering of the content encoding technologies in the figure above?  (For example, to me, the placement of Javascript seems low on the “affordance” scale.  I would have thought it would be a very fruitful vehicle for spammer tricks, but maybe I’m missing something obvious)
3. Do you think I have placed PDF and Flash in the right place?  According to a technology lifecycle perspective, it would appear that the spam potential of PDF and Flash had hardly been explored in 2007, but I don’t know if I can justify placing them so high on the affordance scale.
4. Any other thoughts?

Thanks.  You can also email me privately at russell <dot> thomas AT meritology <dot> com.

But you can find them all through Google using this search string because they put “NOI” into every file name:

NOI site:http://www.nist.gov/itl/upload/

You’ll see official submissions from Microsoft, IBM, Google, Verisgn, Cisco, TechAmerica, US Chamber of Commerce, plus a few submissions from crazy individuals like me.

“…we are recommending that the Director of the Office of Science and Technology Policy, in conjunction with the national Cybersecurity Coordinator, direct the Subcommittee on Networking and Information Technology Research and Development to exercise its leadership responsibilities…”

We could paraphrase this by quoting Spike Lee’s movie title: “Do the right thing.”

The only problem with this is recommendation is that NITRD’s Cyber Security and Information Assurance Working Group has specifically defined it’s role as facilitator, not a leader (p15). Wishing that they would take the lead won’t make it so.

"Time flies like an arrow, but fruit flies like bananas" -- Groucho Marx

“Data” tells you about the past

“Data” is the output of some observation or measurement process.  If your data is about some states of the world, then by definition your data lives in the past.  You did your measurements or your experiments, generated your data, and then time passed as you assess it, report it, and act on it.  Thus, your data is reporting on history.  Only by acts of inference can you connect your data with the present state of the world or the future state.

In the physical sciences and engineering, they can safely assume that the system under study is the same over time — past, present, and future.  This is called the ergodic hypothesis.  In statistics, the underlying stochastic process is treated as stationary.   This makes it possible to extrapolate the past into the present and future using regression and other techniques.

There are people in the security metrics community that only want to operate on data.   They view anything that is not the result of empirical measurement is pure speculation or a dangerously-seductive “model”.    (See Models are Distracting, and Measurement over Models)    Being an engineer myself, I’m all in favor of empirical data, measurment, and experiments.  But I contend that we will never get to measures of “security” or “risk” through empirical data alone.   Our systems are non-stationary and non-ergodic.

“Security” is a judgement about the present

If we start with the simple high-level question: “Am I secure?”, it becomes clear that any measurement of security must relate to the present time (or possibly a retrospective view on a previous time, i.e. past perfect tense, or prospective view on a future time, i.e. “will I be secure?”).  I call it a “judgement” because security depends on the threats you are facing.  (I play a historically-realistic computer game with my son, called Total War, that includes features that allow you invest in offensive and defensive capabilities.  How much to invest and how fast to invest depends on who you are facing.  A wooden pallisade will be an adequate defense against peasants and spear militia, but hopelessly inadequate against onagers and trebuchets, backed by armored cavalry!)

Thus, you can measure anything and everything you want about security, generating tons of data, and in the end you will have to make a judgement:  “Am I secure?” — or are my security provisions adequate given the threats we face?   Seen this way, your data is really just evidence that is used in this judgement (and inference) process.   What I mean by this is that I don’t think you can simply calculate your way from ground-truth data to any overall security metrics.  There will always be a judgement or inference step(s).

Why?  Because we must account for events, circumstances, and scenarios that haven’t happened yet, or happen so rarely that we have no relevant data, or are beyond the reach of measurements.  (Afterall, the miscreants often do their best to hide their actions.)   On top of this, the security landscape changes rapidly and occasionally dramatically.  Our judgement about security must factor in these changes, to the best of our knowledge.   Finally, our judgement about “are we secure?” is predicated on our risk tolerence.  But what is “risk”?

“Risk” is a cost of the future, brought to the present

This is the economist’s definition of risk, where “cost” here means downside cash flows that are beyond some  threshold of expectation or variability.  Those costs become “risk” when you can account for them in present dollars using some discounting and insurance method.  (This says nothing about the “insurability” of the risk, only about the theoretical possibility of accounting for risk in present dollars by some reasonable method.  The “insurance method” might be diversification, hedging, self-insurance, risk pooling, contingent contracts, or traditional insurance.)

This parallels Peter Drucker’s characterization of profit: “Profit is … needed to pay for attainment of the objectives of the business. Profit is a condition of survival. It is the cost of the future.  The cost of staying in business.” [emphasis added]   Ontologically, “profit” and “risk” are in the same category, which is why it makes sense to measure “risk-adjusted return” and the like.

From the viewpoint of risk, what you have spent in the past is irrelevant  (“sunk costs”).  All rational decisions are based on future cash flows and options.  The only value of the past is if it helps you predict or forecast the future.  Thus, you can’t reach a final judgement about security in the present if you don’t also have some useful estimate of risk in the future.   If the answer to “Am I secure?” is “Yes”, then the implication is that you can live with the risk associated with this level of security.   By “useful”, I mean sufficiently discriminating to inform the judgement — “bigger than a breadbox, smaller than a house”.

This is where information security deviates from reliability engineering.   In the latter, the ergodic hypothesis holds and the dynamics are sufficiently “tame” to permit statistical data analysis for inference and forecasting.  Even when there are “humans in the loop”, their behavioral tendencies can often be characterized by stable probability distributions.  In information security, we are dealing with adaptive, intelligent, strategic players — not only miscreants, but also “ancillary players” like end-users, auditors, supply chain partners, and so on.  This makes risk estimation a ”wicked problem“.  But is it hopeless?

Estimating risk may be hard, but not impossible

Plenty of smart security people contend that quantitative risk estimation is impossible or infeasible in principle.  Proving or disproving this assertion would take heavy-duty theoretical analysis (and I may do it some day).  But for now consider two extreme situations.

Think of security and risk as a black-box process that generates a continuous stream of cash flows in time (i.e. total spending on security and losses in that time period).  At one extreme, the output is a stationary function or stochastic process.  This is the relm that Nicholas Taleb called “Medicoristan“, since the data stream is well-behaved enough that nothing very surprising happens.  With enough historical data and enough data analysis, I think we’d all agree that risk estimation is feasable with current methods.

At the other extreme, the output is generated by a strategic agent (inside the box) whose sole purpose is to screw up our risk estimation process.  Let’s call this Descartes’ Demon, after Rene Descartes, who introduces a skeptical scenario called the deceiving demon argument to challenge our beliefs that an external world exists; in particular, it raises the possibility that some sort of malicious, demonic non-God, has “employed all his energies in order to deceive me”.    If Descartes’ Demon can maintain history of the output and also has information about our risk estimation process, he can mimic any output pattern and change those patterns arbitrarily to defeat any estimation process we might apply.   (This is more extreme than Taleb’s “Extremestan” in terms of defying estimation or prediction.)   In this case, I believe it could be proved that estimation is impossible (or undecidable or infeasable from a computation point of view).

Some people might argue that information security is exactly in this latter extreme situation, but I don’t think so.  The reason is that all the players have much stronger motives and forcing functions than to subvert the risk estimation processes.  Bad guys want to make money or cause harm.  End users want to avoid hassles and minimize effort and get their job done.  Managers want to manage their business while avoiding negative repercussions.  All of these factors add some elements of predictability and understandability.

But it may only be possible to factor all of these in through the use of models and simulations that represent our best knowledge, our best estimates, and our best beliefs about how they all relate to each other and the overall results.

The marriage of data, security, and risk = social learning processes

Putting this all together, we need to gather a lot of empirical data to understand relationships, patterns, and dependencies.  But to measure security we need to add inference and judgement processes that extend our data into the present, given the threat landscape we believe we are facing.  But to make a judgement about security and make decisions about alternative security postures, we need a useful estimate of risk to decide how much security is enough.  To tie these all together over time requires effective social learning processes, including model validation through experiments and data analysis.  Likewise, risk estimation and security judgement processes tell us what data we need to collect and how to analyze it.

Whether you agree with this framework or not, you should make explicit and consistent definitions of the time dimension relative to your metrics.