“While many are pushing back on cloud computing due to security concerns, cloud computing is, in fact, as safe as or better than most on-premises systems. You must design your system with security, as well as data and application requirements in mind, then support those requirements with the right technology. You can do that in both public or private clouds, as well as traditional systems.”

In a sense, David is right, the ability to develop a relatively secure computing architecture in a cloud environment, in theory, may be reasonably similar to “traditional” computing.  But there’s two things I hate about this paragraph.  First, it seems to reflect this naive notion that systems are deployed secure until vulnerability happens. Second, it completely misses the issue facing security management.  The problems facing management re: The Cloud have nothing to do with ability to architect “secure”.  They have to do with the ability to manage risk.

A Primer About Information Security and Risk Management

Security, at its fundamental core, is not problem of poor network architecture development or poor software development practices.  Security is a problem of behaviors, those having to do with the interrelation of systems and people.  Managing risk is related, but very different in it’s nature.  Information risk management is a problem of information quality and decision making around those behaviors.  Information risk management requires:

• Knowledge about the asset landscape – Data from what studies we do have about data breaches and successful IT operations strongly correlate visibility (even the degree of visibility) and variability in the asset landscape to success and failure in IT and IT security.
• Knowledge about the threat landscape – types, frequency, strength, capability, and adaptability of the threat agents are among the bits of information required to know and understand risk.
• Knowledge about the controls landscape – control information is the ability to resist threats, so not just the technical feasibility of resistance, but also the operational (human skills/resources) contributions to that ability to resist.
• Knowledge about the impact landscape - impact information from pressures within the organization (things like response costs, downtime, and productivity losses) and from outside the organization (compliance fines, legal judgments, the consequences of IP loss, brand damage…).

In addition, there’s knowledge we synthesize when we consider one landscape in the context of another (vulnerability might be said to be the a state we develop when we consider threat, asset, and control landscape information, risk is what we  understand when we consider the information we have from all four).  In the diagram, it’s where the circles overlap.

I’m sorry if this is basic for many of you readers out there, but I thought this content was necessary – because it’s obvious cloud architect types are obsessing over the ability to build a similar technical environment without understanding the basics of managing risk.

What Really Bugs Security Managers About Cloud Computing

So the issue with moving to “the cloud” for a CISO has to do with two basic things:

1. Information quality
2. Responsibility (data ownership in CISSP terms).

For information quality, we are concerned with:

• A – The ability to get reasonably similar information for the technical behavior information of our systems, and
• B – The human behavior information from both the threat and the controls landscape.

For Responsibility, we are concerned with:

• If the information is bad news, who is repsonsible for what actions?
• Given threat execution (the bad news isn’t just an attack, it’s a compromise) When a data breach occurs, where will the buck ultimately stop?

For that last bullet, PCI is sort of establishing a “case law” for us already.  The lesson to take away from the experiences of others is this:  Following the suggestions of CSA documents and Cloud Audit information (excellent, necessary, and as useful as that documentation is/will be) isn’t going to be enough to manage risk in the cloud with the same quality as “traditional” architectures for many people.  And it looks like you will be left as “custodians” of the data regardless of who is paying the W-2 of the guy at the SEIM console.  More colloquially, “Crap will continue to run downhill, you’ve just diverted it a ways upstream.”

Takeaways

The objections to cloud adoption from an information risk management standpoint have nothing to do with the ability to engineer “secure”.   It is about an ability to manage risk.  There’s a significant difference there that this sort of advocacy continues to gloss over.  Of course, given how nascent information security and information risk managent are as disciplines, however, if you can transfer full responsibility to a cloud provider who is stupid enough to believe things like “we can secure your systems better than you can”, I say go for it!

“This oil spill is a classic example of a black swan (events with the potential for severe impact to business and a low rate of occurrence)[vi].”

No.  No it’s not.  A Black Swan is something for which our prior distributions are completely uninformative.  In this case there was plenty of prior information about Deepwater, both from a “macro-analytical” standpoint (frequency & impact of oil well accidents) and from a “micro-analytical” standpoint (there were plenty of warnings about mis-management leading right up to the spill).

Now some of you readers will be thinking “there goes Alex again, waging war against Taleb’s stupid mischaracterization of ‘black swan’” and yes, Gideon is using “black swan” when he means “tail event” – I don’t blame him for that, it’s a common error perpetuated by that awful book.  Bear with me….

That’s not my point today.  What is important is this:

We (the risk & data analysts of the world) need to be really careful about how we’re communicating to management.  Saying that Deepwater was a “Black Swan”  or more properly, a “tail event” can allow someone to think that they just got “unlucky”.  This is crap.  BP did not get unlucky, they got cheap, lazy, and sloppy.  And not just at the well, either.  If (and this is just an “if”) upper management’s tolerance for risk was NOT reflected by the singular judgement calls made to circumvent appropriate safety controls, then upper management suffered what some would call a “governance” problem (I use the term very begrudgingly here – more on that in a bit), and a significant one at that.  And since rant mode is on, let me explain that this is one thing that bugs me about IT or Op risk assessments – the impact of organizational behavior is rarely taken into account.  Take, for example, R=TxVxI (please?).  ”V” is not just the weakness in the system we see, it is a cocktail of operational skills, resources, management (don’t make me say governance here, please), and yes, even “luck”.

SO the lesson here might just be that risk communication (and before you go there, IHMO COSO is self-defeating – see below) is a significant part of the risk analysis determination.  We security people focus on “upwards” communication of risk – trying to educate C-levels about the dangers they face.  But I’d bet that if an organization is incapable of communicating tolerance effectively from the top down, then they are likely to have more problems than those that don’t. There can be a time-lapse problem (Jaynesian entropy if I can use that term) between the operational happenings (what’s going on at the well) and the ability of those ultimately accountable (sr. mgmt) to detect, respond, and prevent risk issues from happening.

Even worse?  We’re keen on adding more bureaucracy to solve the communications problem in the name of “recognizing” and “managing” risk (GRC, ERM councils, Legal departments, bleh).  But in an organization the size of BP, a “GRC Dashboard” just isn’t going to solve the “micro-analytical” problems faced at Deepwater (assuming that BP executive management would have had a lower tolerance for probable incidents than the decision makers at the well).

The lumbering ogre of Enterprise Governance is no replacement for real quality management.

One can only imagine if BP had an Operational Risk Program like our standards and consultants tell us we should be operating.  What are the chances that the problems at the well would have been politically covered up, or been part of a 24 month “Enterprise Risk Assessment”, with Deepwater’s issues being one of (hundreds of?) thousands of individual risk issues documented very nicely and expensively, but never effectively communicated to the board?

There has GOT to be a better way.

CobIT allows to segregate what is called IT in analysable parts.  Different Risk models apply to those parts. e.g. Information Security, Architecture, Project management. In certain areas the risk models are more mature (Infosec / Project Management) and in certain they are not (software distribution). That is for the risk modelling (sic) part.

For risk identification and KRIs (note to readers:  I’m assuming Oliver means Key Risk Indicator – a useful but loaded phrase itself), an internal control framework which is based on cobit allows an adequate and comprehensive net of indicators for risk assessment based on operational performance.

If you think that “some things can’t be measured” will prove your thesis, you don’t know Risk Management at all.

There is no mathematical voodoo to model a risk exposure which is 100% correct.

You have to keep the purpose in mind and also use professional judgment based on your experience (which CRISC by the way tries to attestate)

You fight against an attestation which takes into full consideration your own challenge.

(…I’d argue that IRM shouldn’t be part of an MIS course load, rather it’s own tract with heavier influences from probability theory, history of science, complexity theory, economics, and epidemiology than, say, Engineering, Computer Science or MIS.)

IRM is not (just one) “process”. Now obviously certain risk management standards (document a simple) process. In my opinion, most risk management standards are nothing BUT a re-iteration of a Plan/Do/Check/Act process. And just to be clear, I have no problems if you want to go get certified in FAIR or OCTAVE or Blahdity-Blah – I’m all for that.  That shows that you’ve studied a document and can regurgitate the contents of that document, presumably on demand, and within the specific subjective perspective of those who taught you.

And similarly if ISACA wants to “certify” that someone can take their RiskIT document and be a domain expert at it, groovy.  Just don’t call that person “Certified in Risk and Information Systems Control™” because they’re not.  They’re “Certified in our expanded P/D/C/A cycle that is yet another myopic way to list a bajillion risk scenarios in a manner you can’t possibly address before the Sun exhausts it’s supply of helium.” “TM”

Lets be PROACTIVE instead of critical. I would love to hear about what CAN be a better job practice and skill set that is needed. I am working on both the commercial and Department of Defense and develop programs for training and coaching the skills from MBA to IT Audit and all of technical security for our Certification of Information Assurance Workforce and conduct all the CISM/CISA training and review courses for ISACA in both commercial and military environments. I have worked on Risk Management for years at ERM as well as IT Security/Risk, and A common theme in all of this is RISK MANAGEMENT. When I discuss the Value of IT with MBA students or discuss CMMI with MIS students or development houses, or discuss why ITIL/Cobit or other discuss with business managers what will keep them from reaching their goals and objectives, it is ALL risk management put into a different taxonomy that that particular audience can understand.

I have not been impressed with the current Risk Management certifications that are available. I did participate in the job task analysis of ISACA (which is a VERY positive thing about how ISACA keeps their certifications) more aligned to practice. It is also not perfect, but I think it is a start. If we contribute instead of just complain, it can get better, or we can create something better. What can be better?

So Alex I welcome a personal dialog with you or others on what and how we can do it better. I can host a web conference and invite all who want to participate (upto 100 attendee capacity).

Recently, a couple of folks have been talking about how security should just be a “diligence” function, that is, we should just prove that we’re doing best efforts and that should be enough.  Now conceptually, I love the idea that we can prove our “compliance” or diligence and get a get out of jail free card when an incident happens.  I always think it’s lame when good CISO’s get canned because they got “unlucky”.

Unfortunately, if risk management is infeasible, I’ve been thinking that the concept of Due Diligence Security is complete fantasy.  To carry the analogy, if Risk Management is the United Nations, then Due Diligence Security is the Justice League of Superfriends.  With He-Man.  And the animated Beatles from Yellow Submarine.  That live in the forrest with the Keebler elves and the Ewoks and where the glowing ghosts of Anakin, Obi-Wan and Yoda perform the “Chub-Chub” song with the glowing ghosts of John Lennon and George Harrison. That sort of fantasy.

DUE DILIGENCE BASED SECURITY IS AN ARGUMENT FROM IGNORANCE

Here’s the rub – lets say an incident happens.  Due Diligence only matters when there’s a court case, really.  And in most western courts of law these days, there’s still this concept of innocent until proven guilty.  This concept is known as the argument from ignorance in logic and it is known as a logical fallacy.

Now arguments from ignorance are known as logical fallacies thanks to the epistemological notion of falsification.  Paraphrasing Hume paraphrasing John Stuart Mill – we cannot prove “all swans are white” simply because we’ve observed all white swans -  BUT the observation of a single black swan is enough to prove that “not all swans are white”.   This matters in a court of law, as your ability to prove Due Diligence as a defendant will be a function your ability to prove all swans white – all systems compliant.  But the prosecution only has to show a single black swan to prove that you are NOT diligent.

Sir Karl Popper says, “Good luck with that, Mr. CISO”.

IT’S A TRAP!!!

The result is this – the CISO, in my humble opinion, will be in a worse condition because we have a really poor ability to control the expansion of sensitive information throughout the complex systems (network, system, people, organization) for which they are responsible.  Let me put it this way:  If information (and specifically, sensitive information) operates like a gas, automatically expanding to where it’s not controlled – then how can we possibly hope that the CISO can control the “escape” or leakage of information 100% of the time with no exceptions?  And a solitary exception in a forensic investigation becomes our black swan.

And therefore…   When it comes to proving Due Diligence in the court of law  – Security *screws* the CISO.  Big Time.

“A metric for Governance is only useful inasmuch as it describes an ability to manage risk”

True or False, why, and what are the implications if true or false.

Please discuss.

#newschoolsecurity

Our favorite colliquist, Anton Chuvakin, posted a provocative challenge in his blog post “Is Risk Just Too Risky?” :

What is the risk-driven, correct frequency of changing my email password?

<crickets…. silence… more silence>

Yes, we all can quote that “PCI DSS says 90 days” or “whatever regulation says 30 days”, but what does risk say? What actuarial information we need – if we are to define risk through probability of loss? What info about my email usage? Value of information stored there? Frequency of attacks on other similar email accounts? Chances of attack success? My approach to protecting the password? My personal password reuse “policy?” Anything else? On a related note, maybe this is simpler: what is my risk [of having the account compromised] if I change the password every 30 days, 90 days, 300 days?

So, any idea how to go about it?

This little experiment might well show us that “risk-based security” is an awesome thing – but not one achievable in this world today… [emphasis in original]

I wanted to blog about this, but hadn’t collected enough specifics.  Now I can, thanks to the blog conversation by David Mortman, Rich Mogull,  Chris Popper, and “Steve”, we have some smart/experienced people providing the needed detail.

Below, I offer a method for reasoning in order to estimate relative risk of alternatives that is compatible with quantitative risk analysis management, but doesn’t require massive amounts of risk calculations.  I use the conversation by Mortman, et. al. as an example of this method in action (armchair-style).

The Method — Abductive Validation

The following is a fairly generic method to guide decisions when you only have partial information/evidence and a rough estimate of overall risk.  It is a form of abductive validation, which is reasoning to the best explanation from available evidence along with evaluation criteria.  [Update: it's also called "Analysis of Competing Hypothesis" in the Intelligence community.  C.f.  book .  There is a cool extension using Subjective Logic described in slides and a paper]

Step 1.  Frame your decision alternatives in terms of hypotheses that could, in principle, be refuted with enough evidence.  In this case, I propose the following hypotheses:

1. Risk (Policy: “Change password every 90 days”) < Risk (Policy: none)   
   [the policy decreases risk ]
2. Risk (Policy: “Change password every 90 days”) > Risk (Policy: none)  
   [the policy increases risk]
3. Risk (Policy: “Change password every 90 days”) = Risk (Policy: none)   
   [the  policy makes no difference in risk]
4. Risk (Policy: “Change password every 90 days”) ? Risk (Policy: none)    
   [i.e. "we don't know".  It's indeterminant, unresolved, etc.]

[Update:  having these four hypotheses is critical to the method, rather than just a single hypothesis such as #1.  The reason is that different pieces of evidence may support or contradict one or more of these hypothesis.  For example, just because a given piece of evidence contradicts #1 doesn't mean it supports #2 or #3.]

Step 2. Define the Risk( ) function to make it operational.  

Start with a metric for aggregate risk for the whole business unit.   I prefer “Total Cost of Security” but other aggregate risk assessment methods  will do (e.g. NIST 800 series) .  Next, go through the steps to reach at least a rough estimate of aggregate risk.  In the process, you should be able to identify the operational factors that have the biggest effect on aggregate risk.  These are called “risk drivers“, and the concept is analogous to  “cost drivers” in Activity-based Costing.  You should be able to rank order them by degree of impact.  Any thing that increases the risk drivers also tends to increase the aggregate risk metric.

Then, define your Risk( ) function in relation to these risk drivers.  This is the key to making this method tractable yet also sufficient to guide decisions using quantitative risk management principles.  In other words, you aren’t seeking a risk function that relates the policy variable directly to aggregate risk.  Instead, you relate the policy variable to the risk drivers. 

Here’s a simple example using email password policy.  Let’s say your organization has only three risk drivers: 1) Number of confidentiality breaches of person-to-person communications, 2) Number of breaches of end-user email accounts, and 3) Number of non-employees who have access internal information system.  The Risk( ) function would be defined according to the effect that password policy had to increase or decrease any of these risk drivers.  You can either use quantitative or qualitative functions to evaluate impact on the risk drivers.

Step 3. Collect evidence regarding each hypothesis, both pro and con.  Evidence can be quantiative, qualitative, or a mix.   (You’ll need formal rules for evaluating the quality and strength of the evidence, but to keep this post from getting too long, I won’t go into that. ) Stop when you have enough evidence to choose among the hypotheses, or you run out of time or money.  (If this happens, you are left with the inconclusive hypothesis #4, above.)

Step 4. Evaluate the evidence and decide to support or refute each hypotheses.  If multiple conflicting hypotheses are supported, then either collect more evidence to exclude one of the hypotheses, or accept a mixed or ambiguous conclusion.

Example of Arguments and Evidence — Pro and Con

Mortman’s blog conversation is an excellent example of how this evidence-based argumentation should be done.  This is an armchair debate, so they don’t follow the steps exactly, the discussion is very incomplete.  But I think it has enough specifics to show how a formal study could proceed.  (BTW, all uses of the word “evidence” in the commentary below is short-hand for “evidence that he thinks someone could collect…”.  No one is actually pointing to solid evidence.)  

Mortman starts with a challenge statement that expresses his support for hypothesis #3 (PW change policy makes no difference):

Show me any reasonable evidence that changing all your users’ passwords every 90 days reduces your risk of being exploited.

In the first comment, Steve offers evidence in support of hypothesis #1 (PW change policy reduces risk):

Aside from regular password change intervals, is there a way to mitigate offline brute-force attack?  Assuming an attacker uses any of a number of methods to grab a password hash, and that the hash isn’t some sort of weak LM silliness, an attacker is left with a long-running brute force process, depending on the computational power available.  For most organizations, a password change policy of 90 or 180 days would likely make the results of the brute force moot.

Given that offline brute-force is a realistic threat, isn’t a password change policy a reasonable control?

Mortman counters this argument in comment 2, not by refuting it, but by offering more evidence in support of hypothesis #3 (no difference):

It sounds like a realistic threat, except for the fact, that if someone has been able to get your password hashes, then they are unlikely to need to brute force passwords. They already have the access they need to get to the data that they want. If you own the authentication system, passwords no longer matter. Even if they need or want passwords, they now have the ability to capture them at will.

Steve counters Mortman in comment 3, again by offering evidence in support of hypothesis #1:

The specific scenario I was thinking of involved cracking an Active Directory domain member, and then dumping the hashes of the last 10 logged-in users (which is used to authenticate users when the domain controller is unavailable).  There’s a good chance that a regular user’s PC will have had a Help Desk or other more-privileged account logged in within the last 10, and by cracking that hash, the attacker would gain access to higher privileges.

Chris Pepper chimes in, basically agreeing with Mortman on hypothesis #3, appealing to evidence that such attacks are not frequent in most scenarios:

90-day password change policies are stupid in >90% of scenarios. There are probably some DoD scenarios under active attack where they make sense. The clowns who insist normal business systems need 30 or 90 day password expirations don’t mean *all* users should disbelieve *all* professional security advice.

Then he offers a counter argument to Steve, with evidence in support of hypothesis #3 (i.e. password strength matters much more than change frequency):

If your passwords are realistically brute-forceable in 90 days, they’re too short.

Then he offers evidence supporting Steve’s evidence in favor of hypothesis #1:

There are various ways you might get a UNIX /etc/shadow file from backups, so reversing password hashes is a real threat.

Mortman responds to Steve by proposing a sub-problem to evaluate (i.e. strength of has relative to time to crack):

Okay so lets take your scenario. The question you have to ask, is how long is the hash going to stand up to attack. With a strong hash, it’s going to be a heck of a lot longer then 90 or even a 180 days, possibly years. In that case, what’s the justification for changing it on a 90-180 day schedule?  Realistically though, what this means is that you now have a more complex risk question around how likely is it that someone is going to break in and get the hashes and how long are you willing for them to have use of those passwords?

Then Mortman asserts that this sub-problem may be unresolvable due to lack of evidence:

We at this point have little to no data on how likely this sort of attack is to occur so we can’t even take even a bad guess at if 90 days is a good number or a bad number. But until we have some data, we’re just making stuff up so make ourselves feel like we’re doing something.

Mogull chimes in, supporting hypothesis #3 (no difference) with other evidence (rarety of these sorts of attacks, agreeing with Chris Pepper):

Based on all the recent breach reports and investigations, it doesn’t look like password cracking is a major vector anymore (I’m not willing to stand behind that statement, but that’s my reading of these reports).

Finally, Mogull hints that there might be evidence in favor of hypothesis #2 (policy increases risk by increasing total costs):

With modern systems (no more NTLANMAN) is it really a risk? Is that risk greater than the cost of password rotations?

—–

And so on…

I hope this example gives you a feeling of how the Abduction Validation method can work in practice.  If you really care about the quality of the answer, you would need to be formalized through investigation, data collection, and experiements.  Also, by enumerating hypotheses in the way I described, this method has the great advantage of telling you when the evidence is inadequate to support any hypothesis or alternative.

“Baseball’s love of statistics is taking over football“

Those who indulge my passion for analysis and for sport know that I love baseball and love how the “Moneyball” approach challenged decades of dogma in the national pastime with scientific analysis.  Today’s financial times discusses how Chelsea (“The Blues” – UK football team) collaborates with the Boston Red Sox (the most superficial bandwagon team ever in baseball) on decision making and analytics.

Best lines:

“Mike Forde, Chelsea’s performance director, visits the US often. “The first time I went to the Red Sox,” he says of the Boston baseball team, “I sat there for eight hours, in a room with no windows, only flipcharts. I walked out of there saying, ‘Wow, that is one of the most insightful conversations on sport I have ever had.’ It was not: ‘What are you doing here? You do not know anything about our sport.’ That was totally irrelevant. It was: ‘How do you make decisions on players? What information do you use? How do we approach the same problems?’”

and:

“Forde sees his task as “risk management”.

Huh.

1. Assumption
2. Reasoning: The basis for the assumption.
3. Indicators: Specific cues that indicate whether the assumption is accurate or if there’s a problem in that area.
4. Controls: The security/recovery/safety controls to mitigate the issue.

Nothing earth shattering here.  And like much of Rich’s work, there is an elegance, almost a minimalism to what he offers.

JUST BECAUSE I CAN’T LEAVE WELL ENOUGH ALONE….

What immediately struck me was how similar Rich’s assumption was to a little something I like to call “scientific method”.  In scientific method, we essentially have (the following shamelessly pasted from Wikipedia):

• Characterizations (observations,^[24] definitions, and measurements of the subject of inquiry)
• Hypotheses^[25][26] (theoretical, hypothetical explanations of observations and measurements of the subject)^[27]
• Predictions (reasoning including logical deduction,^[28] from the hypothesis or theory) or the identification of distinct and (ideally) mutually exclusive possible discernible outcomes
• Experiments^[29] (tests of all of the above)

So if we were to add to Rich’s assumption process above, we’d simply add the “experiments” bits up there.  If we’re building controls in like Rich’s examples in his blog post, we might try a “test” that “penetrates” those controls (or, as I believe Richard Bejtlich smartly tries to get us to say, perform “Adversary Simulation”).

Also, though it will probably sour his stomach a bit, we’d also probably want to make Rich’s assumption steps a hamster-wheel-of-pain(TM) by suggesting that since every so often, the threat landscape will change which will challenge our assumptions/conclusions/hypothesis and so re-testing is necessary.

IF I HAD ANY INDICATION…

Rich does have a certain “informality” around his evidence “indication” step that I’d like to build upon.  Let me offer that when discussing probability of failure in a complex IT system, there are only four basic categories of information indicators we need to consider in Information Assurance/Security/Risk Management/Protection/Whatever.  There might be evidences around:

• Assets (the things we want to protect and their state)
• Threats (the things that want to harm our assets and their state)
• Controls (the things that resist the threats and their state)
• Impacts (the things that will happen if we are unable to resist the threat)

And if you’re going to look for clues to suggest whether there might be a problem, look no further than these basic categories for evidence.  If you’d like, you can build structure around what “state” means for each category and further develop taxonomies and metrics and whatnot.  That’s the fun bits and I’ll let you be creative rather than write too much this morning.

Note that where these categories applied to Assumption may break down is in discussing management capabilities (are we operating well enough and so forth).  Rich’s assumptive process (must.resist.urge.to.make.acronym – RAP) can certainly be used here, I’m just not sure if there wouldn’t be a better taxonomy of indicators.

Jerry escapes death, but is it cost-free?

If one of your security metrics is Data Breach Cost, what is the cost of a near miss incident? This seemingly simple question gets at the heart of security metrics problem.

Consider the gleeful Jerry Mouse in this cartoon. Tom the Cat has just missed in his attempt to swat Jerry and turn him into mouse meat. Is there any cost to Jerry for this near miss? Is Jerry’s cost any different than if he was running with Tom no where in sight?

By “near miss” I mean a security incident or sequence of incidents that could have resulted in a severe data breach (think TJX or Heartland), but somehow didn’t succeed. Let’s call the specific near-miss event “NM” for short. For sake of argument, let’s assume that the lack of attack success was due to dumb luck or attacker mistakes, not due to brilliant defenses or detection. Let’s say that you only discover NM long after the events took place. For simplicity let’s assume that discovering NM doesn’t result in any extraordinary costs, meaning that out-of-pocket costs are the same just before and immediately after NM. Finally, assume that your expected cost of a successful large-scale data breach is on the order of tens of millions, with the worst case being hundreds of millions of dollars.

How much does NM cost?  The realist answer is “zero”.  (Most engineers are realists, by disposition and training.)  There is a saying in street basketball that expresses the realist philosophy about losses and associated costs: “No blood, no foul”.  If you ask your accountants to pour over the spending and budget reports, they will probably agree. Case closed, right?

Not so fast….

The big problem with the realist approach is that it ignores the future and our rational expectations about future loss events. In other words, it ignores risk. It’s like the old joke about the guy who fell out of a 20-story building. As he passed the 4th floor, someone called out to him, “ARE YOU OK?”, to which he replied: “SO FAR, SO GOOD!!”. (Moments later… splat!)

We know intuitively that there is something wrong with the answer “so far, so good” when the signs of pending disaster appear.

Economists will arrive at a very different answer to account for this intuition. For economists, valuation and risk decisions are about the future, and especially about rational expectations about future cash flows and future valuations given available information. If you get significant new information that changes your expectations, then your risk and value metrics will change.

You could hardly imagine a more meaningful signal regarding risk than a near miss event. Safety engineers have known this for decades and it’s central to their practice.  (For example, see the book: Safety Management: A Qualitative Systems Approach and the web page: “Three Simple Things to Improve Process Safety Management”.)  What ever your estimation of risk before NM, it will probably go way up after NM.  Economists would argue that this increases your data breach costs, since your expectation of future cash flows has increased.

Does this economic cost of a data breach have any reality?  How could it be made tangible and meaningful for accountants and ordinary realistic managers?  Yes it can, through insurance. Imagine that your organization pays a regular insurance premium that is a probabilistic function of future data breach costs, based on all available information about likelihood and severity. (Assume either self-insurance or commercial insurance, or some combination. Assume “perfect pricing” and complete information sharing, etc.)  Forget about risk transfer. The purpose of insurance in this case is simply bringing the cost of risk into the present.

With this insurance in place, your data breach cost becomes not only the actual cash flows associated with loss events, but also the periodic insurance premiums, which would rise or fall based on risk factors and risk estimates. We are familiar with this from our experience with auto insurance, property and casualty insurance, etc.

The great advantage of this approach is that your data breach cost metrics will become a meaningful signal for management decision-making, performance management, and incentive instruments. All stakeholders will be more likely to pay attention to near misses and, hopefully, do their best to learn from them and mitigate risks.

Whether or not you buy into the details of the insurance mechanism, I hope that I have convinced you that there is a qualitative difference between “ground truth data” (in this case, historical cash flow) and overall security metrics, which need to reflect our estimates about the future, a.k.a. risk.