“This oil spill is a classic example of a black swan (events with the potential for severe impact to business and a low rate of occurrence)[vi].”

No.  No it’s not.  A Black Swan is something for which our prior distributions are completely uninformative.  In this case there was plenty of prior information about Deepwater, both from a “macro-analytical” standpoint (frequency & impact of oil well accidents) and from a “micro-analytical” standpoint (there were plenty of warnings about mis-management leading right up to the spill).

Now some of you readers will be thinking “there goes Alex again, waging war against Taleb’s stupid mischaracterization of ‘black swan’” and yes, Gideon is using “black swan” when he means “tail event” – I don’t blame him for that, it’s a common error perpetuated by that awful book.  Bear with me….

That’s not my point today.  What is important is this:

We (the risk & data analysts of the world) need to be really careful about how we’re communicating to management.  Saying that Deepwater was a “Black Swan”  or more properly, a “tail event” can allow someone to think that they just got “unlucky”.  This is crap.  BP did not get unlucky, they got cheap, lazy, and sloppy.  And not just at the well, either.  If (and this is just an “if”) upper management’s tolerance for risk was NOT reflected by the singular judgement calls made to circumvent appropriate safety controls, then upper management suffered what some would call a “governance” problem (I use the term very begrudgingly here – more on that in a bit), and a significant one at that.  And since rant mode is on, let me explain that this is one thing that bugs me about IT or Op risk assessments – the impact of organizational behavior is rarely taken into account.  Take, for example, R=TxVxI (please?).  ”V” is not just the weakness in the system we see, it is a cocktail of operational skills, resources, management (don’t make me say governance here, please), and yes, even “luck”.

SO the lesson here might just be that risk communication (and before you go there, IHMO COSO is self-defeating – see below) is a significant part of the risk analysis determination.  We security people focus on “upwards” communication of risk – trying to educate C-levels about the dangers they face.  But I’d bet that if an organization is incapable of communicating tolerance effectively from the top down, then they are likely to have more problems than those that don’t. There can be a time-lapse problem (Jaynesian entropy if I can use that term) between the operational happenings (what’s going on at the well) and the ability of those ultimately accountable (sr. mgmt) to detect, respond, and prevent risk issues from happening.

Even worse?  We’re keen on adding more bureaucracy to solve the communications problem in the name of “recognizing” and “managing” risk (GRC, ERM councils, Legal departments, bleh).  But in an organization the size of BP, a “GRC Dashboard” just isn’t going to solve the “micro-analytical” problems faced at Deepwater (assuming that BP executive management would have had a lower tolerance for probable incidents than the decision makers at the well).

The lumbering ogre of Enterprise Governance is no replacement for real quality management.

One can only imagine if BP had an Operational Risk Program like our standards and consultants tell us we should be operating.  What are the chances that the problems at the well would have been politically covered up, or been part of a 24 month “Enterprise Risk Assessment”, with Deepwater’s issues being one of (hundreds of?) thousands of individual risk issues documented very nicely and expensively, but never effectively communicated to the board?

There has GOT to be a better way.

CobIT allows to segregate what is called IT in analysable parts.  Different Risk models apply to those parts. e.g. Information Security, Architecture, Project management. In certain areas the risk models are more mature (Infosec / Project Management) and in certain they are not (software distribution). That is for the risk modelling (sic) part.

For risk identification and KRIs (note to readers:  I’m assuming Oliver means Key Risk Indicator – a useful but loaded phrase itself), an internal control framework which is based on cobit allows an adequate and comprehensive net of indicators for risk assessment based on operational performance.

If you think that “some things can’t be measured” will prove your thesis, you don’t know Risk Management at all.

There is no mathematical voodoo to model a risk exposure which is 100% correct.

You have to keep the purpose in mind and also use professional judgment based on your experience (which CRISC by the way tries to attestate)

You fight against an attestation which takes into full consideration your own challenge.

(…I’d argue that IRM shouldn’t be part of an MIS course load, rather it’s own tract with heavier influences from probability theory, history of science, complexity theory, economics, and epidemiology than, say, Engineering, Computer Science or MIS.)

IRM is not (just one) “process”. Now obviously certain risk management standards (document a simple) process. In my opinion, most risk management standards are nothing BUT a re-iteration of a Plan/Do/Check/Act process. And just to be clear, I have no problems if you want to go get certified in FAIR or OCTAVE or Blahdity-Blah – I’m all for that.  That shows that you’ve studied a document and can regurgitate the contents of that document, presumably on demand, and within the specific subjective perspective of those who taught you.

And similarly if ISACA wants to “certify” that someone can take their RiskIT document and be a domain expert at it, groovy.  Just don’t call that person “Certified in Risk and Information Systems Control™” because they’re not.  They’re “Certified in our expanded P/D/C/A cycle that is yet another myopic way to list a bajillion risk scenarios in a manner you can’t possibly address before the Sun exhausts it’s supply of helium.” “TM”

Lets be PROACTIVE instead of critical. I would love to hear about what CAN be a better job practice and skill set that is needed. I am working on both the commercial and Department of Defense and develop programs for training and coaching the skills from MBA to IT Audit and all of technical security for our Certification of Information Assurance Workforce and conduct all the CISM/CISA training and review courses for ISACA in both commercial and military environments. I have worked on Risk Management for years at ERM as well as IT Security/Risk, and A common theme in all of this is RISK MANAGEMENT. When I discuss the Value of IT with MBA students or discuss CMMI with MIS students or development houses, or discuss why ITIL/Cobit or other discuss with business managers what will keep them from reaching their goals and objectives, it is ALL risk management put into a different taxonomy that that particular audience can understand.

I have not been impressed with the current Risk Management certifications that are available. I did participate in the job task analysis of ISACA (which is a VERY positive thing about how ISACA keeps their certifications) more aligned to practice. It is also not perfect, but I think it is a start. If we contribute instead of just complain, it can get better, or we can create something better. What can be better?

So Alex I welcome a personal dialog with you or others on what and how we can do it better. I can host a web conference and invite all who want to participate (upto 100 attendee capacity).

“A metric for Governance is only useful inasmuch as it describes an ability to manage risk”

True or False, why, and what are the implications if true or false.

Please discuss.

#newschoolsecurity

Jerry escapes death, but is it cost-free?

If one of your security metrics is Data Breach Cost, what is the cost of a near miss incident? This seemingly simple question gets at the heart of security metrics problem.

Consider the gleeful Jerry Mouse in this cartoon. Tom the Cat has just missed in his attempt to swat Jerry and turn him into mouse meat. Is there any cost to Jerry for this near miss? Is Jerry’s cost any different than if he was running with Tom no where in sight?

By “near miss” I mean a security incident or sequence of incidents that could have resulted in a severe data breach (think TJX or Heartland), but somehow didn’t succeed. Let’s call the specific near-miss event “NM” for short. For sake of argument, let’s assume that the lack of attack success was due to dumb luck or attacker mistakes, not due to brilliant defenses or detection. Let’s say that you only discover NM long after the events took place. For simplicity let’s assume that discovering NM doesn’t result in any extraordinary costs, meaning that out-of-pocket costs are the same just before and immediately after NM. Finally, assume that your expected cost of a successful large-scale data breach is on the order of tens of millions, with the worst case being hundreds of millions of dollars.

How much does NM cost?  The realist answer is “zero”.  (Most engineers are realists, by disposition and training.)  There is a saying in street basketball that expresses the realist philosophy about losses and associated costs: “No blood, no foul”.  If you ask your accountants to pour over the spending and budget reports, they will probably agree. Case closed, right?

Not so fast….

The big problem with the realist approach is that it ignores the future and our rational expectations about future loss events. In other words, it ignores risk. It’s like the old joke about the guy who fell out of a 20-story building. As he passed the 4th floor, someone called out to him, “ARE YOU OK?”, to which he replied: “SO FAR, SO GOOD!!”. (Moments later… splat!)

We know intuitively that there is something wrong with the answer “so far, so good” when the signs of pending disaster appear.

Economists will arrive at a very different answer to account for this intuition. For economists, valuation and risk decisions are about the future, and especially about rational expectations about future cash flows and future valuations given available information. If you get significant new information that changes your expectations, then your risk and value metrics will change.

You could hardly imagine a more meaningful signal regarding risk than a near miss event. Safety engineers have known this for decades and it’s central to their practice.  (For example, see the book: Safety Management: A Qualitative Systems Approach and the web page: “Three Simple Things to Improve Process Safety Management”.)  What ever your estimation of risk before NM, it will probably go way up after NM.  Economists would argue that this increases your data breach costs, since your expectation of future cash flows has increased.

Does this economic cost of a data breach have any reality?  How could it be made tangible and meaningful for accountants and ordinary realistic managers?  Yes it can, through insurance. Imagine that your organization pays a regular insurance premium that is a probabilistic function of future data breach costs, based on all available information about likelihood and severity. (Assume either self-insurance or commercial insurance, or some combination. Assume “perfect pricing” and complete information sharing, etc.)  Forget about risk transfer. The purpose of insurance in this case is simply bringing the cost of risk into the present.

With this insurance in place, your data breach cost becomes not only the actual cash flows associated with loss events, but also the periodic insurance premiums, which would rise or fall based on risk factors and risk estimates. We are familiar with this from our experience with auto insurance, property and casualty insurance, etc.

The great advantage of this approach is that your data breach cost metrics will become a meaningful signal for management decision-making, performance management, and incentive instruments. All stakeholders will be more likely to pay attention to near misses and, hopefully, do their best to learn from them and mitigate risks.

Whether or not you buy into the details of the insurance mechanism, I hope that I have convinced you that there is a qualitative difference between “ground truth data” (in this case, historical cash flow) and overall security metrics, which need to reflect our estimates about the future, a.k.a. risk.

Does it make sense to never change your information security strategy? That’s a possible consequence of the so-called two-envelope paradox. This is a problem in probability theory that has confused students of probability theory for over 50 years.

[explanation of the problem and how it applies to InfoSec investments]

The bottom line is probably that probability is a complicated and subtle concept, which means that risk management, which relies on it, also is.

Luther and I agree on his bottom-line statement.  This stuff can be complicated and subtle.  It is easy for college-educated professionals and executives to make mistakes that lead to erroneous conclusions.  In fact, Luther’s own blog post is a case study in how easy it is to make mistakes.  

Luther uses the “Two Envelopes Problem” in probability theory as an example where Bayesian (subjectivist) probability methods seem to break down in paradox.  Here’s the problem and paradox, leaving out the math for now  (details at the end):

The player is given two indistinguishable envelopes, each of which contains a positive sum of money. One envelope contains twice as much as the other. The player may select one envelope and keep whatever amount it contains, but upon selection, is offered the possibility to take the other envelope instead ["switch"]. 

[...Bayesian analysis, leading to the decision to "switch", and then "switch" again, ad infinitum...]

As it seems more rational to open just any envelope than to swap indefinitely, the player is left with a paradox.

Following Wikipedia, Luther points out that ”There’s a problem with this argument, of course, but it’s fairly subtle. Even specialists in probability theory don’t agree what the problem actually is,…”.  Luther then applies Two Envelopes Problem structure to an analogous problem in information security (I’ll call it “Two Technologies”):

Now let’s suppose that we can’t find a flaw in the above argument and we apply it to our information security strategy. Let’s suppose that we have some initial set of technology, policies and procedures that end up giving us some exposure to risk that we’ll denote R, and if we change to a different set of technology, policies and procedures, we might either increase the risk to 2R or decrease it to R/2. If we apply the same reasoning that we applied above, we find that it never pays to change, because the alternative always has a greater than the risk than what we have now. This clearly doesn’t make sense, but it’s what you might get if you do a risk analysis that isn’t as careful as it could be.

This seems OK on the surface, but Luther has made a mistake in framing the Two Technologies problem.  Simply put, his Two Technologies problem is not structured the same as the Two Envelopes problem.  Therefore Bayesian formulas for the Two Envelopes does not fit his Two Technologies problem.  Even if it did, a proper framing of the Bayesian analysis avoids the paradox. (For math details, see below).

So, yes, probability and risk management can be complicated and subtle.  But Luther’s use of the Two Envelopes Problem and his attempt to construct the same problem in InfoSec only supports this conclusion through the “covert channel” of his own mistakes.   

Is Luther implying that we shouldn’t even try risk assessment and management because it’s beyond our ken?  “Yes” seems to be the answer, if I understand his posts in the “risk” category.   What alternative to risk management is Luther proposing?  In another post, he suggest that we should just use trial-and-error to muddle through:

In the absence of reliable risk information, a similar approach to information security may be the best that we can do – just try different things and see which works the best. You might call this approach “experimental security.” There may be no better approach.

If someone can prove that risk assessment/management for InfoSec is impossible in principle, then Luther would be right.  But I know of no such proof of impossibility.  Just because we currently find risk management to be “complicated and subtle” doesn’t mean we should dump it.  We do need tools and frameworks that are up to the job and sufficiently usable to help ordinary people avoid the paradoxes and ratholes in the analysis.   As our academic colleagues would say: “further research is needed”.

(For mathematical details, read on…)

Mathematics of the Two Envelopes Problem, and a Solution

Luther describes the Two Envelopes Problem, then frames it using Bayesian (subjectivist) probability:

Suppose that you’re given two envelopes and you’re told that one envelope contains twice as much money as the other. You then open one of the envelopes and see how much money it contains. Based on this information, you decide to either keep the contents of the first envelope or to switch its contents for the contents of the second, unopened envelope.

It might seem that it always pays to switch.

Suppose you find $2 in the first envelope. You know that the other envelope either contains $1, which happens with probability 0.5, or it contains $4, which also happens with probability 0.5. So you can calculate the expected value of the second envelope as $1 x 0.5 + $4 x 0.5 = $2.5. Because this is greater than $2, it always pays to switch.

The paradox arises because the same logic applies after your first switch (assuming you don’t open the second envelope), which justifies a switch back to the first, and so on, ad infinitum.  The Wikipedia article lists several proposed solutions, but it may be hard for non-specialists to pick one over the other.  You might conclude that the solutions make the analysis more arcane and therefore less usable.  That may be true as long as you stick with the basic Bayesian formulation.

However, reframed in terms of ”possible worlds”, the Two Envelopes Problem ceases to be a paradox.  (For a thorough explanation of this approach, see: ”A Logical Approach to Reasoning About Uncertainty: A Tutorial“, by Joseph Halpern).    Here are the instructions:

1.  There are two envelopes, labeled arbitrarily “a” and “b”
2. Contents of a =X or 2X  (X is constant but unknown)
3. Contents of b =X if a=2X, else b =2X
   Thus, there are two possible worlds:
   W1: a =2X , b =X
   W2: a =X , b =2X
4. p = subjective probability or belief in a possible world
5. Pick a, and open it
6. Decide whether to keep a or switch to b

Possible worlds framework for Two Envelopes Problem

There are two decision alternatives to evaluate over the possible worlds: “keep” and “switch”.  To evaluate each alternative, trace each possible world path and multiply the outcome value by the subjective probability of that possible world (which, in this case, is identical for each = 0.5, because of no prior information):

• “Keep” outcome    = p(W1)* 2X + p(W2) * X  = 0.5 * 2X + 0.5 * X = 1.5X
• “Switch” outcome = p(W1)* X + p(W2)* 2X  =  0.5 * X + 0.5 * 2X = 1.5X

Result: the expected value for each of these decision alternatives is the same, 1.5X, which is the average value of the two envelopes.  Thus, you are indifferent on whether you should “keep” or “switch”, which matches common sense, absent any other information.

Very important to this analysis is that the actual dollar value revealed when you open the first envelope is not informative.  This should be a red flag when someone proposes to name a variable set to that value, as was done in the Bayesian analysis in the Wikipedia article, and also Luther’s post.

Mathematics of Luther’s “Two Technologies” Problem

Again, here is Luther’s description:

Now let’s suppose that we can’t find a flaw in the above argument and we apply it to our information security strategy. Let’s suppose that we have some initial set of technology, policies and procedures that end up giving us some exposure to risk that we’ll denote R, and if we change to a different set of technology, policies and procedures, we might either increase the risk to 2R or decrease it to R/2. If we apply the same reasoning that we applied above, we find that it never pays to change, because the alternative always has a greater than the risk than what we have now. This clearly doesn’t make sense, but it’s what you might get if you do a risk analysis that isn’t as careful as it could be.

Here are the instructions for framing the Two Technologies problem in terms of possible worlds:

1. There exists a new incremental investment in information security technology, T
2. Information security risk is r = probability of breach * loss due to breach
3. Without T, r=R (a constant)
4. With T, r= R/2 or 2R
   Thus, there are three possible worlds :
   W0: r=R
   W1: r= 2R
   W2: r= R/2
5. p = subjective probability or belief in a possible world
6. Decide whether to invest in T 

Possible Worlds framework for Two Technologies Problem

There are two decision alternatives to evaluate over the possible worlds: “No T”  (don’t invest) and “Yes T” (do invest).  To evaluate each alternative, trace each possible world path and multiply the outcome value by the subjective probability of that possible world:

• “No T” outcome   r = p(W0)* R  = 1 * R = R
• “Yes T” outcome r = p(W1)* 2R + p(W2)* R/2  =  0.5 * 2R + 0.5 * R/2 = 1.25R

Result: the expected value “No T” (don’t invest) is R, which is less than the expected value of “Yes T” (do invest).  Therfore, don’t invest in the new technology.  This is the result that Luther reported in his post.

(Note: p sums to 1 for sets of possible worlds that are mutually exclusive, given any preceding decisions.  In this problem, there are two sets of mutually exclusive possible worlds: 1) W0 and 2) W1, W2.)

Here’s what’s wrong:  Luther made some errors in defining the Two Technologies problem, as you can see from a quick comparison of the two diagrams. They are not the same problems at all!  Specifically, there are three errors that make these different problems:

1. The decision variable comes after the state-of-the-word variable in the Two Evelopes problem, while in Luther’s Two Technologies problem they are reversed. 
2. More important, there is no uncertainty if the decision is “No T“.  The level of risk is deterministic.  This makes the problem different than Two Envelopes, where symmetrical uncertainty exists for both decision alternatives. 
3. Finally, impact of technology T on risk r is defined to be superficially similar to the Two Envelop problem with two possibilities: 1) risk is doubled or 2) risk is cut in half.   It is as if Luther is trying to embed the Two Envelopes problem inside the Two Technologies problem, but only along one decision path.   If, instead, the same uncertain state-of-the-world existed on both decision paths, then the results would have been the same as Two Envelopes problem we saw above: outcome values are the the same for both decision alternatives. 

If you think about the Two Technologies problem in simple language, then it becomes obvious why the new technology investment is unjustified.  Who would invest in security technology that has an equal chance of either reducing InfoSec risk by 50% or increasing it by 100%?  Change those risk impact possibilities to something more reasonable, like “no impact” vs. “50% reduction”, and the investment looks justified.  The Two Envelopes Problem is irrelevant in this case.

Conclusion

I hope I’ve been able to convince you that the Two Evelope problem really doesn’t say anything significant about information security risk management decision, and that Luther made some mistakes in trying to create an analogous Two Technologies problem. 

I also hope that this demonstration will help anyone who might be doing InfoSec risk analysis, especially how to frame the problem in a way that accounts for various forms of uncertainty, decisions, etc.  In addition to the Halpern referenced above, I also recommend his book: Reasoning About Uncertainty, MIT Press, 2003. Paperback issue published in 2005.

(Caution: This isn’t a simple how-to book. This book is fairly advanced, technical, and academic, though some of the introductions are fairly accessible to the non-specialist.  If you want to dip into specific topics, you can go to Halpern’s publication page.  However, these are almost all academic papers, which can be hard to read by non-specialists. 

If you are among the people who think probability, even Bayesian probability, is the be-all and end-all for reasoning about risk and uncertainty, then this material will hit you like a cold shower!)

I say “somewhat” modeling because I find interesting is that Richard seems to be discussing probabilistic statements in the “greens” there (“my opinion/guess is that it would take some degree of force to compromise asset in question based on my admittedly incomplete knowledge of the control state”) , and then adding to that nature-state assessments (“here is the current state of attack/compromise”) in the rest of the spectrum.

Now obviously the probabilistic statements would eventually require more effort once some substantial level of organizational maturity has been achieved, but this sort of asset-state spectrum would be useful in establishing high-level assessments of the asset landscape.  It would allow the CISO (esp. a new CISO who finds herself in an environment fraught with “unknown, unknowns”) to create a relatively quick view of enterprise “vulnerability” (to borrow FAIR’s concept) and help begin the achievement of knowledge around enterprise risk management capability.

Other uses could eventually contribute significantly to risk models.  Obtaining accurate current/past Category 3 (the orange there) information would be *really* useful for creating probabilistic statements around threat models and risk models.  As I mentioned above, the Vuln ratings can be broken down to assist risk and risk management modeling (yeah, I continue to maintain that they are two different things despite what all the standards out there say, more on that tomorrow).  Finally, studying time frames and abilities around the post-compromise state (Breach 3-1, there) around incidents would be useful in establishing risk management model belief statements, as well.

One small problem I would have using this on an aggregate level would be the lack of granularity around the threat community w/regards to “impact” levels 1-5.  I might not really care if a script kiddie is doing reconnaissance, but if the recon is coming from someone in an internal administrative role, a contractor who has privileges, etc…

That said, my significant problem with Bejtlich’s model there has to do with *why* he created it.  He asks:

What do you think of this rating system? I am curious to hear how others explain the seriousness of an incident to management.

To which my obvious response is, “why should management care about any of this when it doesn’t include impact”?  Even at it’s darkest colors, the scale is subject what I call the tree falling in the woods problem – does vulnerability or compromise matter independent of an impact statement?  Without impact, are we just “multiplying by zero”?

More directly, can you imagine going to a CFO with this but without impact information?  Wouldn’t you kind of look foolish if you said, when questioned about (probable) impact, “Well, then we would be modeling risk, and you know we can’t do that”?