It’s a tool! No one claimed it was a silver bullet!

Threat modeling is yet another input into an over all risk analysis. And you know what? Risk analysis/Risk management, whatever you want to call it won’t be perfect either. Threat modeling is in itself a model. All models are broken. We’ll get better at it.

But claiming that something is a failure because it’s not perfect and that it doesn’t always work, is one of the cardinal sins of infosec from my perspective. Every time we do that, we do ourselves and our industry a disservice. Stop letting the perfect be the enemy of the useful.

The past 10 years have been the best in the country’s aviation history with 153 fatalities. That’s two deaths for every 100 million passengers on commercial flights, according to an Associated Press analysis of government accident data.

The improvement is remarkable. Just a decade earlier, at the time the safest, passengers were 10 times as likely to die when flying on an American plane. The risk of death was even greater during the start of the jet age, with 1,696 people dying — 133 out of every 100 million passengers — from 1962 to 1971. The figures exclude acts of terrorism.

…
There are a number of reasons for the improvements.

• The industry has learned from the past. New planes and engines are designed with prior mistakes in mind. Investigations of accidents have led to changes in procedures to ensure the same missteps don’t occur again.
• Better sharing of information. New databases allow pilots, airlines, plane manufactures and regulators to track incidents and near misses. Computers pick up subtle trends. For instance, a particular runway might have a higher rate of aborted landings when there is fog. Regulators noticing this could improve lighting and add more time between landings.

(“It’s never been safer to fly; deaths at record low“, AP, link to Seattle PI version.)

Well, it seems there’s nothing for information security to learn here. Move along.

]]> http://newschoolsecurity.com/2012/01/aviation-safety/feed/ 0 http://newschoolsecurity.com/2011/12/discussing-norm-marks-grc-wishlist-for-2012/ http://newschoolsecurity.com/2011/12/discussing-norm-marks-grc-wishlist-for-2012/#comments Wed, 21 Dec 2011 23:52:35 +0000 alex http://newschoolsecurity.com/?p=2390 Norm Marks of the famous Marks On Governance blog has posted his 2012 wishlist.  His blog limits the characters you can leave in a reply, so I thought I’d post mine here.

1.  Norm Wishes for “A globally-accepted organizational governance code, encompassing both risk management and internal control”

Norm, if you mean encompassing both so that they are tightly coupled, I respectfully disagree.  Ethically, philosophically, these should be separate entities and ne’r the twain should meet.  Plus, accountants & auditors make poor actuaries.  See the SoA condemnation of RCSA.

Second, the problem with a globally accepted something is that it limits innovation.  We already have enough of this “we can’t do things right because we’ll have to justify doing things differently than the global priesthood says we have to” problem to deal with now.  Such documentation will only exacerbate the issue.

2.  Norm wishes for: “The convergence of the COSO ERM Framework and the global ISO 31000:2009 risk management standard.”

See #1, part 2 above.

3.  Norm wishes for:  “An update of the COSO Internal Control Framework that recognizes that internal controls are the organization’s response to uncertainty (i.e., risk), and you need the controls to ensure the likelihood and effects of uncertainty are within organizational tolerances.”

First, risk only equals uncertainty if you’re one of those stuck in the early 20th century Knightians.  For those that aren’t, and esp. actuaries and Bayesians alike, uncertainty is a factor in risk analysis – not the existence of risk.

Second, this wish seems to be beholden to the fundamental flaw of the Accounting Consultancy Industrial Complex – that Residual Risk = Inherent risk – Controls.  Let me ask you, what controls do you personally have against an asteroid slamming into your house?  But is that “high” risk?  Do you operate daily as if it’s “high” risk?  Why not?  Certainly you have weak controls, and most people would argue that their house and familys are of high value…

The reason it’s not “high risk” is because of frequency.  Yes, frequency matters in risk – and your RCSA process doesn’t (usually, formally) account for that.

4.) Norm wants “guidance that explains how you can set guidance on risk-taking that works not only for (a) the board and top management (who want to set overall limits), but also for (b) the people on the front lines who are the ones actually making decisions, accepting risks, and taking actions to manage the risks. The guidance also has to explain the need to measure and report on whether the actions taken on the front lines aggregate to levels within organizational tolerances.”

Great idea, but for this one to work, you’d have to establish guidance around reward-taking, tolerance, etc., too.

5.) Norm wants “A change to the opinion provided by the external auditors, from one focusing on compliance with GAAP to one focusing on whether the financial reports filed with the regulators provide a true and fair view of the results of operations, the condition of the organization, and the outlook for the future.”

I’m going with “bad idea” on this one.  Accountants != entrepreneurs.  Despite all their longing for control, power, and self-importance.

6.)  For Norm, Regulators should receive ” An opinion by management on the effectiveness of the enterprise-wide risk management program. This could be based on the assessment of the internal audit function”

I’m confused, how is the internal audit function in any way at all related to the quality of decision making?  Assurance is *an* evidence, a confidence value for specific risk factors.  It seems that Norm is saying that assurance is *the* evidence in total.

Frankly, very few accountants have training or exposure to probability theory, decision theory, or complexity theory.  Until they *do*, my wish for 2012 is that CPAs  reserve judgement on people trying to use real methods to solve real problems.

7.) Norm wants:  A change in attitude of investor groups, focusing on longer-term value instead of short-term results.

AGREED and +1 to you Norm!

In 10.) a, Norm desires that “audit engagements should be prioritized based on the risk to the organization and the value provided by an internal audit project.”

ABSOLUTELY NOT.  Unless Audit engagements are to be prioritized by the faulty idea of “Inherent Risk”.

Example, as a risk manager – I may have relatively stable frequency and magnitude of operational losses.  They may fall into a “low” tolerance range established by an ERMC or something.  But even though I am doing a good job (or really lucky) I may really be concerned about the process enough to warrant a high frequency of audit.  There are just so many concerns about this sort of approach by an auditor (from a risk/actuary standpoint) that I can’t disagree more.

In point 11 Norm’s wish is “An improved understanding by the board and top management of the value of internal audit as a provider of assurance relative to governance, risk management”

Me too, but I don’t think Norm and I agree on that “value.”

Again, for a mature risk management group, the value of assurance is simply the establishment of confidence values for certain inputs.  And frankly, if the board and top management understood that, I’m not sure Norm would really want them too, because many times the assurance is really a reinforcement of confidence/certainty, and frankly is a job that can easily be done with a risk model that reduces SME bias.

Finally, Norm “would like to see the term “GRC” disappear”

AMEN.  To use the ISACA/Audit terminology, Compliance is just “a risk.”  To use risk terminology, Compliance is a factor that contributes to secondary or indirect losses.

So, I’m with you – I’d like to see GRC taken out behind the shed.  Where I differ is that’s not because it becomes coupled with risk management, but rather because for me compliance aligns better with the authoritarian world of audit rather than a discipline like risk whose goal is to reduce subjectivity, or a discipline like governance whose role is to optimize resource expenditures.

First Mr. Lacey writes:

1. Risk assessment is objective and repeatable

It is neither. Assessments are made by human beings on incomplete information with varying degrees of knowledge, bias and opinion. Groupthink will distort attempts to even this out through group sessions. Attitudes, priorities and awareness of risks also change over time, as do the threats themselves. So be suspicious of any assessments that appear to be consistent, as this might mask a lack of effort, challenge or review.

Sounds reasonable, no?  Except it’s not alltogether true.  Yes, if you’re doing idiotic RCSA of Inherent – Control = Residual, it’s probably as such, but those assessments aren’t the totality of current state.

“Objective” is such a loaded word.  And if you use it with me, I’m going to wonder if you know what you’re talking about.  Objectivity / Subjectivity is a spectrum, not a binary, and so for him to say that risk assessment isn’t “objective” is an “of course!”  Just like there is no “secure” there is no “objective.”

But Lacey’s misunderstanding of the term aside, let’s address the real question: “Can we deal with the subjectivity in assessment?”  The answer is a resounding “yes” if your model formalizes the factors that create risk and logically represents how they combine to create something with units of measure.  And not only will the right model and methods handle the subjectivity to a degree that is acceptable, you can know that you’ve arrived at something usable when assessment results become “blindly repeatable.”  And yes, Virginia, there are risk analysis methods that create consistently repeatable results for information security.

2. Security controls should be determined by a risk assessment

Not quite. A consideration of risks helps, but all decisions should be based on the richest set of information available, not just on the output of a risk assessment, which is essentially a highly crude reduction of a complex situation to a handful of sentences and a few numbers plucked out of the air. Risk assessment is a decision support aid, not a decision making tool. It helps you to justify your recommendations.

So the key here is “richest set of information available” – if your risk analysis leaves out key or “rich” information, it’s pretty much crap.  Your model doesn’t fit, your hypothesis is false, start over.  If you think that this is a trivial matter for him to not understand, I’ll offer that this is kind of the foundation of modern science.  And mind you, this guy was supposedly a big deal with BS7799.  Really.

4. Risk assessment prevents you spending too much money on security

Not in practice. Aside from one or two areas in the military field where ridiculous amounts of money were spent on unnecessary high end solutions (and they always followed a risk assessment), I’ve never encountered an information system that had too much security. In fact the only area I’ve seen excessive spending on security is on the risk assessment itself. Good security professionals have a natural instinct on where to spend the money. Non-professionals lack the knowledge to conduct an effective risk assessment.

This “myth” basically made me physically ill.  This statement “I’ve never encountered an information system that had too much security” made me laugh so hard I keeled over and hurt my knee in the process by slamming it on the little wheel thing on my chair.

Obviously Mr. Lacey never worked for one of my previous employers that forced 7 or so (known) endpoint security applications on every Windows laptop.  Of course you can have too much !@#%ing security!  It happens all the !@#%ing time.  We overspend where frequency and impact ( <- hey, risk!) don’t justify the spend.  If I had a nickel for every time I saw this in practice, I’d be a 1%er.

But more to the point, this phrase (never too much security) makes several assumptions about security that are patently false.  But let me focus on this one:  This statement implies that threats are randomly motivated.  You see, if a threat has targeted motivation (like IP or $) then they don’t care about systems that offer no value in data or in privilege escalation.  Thus, you can spend too much on protecting assets that offer no or limited value to a threat agent.

5. Risk assessment encourages enterprises to implement security

No, it generally operates the other way around. Risk assessment means not having to do security. You just decide that the risk is low and acceptable. This enables organisations to ignore security risks and still pass a compliance audit. Smart companies (like investment banks) can exploit this phenomenon to operate outside prudent limits.

I honestly have no idea what he’s saying here.  Seriously, this makes no sense.  Let me explain.  Risk assessment outcomes are neutral states of knowledge.  They may feed a state of wisdom decision around budget, compliance, and acceptance (addressing or transferring, too) but this is a logically separate task.

If it’s a totally separate decision process to deal with the risk, and he cannot recognize this is a separate modeling construct, these statements should be highly alarming to the reader.  It screams “THIS MAN IS AUTHORIZED BY A MAJOR MEDIA OUTLET TO SPEAK AS AN SME ON RISK AND HE IS VERY, VERY CONFUSED!!!!”

Then there is that whole thing at the end where he calls companies that address this process illogically as “smart.”  Deviously clever, I’ll give you, but not smart.

6. We should aspire to build a “risk culture” across our enterprises

Whatever that means it sounds sinister to me. Any culture built on fear is an unhealthy one. Risks are part of the territory of everyday business. Managers should be encouraged to take risks within safe limits set by their management.

So by the time I got to this “myth” my mind was literally buzzing with anger.  But then Mr. Lacey tops us off with this beauty.  This statement is so contradictory to his past “myth” assertions, is so bizarrely out of line with his last statement in any sort of deductive sense, that one has to wonder if David Lacey isn’t actually an information security surrealist or post-modernist who rejects ration, logic, and formality outright for the sake of random, disconnected and downright silly approaches to risk and security management. Because that’s the only way this statement could possibly make sense.  And I’m not talking “pro” or “con” for risk culture here, I’m just talking about how his mind could possibly conceptually balance the concept that an “enterprise risk culture” sounds sinister vs. “Managers should be encouraged to take risks within safe limits set by their management” and even “I’ve never encountered an information system that had too much security.”

(Mind blown – throws up hands in the air, screams AAAAAAAAAAAAAAAAAHHhHHHHHHHHHHHHH at the top of his lungs and runs down the hall of work as if on fire)

See?  Surrealism is the only possible explanation.

Of course, if he was an information security surrealist, this might explain BS7799.

———————-

Risk != uncertainty (unless you’re a Knightian frequentist, and then you don’t believe in measurement anyway), though if you were to account for risk in an equation, the amount of uncertainty would be a factor.

risk != “likelihood” (to a statistician or probabilist anyway). Like uncertainty, likelihood has a specific meaning.

What is risk? It’s a hypothetical construct, something we synthesize in our brains to describe the danger inherent in the various inputs we’re processing around a certain situation. Depending on the situation, it can be either very difficult to create an “immaculate” equation for risk (such as R = T x V x I), or in many cases, impossible.

As an example, in IT, and especially for a large enterprise, we may have a complex adaptive system with characteristics of strong emergence. We see the same thing in medicine and various fields of biology. As such, point probabilities are pretty much impossible, or require so much simulation effort as to be difficult to produce.

Also, because it is a hypothetical construct we create in our brains, risk is also subject to the perspective of the observer. It is poly-centric. And because humans have a very difficult time divorcing their own risk tolerance derived from their own internal ad-hoc assessment from the information they have and the way they’re required to use that information, the true nature of the inherent danger, the majority of the “risk” is left unexpressed.

In my opinion, it’s important to dwell on these two pieces of information (risk may apply to a CAS with strong emergence, risk is subjective to the viewer) because they explain why the information security bureaucracies (ISACA, the ISO, NIST, most standards bodies, in fact) do us a huge disservice.

First, what our standards bodies do is typically do is enable us to justify our perspective by manipulating the inputs into a completely false model (jet engine x peanut butter = shiny!). This is the first significant way we give false (or at best, such poor information as to be incapable of creating a state of knowledge) information to decision makers.

Second, standards bodies, in the rush to provide value through “certification” have prematurely standardized processes to do “risk management.” This is the second way we are left giving false information to decision makers. Standardization without acknowledging the nature of risk (CAS, emergence, poly-centrism) results in the analyst ignoring critical pieces of the complex system that certainly contribute (sometimes significantly) to a full understanding of the situation.

Bottom line, IT risk is something created without being understood. It is the most important concept in information security, and the most abused. Until we have data, evidence of significant quality (see evidence-based practices) we cannot derive sane models, we cannot begin to understand the problem space.

As such, “risk” probably encompasses all of the above statements made in this thread, while in truth not resembling them at all (1).

————
1.) The thread was full of people explaining their “likelihood x impact” models. Variations on a theme, mainly.

Rob Graham called me out on something I retweeted here (seriously, who calls someone out on a retweet?  Who does that?):

http://erratasec.blogspot.com/2011/03/fukushima-too-soon-for-hindsight.html

And that’s cool, I’m a big boy, I can take it.  And Twitter doesn’t really give you a means to explain why you think that it’s too early to do a hindsight review of Fukushima, so I’ll use this forum.

Here’s the TL;DNR version: It’s too early to do hindsight or causal analysis on Fukushima – there is still a non-zero chance that something really bad could happen, we’re not at a point where the uncertainty in our information has stabilized, and any analysis done now would still be predictive about a future state.

But if you’re interested in the extended remix, there are several great reasons NOT to use Fukushima for a risk management case study just yet:

1. Um, the incident isn’t over. It’s closer to contained, sure, but it’s not inconceivable that there’s more that could go seriously wrong.  Risk is both frequency and impact, an incident involves both primary and secondary threat agents.  Expanding our thinking to include these concepts, it’s not difficult to understand thatwe’re a long way from being “done” with Fukushima.
2. Similarly, given the forthrightness of TEPCO, I’d bet we don’t know the whole story of what has happened, much less the current state. The information that has been revealed has so much uncertainty in it, it’s near incapable of being used in causal analysis (more on that, below).
   
3. The complexity of the situation requires real thought, not quick judgment.

Now Rob doesn’t claim to be an expert in risk analysis (and neither do I, I just know how horribly I’ve failed in the past).  So we can’t blame him. But let’s discuss two basic reasons why Rob completely poops the bed on this one, why the entire blog post is wrong:  Post-incident, our analytics aren’t nearly as predictive as pre-incident or during incident analytics. They can still be predictive (addressing the remaining uncertainty in whatever prior distributions we’re using), but they are generally much more accurate.

Second, what Rob doesn’t seem to understand is that post-incident risk management is kind of like causal analysis, but (hopefully) with science involved.  It’s a different animal.

Post-incident risk analysis involves a basic model fit review, identifying why we weren’t precise in those likelihood (1) estimations Rob talks about. It’s in this manner that Jaynes describes probability theory as the logic of science, as the hypothesis you make (your “prediction”) most be examined and the model adjusted post-experiment.  It’s basic scientific method.  I don’t blame Rob for getting these concepts mixed up. I see it as a function of what Dan Geer calls “premature standardization”: our industry truly believes that the sum of risk management is only what is told to them by ISACA, the ISO, OCTAVE, and NIST about IT risk (as if InfoSec were the peak of probability theory and risk management knowledge).  This is another reason to question the value of the CRISC, if in the (yet unreleased) curriculum there’s no focus on model selection, no model fit determination, or any model adjustment.

So if the idea of doing hindsight is invalid because what we’re dealing with now is a  different animal than what we would be doing post-incident,   what do we do?

First, if you really want, make another predictive model.  The incident isn’t over, we don’t have all the facts, but if you really wanted to you could create another (time-framed) predictive model adjusted for the new information we do have.

Second, we wait.  As we wait we collect more information, do more review of the current model. But we wait for the incident to be resolved, and by resolved I mean where it’s apparent that we have about as much information as we’ll be able to gather.  THEN you do hindsight.

At the point of hindsight you review the two basic aspects of your risk model for accuracy – frequency and impact.  Does the impact of what happened match expectations?  To what degree are you off?  Did you account for radioactive spinach?  Did you account for panicky North Americans and Europeans buying up all the iodine pills?  You get the picture. If, as in this case, there may be long term ramifications do we make a new impact model (I think so, or at least create a  wholly new hypothesis about long term impact)?

Once you’re comfortable with impact review and any further estimation, you tackle the real bear – your “frequency” determination.  Now depending on if you’re prone to either Bayesian probabilities or Frequentist counts, you’ll be approaching the subject differently but have key similarities.  The good news is that despite the differing approaches the basic gist is:  Identify the factors or determinants you missed or were horribly wrong about.  This is difficult because more times than not in InfoSec, that 3rd bullet up there, the complexity thing, has ramifications.  Namely:

There usually isn’t one cause, but a series of causes that create the state of failure. (link is Dr. Robert Cook’s wonderful .pdf on failure)

In penetration testing (and Errata should know this of all people), it’s not just the red/blue team identifying one weakness and exploiting it and then #winning.  Usually (and especially in enterprise networks) there are sets of issues that cause a cascade of failures that lead to the ultimate incident.  It’s not just SQLi, it’s SQLi, malware, obtain creds, find/access conf. data, exfiltrate, anti-forensics (if you’re so inclined).  And that’s not even discussing tactics like “low and slow”.  Think about it, in that very simple incident description there, we can identify a host of controls, processes and policies (and I’m not even bringing error into the conversation) that can and do fail – each causing the emergent properties that lead to the next failure.

This dependency trail is being fleshed out for Fukushima right now, but we don’t know the whole story.  We certainly didn’t count on diesel generators to resist a tsunami, but then there was the incompatibility of the first back ups to arrive, the fact that nobody realized that in a big earthquake/tsunami it would create a transportation nightmare and it would take a week to get new power to the station, and there were probably dozens of other minor causes that created the whole of the incident.  Without being at an end-state determination where we have a relatively final amount of uncertainty around the number and nature of all causes,  it would be absurdly premature to start any sort of hindsight bias exercise.

It’s too early to do hindsight or causal analysis on Fukushima – there is still a non-zero chance that something really bad could happen, we’re not at a point where the uncertainty in our information has stabilized, and any analysis done now would still be predictive about a future state.

Finally, this: “The best risk management articles wouldn’t be about what went wrong, but what went right.” is just silly.  The best risk management articles are lessons learned so that we are wiser, not some self-congratulatory optimism.

–

1.  (BTW gentlereader, the word “likelihood” means something very different to statisticians.  Just another point where we have really premature standardization)

Overview

Application Security Debt is based on the concept of  “technical debt” proposed by Ward Cunningham (a programmer who developed the first wiki program): describes it like this:

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.

Chris adds:

The cost of technical debt is the time and money it will take to rewrite the poor code after you ship and bring it back to the quality required to maintain the software over the long haul.

Here is Chris’ summary of Application Security Debt:

Security debt is similar to technical debt. Both debts are design and implementation constructions that have negative aspects that aggregate over time and the code must be re-worked to get out of debt. Security debt is based on the latent vulnerabilities within an application. Application interest rates are the real world factors outside of the control of the software development team that lead to vulnerabilities having real cost. These factors include the cost of a security breach and attacker motivation to discover and exploit the latent vulnerabilities.

Chris’ second post describes a financial model that estimates the cost of Application Security Debt.  Framing the metric in financial terms will presumably help managers compare the cost of the “debt” to the cost of developing more secure software or costs of fixing the vulnerabilities.  (Note: Veracode provides a range of application security testing services, so they have an interest in economically justifying their services.  This isn’t a criticism of Veracode, Chris, or his proposal.  Just a reality.)

Chris’ model is focused on the simplest case where the application developer and application user is the same organization, so that it bears the costs of development, maintenance, and also any security breaches that result.  Starting with the simplest case is a great idea when proposing a new method.  So far so good.

Chris defines his financial model this way:

The basic financial model for security debt is monetary risk that can be expressed as expected loss. The formula for expected loss is event likelihood X impact in dollars. Event likelihood is based on the makeup of vulnerabilities in the application and the likelihood that the vulnerabilities will be discovered and exploited. The impact is the cost of a security breach based on an exploit of one of those vulnerabilities.  [Emphasis in original]

This is, of course, a version of the bottom-up Annualized Loss Expectancy (ALE) formula for individual risk elements:

• ALE = Single Loss Expectancy X Annual Rate of Occurrence

(Mike Rothman recently crapped on all “risk metrics” by lumping them all into the ALE formula.  I’ll critique ALE and Mike’s post in a separate blog post.)

ALE issues aside, I think Chris is making mistakes in his definition of Application Security Debt that will lead to serious confusion.

Debt = Expected Principal + Interest Costs

Chris made a mistake when he defines monetary value of the Application Security Debt as expected loss due to security breaches.    Instead, the ‘Principal’ part of the debt formula is the cost of fixing security problems beyond what is budgeted. Chris had it right in his summary in the first article:

The cost of technical debt is the time and money it will take to rewrite the poor code after you ship and bring it back to the quality required to maintain the software over the long haul.

Expected losses are in the category of “Interest Costs” as Chris said in his summary:

Application interest rates are the real world factors outside of the control of the software development team that lead to vulnerabilities having real cost.

Putting this together in simple language:

“Application Security Debt is a ‘loan’ with variable principal which could range from 0% to 100% of your original project costs. The ‘principal’ is what you’ll eventually have to pay to fix security bugs or rewrite the code.  It also has varying and uncertain ‘interest costs’, which are the costs of security breaches due to these vulnerabilities. This includes the possibility of the mother-of-all balloon payments (i.e. a huge loss event).”

The good news is that Expected Principal is relatively easy to estimate with good accuracy and without a lot of outside data.  The not-so-good-news is that Interest Cost is a bear to estimate.

Estimating ‘Expected Principal’

For simplicity, let’s assume that cost of fixing code (above the budgeted costs) occurs in discrete increments, F:

1. Zero  (i.e. your debt is ‘forgiven’)
2. Minor fixes and patches (‘Principal’ = 10% increase in project cost)
3. Major fixes and patches  (‘Principal’ = 25% increase in project cost)
4. Substantial rewrite (‘Principal’ = 50% increase in project cost)
5. Total rewrite   (‘Principal’ = 100% increase in project cost, or more)

Thus, the best case is that you owe no principal and the worst case is that you owe principal equal to the entire cost of the project.  You could include other factors such as external costs of schedule delays, costs of rehiring your programmers after you fire them all , or what ever.  My point is that these costs are not open-ended, but are a multiplier on your original development costs.

The Expected Principal (EP) is equal to each of these cost scenarios multiplied by their probability of management choosing that option:

For example, if the original cost of the application development project is $1 million, and there is 5% chance of Zero costs, 80% of Minor code fix costs, and 15% chance of Substantial rewrite costs, then the Expected Principal would be $155,000, or 16% of the original cost.

This is important: Expected Principal is ultimately determined by management decisions and ‘threshold of pain’.  This means that the value of p(F), above, is a subjective probability.  It would be an ideal metric to estimate using prediction markets (PMs).   (PMs have been used successfully in software development to estimate shipment dates and defect rates, for example.)

Another implication: you don’t need to accurately forecast future loss events or their economic impact to get a decent estimate of Expected Principal.  Instead, you only need to estimate the Interest Costs very roughly to determine which code fix scenario is most likely.    You could even estimate p(F) by setting thresholds for the number and severity of vulnerabilities discovered by certain levels of effort.  Better, you could combine these methods to ‘triangulate’ on estimates of p(F).

To calibrate these subjective probability estimates, it would be very helpful to collect historical data on the % of applications that have some level of rewrite or schedule delay due to security problems.  (Hint hint!)

Estimating ‘Interest Costs’ on the Debt will be Hard

The second part of the Application Security Debt formula is ‘Interest Costs’.  This is where things get hairy.   All the members of the ALE family of risk calculations have a similar flaws: 1) prodigious data requirements and 2) propagation of uncertainty through the calculations.  Furthermore, some suffer by using only mean values and ignoring extreme values (i.e. the “tails” of the probability distribution curves).

Chris acknowledges these issues, at least the requirement for more and better data:

Now you are probably thinking that this is getting a little tenuous and it is. We need better data on likelihood type and likelihood of an application breach by industry and other factors like company size.

Data issues aside, I think there are flaws in his use of ALE and calculation methods.  Here’s one thought experiment to show how it could lead to the wrong conclusions, in my opinion.

Let’s use Chris’ ‘baseline expected loss’ table, where he calculates the expected loss for each type of vulnerability.  Imagine that we are comparing two similar applications, A and B.  Assume that each project is expected to have the same number of vulnerabilities, five each.  Let’s say the development cost of each project is $1 million.  Application A has five SQL injection vulnerabilities while application B has one SQL Injection vulnerability and four Remote File Inclusion vulnerabilities.  Doing the calculations:

• A’s expected losses = $19,220,000
• B’s expected losses = $5,074,080

Does project A really have four times more risk than project B? Probably not.  From what I know, the number of vulnerabilities in an application is not proportional to the likelihood that the application will be breached.  Instead, I’d guess that the likelihood of being breached is a function of where the application is in the IT architecture, how accessible it is, how important it is to attackers, etc.

Also, there’s the ‘weakest link’ effect: “given enough random attackers or one persistent attacker, it only takes one vulnerability to lead to a breach”.  Assuming all SQL Injection vulnerabilities are equally discoverable and equally exploitable, then we should estimate that application B with one SQL Injection vulnerability is just as likely to get breached as application A with five, all other things being equal.

(I confess I’m not an expert in application security or vulnerability analysis, so these comments are my interpretation of what others have written or said.)

Even if my logic here is flawed somewhat, my main point is that the relation between number of vulnerabilities and likelihood of being breached is non-linear and it may even be indeterminate if contextual factors dominate.

This example also hints at another severe weakness in the ALE method – it ignores correlation and dependence between risk elements and factors.  We know from forensic analysis and the DBIR that severe security breaches involve a sequence of exploits and attacks.  This means that the likelihood of breach in one application is dependent on the likelihood of breach in other applications and systems.  An application might appear unimportant, but it might be a stepping-stone to other applications, databases, and networks.

It’s hard to account for all these factors and influences together without some sort of over-arching model for enterprise-level information security and risk.   Basically, you are looking for the ‘risk contribution’ of those specific application vulnerabilities to total costs, now and in the uncertain future.    Formally, the ‘Interest Cost’ for any given set of application vulnerabilities is the difference between the Total Cost of Security (TCoS) in two possible worlds: World 1) application A has X vulnerabilities, vs. World 2) application A does not have X vulnerabilities (or if application A is not deployed at all).

What we really need are some short-cut approximations for this that doesn’t require a complete data set and risk estimates for the whole enterprise.  One approach I’m interested is in using modern AI methods (data mining, machine learning, inference methods).  This is on-going research.

Summary

I’m glad Chris proposed his Application Security Debt metric.  I hope my post has been helpful in correcting some of the errors, as I see them.  The good news is that the “Expected Principal” component of the metric looks like it can be estimated fairly easily and with good accuracy.  On the other hand, the “Interest Cost” component needs a lot of work.  I’m happy to collaborate with Chris or anyone else who wants to work on this.

Just because you can codify a standard or practice doesn’t mean that this practice is sane. There’s plenty of documentation around homeopathy, astrology, biorhythms, and other pseudosciences, but that doesn’t make them any more real.

In other words, just being able to reference a document for repeatability does not make the outcome of those acts real or valid. Almost everyone in that thread has focused on our industry’s ability to create documentation, not on the fundamental problems of creating a defensible method for risk expression.

This is why our standards blow.  And yes, I’m going to expand my focus beyond CRISC/Risk IT and include the 800 series from NIST (including the new releases), the ISO 27005/31000 document, and many others.  They are all very heavy on repeating the same idea that risk management is some OODA/PDCA type cycle and subsequent bureaucratic processes and very thin on the actual establishment of useful risk statements. Look, your P/D/C/A policy/procedures only need to be a few pages, and you certainly don’t need the time, expense, and hassle of certification.  Spending the time and effort to tailor a several hundred page document and get people all certifiable on the subject to fit your organizational culture is just a rabbit trail of waste.

I mean, as weird as OSSTMM is – at least Pete has done a really good job of trying to provide metrics and derivative values of meaning that are repeatable.

“trust is for suckers.”

But security is keen on having trendy new memes, things to sell you, and I thought that I might as well weigh in because we’re seeing “trust models” in everything from OSSTMM to NIST 800 series to Cloud model thingies.

The problem I have getting all excited about “trust modeling” is that it’s basically yet.another.hypothetical.construct.  We already have “security” and “risk” that our industry finds problematic to define and measure.  Why are we soooo keen on creating another problem child?

And I have to wonder, is it really necessary?  Think about it: my ability to “trust” – a person, connection, system, whatever – is simply an acknowledgement of the risk in transacting with them.  I “trust” my good friends and certain relatives because I have a bunch of experience (priors) that lead me to believe that they are “trustworthy”, which is a nice way of saying “low risk”.  I know these people will go out of their way *not* to hurt me without just cause.

Similarly, when dealing with car mechanics, salespeople, and politicians – I have very little “trust” because there is a high degree of risk in my transaction and I either have no prior experience with them, or really bad experiences with them.

So really, my amount of “trust” is the inverse of the amount of risk I perceive for whatever reason (past experience, lack of data, expected experiences given some trending data).

THE ONLY TRUST MODELS YOU’LL EVER NEED

So in 2011, if you’re sitting in a meeting and some GRC pusher decides that they need trust models for you to be really good at your jobs, you can use the following and explain to them you already have a very nice one, thanks.

IF YOU USE QUALITATIVE RISK STATEMENTS

Trust = Opposite of Risk

So “Low Risk” becomes “High Trust”.

IF YOU USE RISK SCORING WITHOUT MEASUREMENT SCALES

Trust = 1/Risk

So The larger the risk score, the smaller the trust score.

IF YOU USE QUANTITATIVE ALE – TYPE MEASURES (ALE, FAIR, WHATEVER)

You don’t need to do anything.  You can say “We trust that this partner will have 10 incidents per year causing us between $50,000 and $2,000,000 in damage” or whatever.

There you go.  A Trust Program/Trust model for absolutely free.

So I asked Twitter “What’s the best way to track security updates for wordpress + plugins? I don’t want to have to look at the dashboards daily.” Zot O’Helpful responded “Wait unil your site is hacked, then update.” Mark Adams commented that “I discussed WP recently with @markstanislav. We concluded that vulns are most likely to be in plugins, not the core.” Which is fine as far as it goes, but the vulns are more likely to be discovered in the core, and more likely to be widely exploited there.

But the question remains: how do others keep up with WordPress admin duties?

For bonus points, don’t discuss why doesn’t WordPress have a security announcements blog, twitter stream, mail list or anything else?

[Update: @chrisjager pointed to feed://wordpress.org/news/category/releases/feed/, which is a good start.]