Monthly Archive for July, 2009

Final Post on Mortman/Hutton and the Beginning of the End of the Beginning (Hopefully)

Published by alex on July 29, 2009 in Uncategorized. 2 Comments

The last post on the Mortman/Hutton model today is the most important.  You see, the primary idea (to me) behind the Mortman/Hutton model was never really to come to a strict or broadly accepted model for discussing what factors drive the creation and adoption of exploit code.  That was and is a vehicle for what is my (our) primary aim – the sharing of information to help further our understanding of IT risk management and what we might call (for lack of a better term) the science of information security.

You see, a model is only a hypothesis.  It is for testing, for falsifying, for evolving.  And ours is no different.  We welcome criticism and alternate theories.  Heck, even I have problems with it:  a couple of branches “don’t feel right” (the deductive logic isn’t as strong as I’d like), and the “measurement theory” behind the model as we use it is, to be nice about it, informal.  We do welcome improvements.

And to that extent this welcoming of changes to the model is our primary aim in developing and releasing the Mortman/Hutton model.  We want it to be the first model to be written about and stored here for others to help evolve.  It was birthed to help make this website a resource for those who seek knowledge about what it is that we do.

To that extent we will host our white paper here, and invite others to do the same for the theories/models they produce.  Hopefully in the future we’ll expand the sites capabilities, creating a sort of academic storehouse of information not just about ancillary information security science-like topics (like visualization), but of real discussion about the very epistemic nature of information security.

Epistemological Anarchy and Sensationalist Talk Titles

Which brings me to my second part of this blog post, answering a post Adam made that was discussing why he doesn’t like at least the title of David and my SecurityBSides talk – “Challenging the Epistemological Anarchist to Escape our Dark Age”.  To be fair, Adam hasn’t heard the talk or really discussed what our assertion against Epistemological Anarchy (E.A.) is.  Also, I’m not really all that against aspects of E. A..  But here’s the crux:

Epistemological Anarchy is a philosophical concept offered by Paul Feyerabend.  In short, Feyerabend suggests that there is no universal scientific method, or even that if there was one, we would be in jeopardy of tyranny to it rather than to the search for knowledge.  Thus he proposed anarchy, that as a reduction of Karl Popper’s falsification to the absurd, the only real universal truth to scientific discovery is “anything goes”.

And there is some appealing aspect to E.A., I’ll admit.  At best, it challenges us to not conform to modern theories about our fields of study (and if you’ve read anything by me, you know I love to challenge conventional wisdom about InfoSec – even to a fault).  However, at worst, it suggests that we give credence to even the most absurd irrational assertions (Feyerabend himself suggests that someday Rain Dances and Astrology might be “rediscovered” as having some aspect of truth in their claims).

To this extent, I find that many of our most notable “security rock star” types are readily dismissing our ability to apply any scientific method at all.  Donn Parker and Marcus Ranum immediately spring to mind as those who not only offer no real rational set course of action in order to build knowledge or increase wisdom, but rather suggest that our version of shamanism is just fine and all we can ever ascribe to.  To me, this is a premature abortion of our field, what I would call a newly forming social science.  In fact, take for example, what Thomas Kuhn suggests are stages of a natural science (you can look them up or follow my series over on the Verizon security blog).

I have no problem accepting that Kuhn would label us in what he calls the “protoscientific” stage.  A stage of science that is described by somewhat random fact gathering (mainly of readily accessible data), and a “morass” of interesting, trivial, irrelevant observations.  In the protoscientific stage there are a variety of theories (that are spawned from what he calls philosophical speculation) that provide little guidance to data gathering.(1)

Sound familiar?

Kuhn goes on to describe a second stage, where a theory comes to dominate all others and a “school” of “disciples” establish a discipline around the theory.  Not to re-hash the rest of what Kuhn says about how a science evolves, but I’ll offer this.  While I do want to push us out of Kuhn’s protoscience stage, I don’t see a predominant theory (or even despite the title of this blog – a “school” of disciples) forming just yet.  Rather, NewSchool seems to me, with all apologies to Churchill, just the beginning of the end of the beginning.   Our challenge is to use the positive aspects of Feyerabend’s E.A. to accelerate us past not only Kuhn’s “normal science” stage (where a predominant theory is held up and it spawns ancillary theories and models), but also past his “crisis” stage (where the predominant theory is falsified) into a stage of regular, repeating revolutions (and maybe we can use this crazy Internet technology to our advantage to do so).  That, to me at least (irony noted), is what is NewSchool.

But in doing so, we *have* to move beyond absolute dismissal of Information Security and Risk Management as a social science, beyond the Epistemological Anarchist who suggests that a quest for knowledge is futile.

–

(1) if you’ll indulge a little public naval gazing, I can see how one of my favorite modles, FAIR, could be interpreted to be theory spawned from philosophical speculation.  It is, after all, an almost purely deductive model that many have expressed difficulty resolving it’s approach to estimate-based measurement to their desire for precise results.  I will offer this, I found FAIR to fit many aspects of Kuhn’s requirements for Theory Choice (esp. the Fruitful aspect):

1.    – Accurate – empirically adequate with experimentation and observation
2.    – Consistent – internally consistent, but also externally consistent with other theories
3.    – Broad Scope – a theory’s consequences should extend beyond that which it was initially designed to explain
4.    – Simple – the simplest explanation, principally similar to Occam’s Razor
5.    – Fruitful – a theory should disclose new phenomena or new relationships among phenomena

A Black Hat Sneak Preview (Part 2 of ?)

Published by David Mortman on July 16, 2009 in Science of Risk Management. 8 Comments

Following up on my previous post, here’s Part 2, “The Factors that Drive Probable Use”. This is the meat of our model. Follow up posts will dig deeper into Parts 1 and 2. At Black Hat we’ll be applying this model to the vulnerabilities that are going to be released at the show.

But before we do our digging, two quick things about this model and post.  First, the model is a “knowledge” model (that’s a term Alex just made up, btw to describe the difference between Bayesian and Frequentist approaches to statistical analytics).  It actually measures state of knowledge rather than state of nature.  That means that this is not a probabilistic model that attempts to try to fill in a number count for some amount of population (how many people expect traditional statistical analysis to work), rather this model measures a degree of belief.  Second, the reason we did this was twofold – 1.) establishing accurate information around rate of adoption for what we want to study is difficult,  2.)  If we get to the point where we *can* measure rate of adoption in the wild, there’s a proof that a strong knowledge model  actually can never be worse than a frequentist model, and that sort of data will only increase the accuracy of the knowledge model results.

Finally, we’re using this model with “exploit code” in this post to describe how much we believe a specific exploit will be used by the aggregate threat community.  If you recall the gartner hype cycle graph, it is an attempt to identify and “measure” the factors that drive the curve of “adoption”.  It should be noted that this model might work well for not just describing a belief that exploit code will be well adopted, but also that a given weakness (what is commonly referred to as a vulnerability) will have exploit code developed for it.  The authors intend to play with this model for both purposes.

On To The Model:  The Mortman/Hutton Model for Expectation of Exploit Use

So here it is:

bh31

In this model, essentially, two significant factors begin to determine the fate of the exploit code. First, to understate the simplicity of the real world factors that drive exploit adoption, the exploit must be useful. And in order to be useful, the exploit would need to have valuable systems to penetrate, and people to use it. We break these two factors of usefulness down into two concepts: Saturation of Vulnerable Technology and Exploit Utility.

Saturation of Vulnerable Technology (SVT)

bh09-svt

If an exploit is going to be developed or used, there actually need to be systems to attack.  Saturation of Vulnerable Technology (SVT) reflects that requirement. SVT itself represents the aggregate amount of vulnerable systems, and is driven by two main factors. The number of vulnerable systems is determined by how significant the market penetration is for the technology (Windows = a lot, BeOS = a little), and the ability to compensate for the vulnerability. This ability to compensate is broken down into two sub-elements, the ability to repair (patch the system, for example) and/or the ability to apply compensating controls.

The outcome in SVT is “measured” in a number of probable systems without the ability to compensate. It is a combination of the number of systems with the initial vulnerability, and the probable % of the population that is currently vulnerable.

Exploit Utility

bh09-eu

If SVT attempts to measure the aggregate attack surface available to the exploit code, then Exploit Utility is an attempt to measure how useful the exploit code is to the aggregate threat community. Exploit Utility is dependent on two factors, if the systems that the exploit will be used on can be expected to return value to the threat community, and how pervasive the exploit code is in the threat community. We call these The Expected Value of Vulnerable Systems (EVoVS) and Probability of Code Dissemination, respectively.

The premise behind the Expected Value of Vulnerable Systems is simply an attempt to suggest that exploit code will only be used if the attacker can expect to gain something from the attack. This value can be expressed in the Information it Offers (in volume and expected market return), the Resources it Offers (in power/space/bandwidth), or the Access it Allows (notably, to other systems). The EVoVS can be measured as a range of value compared to the broader distribution of computing assets available as targets to the broader threat community.

The premise behind the Probability of Code Dissemination is simply an attempt to suggest that  the expected amount an exploit is to be used (in aggregate) is a function of how “available” it is. This could simply be stated as a proportion of exploit code inclusion in commonly available toolkit packages. An expected Rate of Exploit Adoption could be said to be dependent on two factors; the ease of use (skills, resources required to use it) and the nature of the exploit author (secretive, non-secretive).

In Conclusion

So there it is.  Developed at several different Columbus area coffee shops and lots of chicken wings and beer after work.  Now many of us have been privy (or hostage) to discussions around the what and why’s of “some sort of vulnerability/exploit was just announced by research team/vendor”.  If anything, hopefully this model will provide a rational basis for coming to a joint conclusion about why we should or shouldn’t care.  This model is released under a creative commons attribute-share alike-non-commercial license.  That means you have to tell people where you found it, if you mess with it – please give back to the community, and please don’t go charging anyone for the use of it.

An Example of Our Previous Graph In Action

Published by David Mortman on July 13, 2009 in Science of Risk Management. 0 Comments

I wanted to throw it out here as an example of how you would the model from my earlier post in real life. So let’s take the recently released Internet Explorer security vulnerability and see how it fits. Now this is a pretty brain-dead example and hardly requires a special tool, but I think it illustrates the use quite nicely. As a refresher, the black line indicates the general path of vulnerabilities for a particular technology (in this case it could be browsers, or IE specifically) and the red line indicates the deployment of that technology within an organization. I’ve added arrows to indicate approximately where on the curve this vulnerability fits and where, for just about every organization on the planet, the deployment of IE is.

tipping-point-v1-ms972890

Looking at the two curves and the placement of the arrows, we can see that, barring other issues, addressing this particular issue should be high priority. In our next post, you’ll see some other issues you may want to take into consideration in order to find the priority of this vulnerability relative to other vulnerabilities you may be facing.

Running from the truth

Published by adam on July 9, 2009 in Uncategorized. 1 Comment

Robin Hanson has an interesting article, “Desert Errors:”

His findings stayed secret until 1947, when he was allowed to publish his pioneering Physiology of Man in the Desert. It went almost entirely unnoticed. In the late 1960s, marathon runners were still advised not to drink during races and until 1977, runners in international competitions were banned from taking water in the first 11 kilometres and after that were allowed water only every 5 kilometres.

So not only were authorities dead wrong, but they were so confidently wrong that, in the name of helping runners, they paternalistically forced runners to do the exact worst thing! How could authorities be so wrong for so long on something that was so easy to personally test, and with such huge consequences? And how could they remain wrong for three decades after careful study had proved them wrong?

In information security, it’s harder to test these sorts of things. And that’s why we need openness. Password complexity rules are the equivalent of the no water rules. Actually, I have no evidence of that. No one does. We don’t know how often passwords are discovered by brute force testing (live), by breaking into the site and stealing either cleartext or hashes, or by phishing or malware. Only two of those five cases are impacted by complexity rules. If we had more data, we could move from edict to engineering.


To tie it back to Robin Hanson’s post, I’m confident that I’m wrong, and I don’t like it.

Business Week on Heartland

Published by David Mortman on July 9, 2009 in Uncategorized. 0 Comments

Not much to add, but a good article in Business Week on Lessons from the Data Breach at Heartland. Well worth reading…

A Black Hat Sneak Preview (Part 1 of ?)

Published by David Mortman on July 6, 2009 in Science of Risk Management. 6 Comments

Alex and I will be on a panel, A Black Hat Vulnerability Risk Assessment, at this year’s Black Hat. We’ll be discussing the need to perform a risk assessment of vulnerabilities as you become aware of them in a deeper context then just looking at the CVSS scores. Things to consider are what compensating controls may be in place, the value of the data being protected and how likely is it that an? attack will happen and how often it will be successful.

Alex and I have developed a model, The Mortman/Hutton Probabilistic Model of Exploit Use, that we hope will assist with this risk assessment. We’ll be posting portions of the white paper we wrote for Black Hat over the next couple of weeks, as well as updates to the model as it evolves in response to our own research and your feedback. We’d love to hear what you think.

Part 1, ” Background: On Technology Adoption” is below:

Tipping Point v2

Tipping Point v2

Using the pre-supposition that the classic Gartner Hype Cycle curve (above) represents a real model for rate of technology adoption and is applicable to the use of an exploit, the Mortman/Hutton model is an attempt to understand the factors that will “drive the curve”. So if we can say that the Gartner Hype Cycle represents the evolution of vulnerabilities with regards to a particular technology or a portion of that technology, then the first peak is what we see when researchers and/or miscreants first start examining that technology. This represents the low hanging fruit, that was either missed or deemed acceptable risk by the producers. After the initial burst of research, activity generally tapers off for a while based on a number of factors (that can be represented in part by the model). Once the research community gets more comfortable with the technology and has enough time to really start digging into its guts, more exploits start to appear and often they are more damaging or harder to fix. Additionally as you move along the curve, exploits start appearing in Metsploit and similar toolsets and eventually into tools that script kiddies and analysts can use.

If the black line represents activity around a vulnerability/exploit, then the red line represents the deployment of that same technology within organizations. Where the two lines overlap is when things start to get interesting, In the example above, the organization already has a heavy deployment of that technology when the second upswing of vulnerability research has started. As such the shaded area represents the tipping point where the CSO should seriously start worrying about the risk of vulnerabilities within there technology. Similarly, if the overlap is in a different place then a different rsik assessmment will result.

The Mortman/Hutton model attempts to help security professionals understand what will make exploit code “move” from 2 to 3 on the X axis. It is an examination of the factors that drive the probable creation and use of exploit tools.