Monthly Archive for October, 2009

Jeremiah Grossman has an article in SC Magazine, “Businesses must realize that full disclosure is dead.” On Twitter, I asked for evidence, and Jerimiah responded “Evidence of what exactly?”

I think the key assertion that I take issue with is bolded in the context below:

Unquestionably, zero-day vulnerabilities have an increasing real-world value to many different parties. We should expect more and more researchers to demand and receive payment from governments, software vendors, security vendors, enterprises or someone on the black market. It has already happened and will continue. The evolution is underway and it will become more prevalent in the next few years as it becomes routine for our systems to be compromised using unknown vulnerabilities. This environment will force us to evolve our thinking and mature our offensive and defensive security strategies – fueling the need for third-party patches, subscriptions to unreleased vulnerability information, and general underground industry intelligence. We’re already seeing these services being offered on the fringe (legally and illegally) and slowly moving towards mainstream acceptance as the business models are better understood. So it’s not a matter of if, but when.

We will need to evolve, yes, but I don’t see that the direction suggested is the one we’ll need to take. In particular, operationally using 0day is a tricky business. You risk discovery and losing a valuable asset by exposing it to a target. So maybe you use something a bit more commonplace. As I recall the Verizon Breach Report, they say that roughly 75% of vulns exploited have been public for a year or more. Yes, there’s a rapidly growing volume of underground stuff, but that’s easy when such things are a tiny fraction of attacks, vulnerabilities, or root causes of bad outcomes.

So I’m curious where is the evidence that undisclosed vulns will come to dominate? Oh, and a second question. Jerimiah, your title seems to imply that this is the most important thing for businesses to realize. Is that really what you meant?

My employer spends a lot of energy on building things to make exploiting unknown vulns harder, but if I wanted to speak for them, I’d do so on my work blog.

[Ooops! Mis-spelt Jeremiah's name. Sorry!]

It’s the probabilistic decision making tool for baseball managers.  On the iPhone.  It’s like a business intelligence application in the palm of your hand

Basically, it takes the probabilistic models of either Win Expectancy or Run Expectancy (any given action has some probability of contributing a run or a win) and given a situation, attempts at offering whether it’s a good idea or bad idea to execute that plan.

Here we see a situation where the manager is wondering if it’s a good idea to attempt a double steal.  An obvious dependency is knowing the stolen base success rate for the runner on second (it also assumes that the catcher will only attempt to throw at the lead runner, a pretty safe assumption).  If we’re baseball freaks, we might also note that there’s not contra-factor around the probability of a pick off move, I don’t see how the catcher’s rate of successful pick offs is factored in, etc. – but we’re nitpicking….

Once the decision to execute is established (press the button! press the button!) then we receive a screen that tells us whether it’s a good or bad idea, respective of how much our win (or run) expectancy increases or decreases.

Now obviously, this is more of an “armchair quarterback” (to mix sports in metaphors) sort of toy, but it got me to thinking that this would be pretty fun for us to have something like this for risk or threat based modeling.  Rather than a baseball diamond, we might conceive of a set of connected IT objects/assets (a business process, maybe), each with their own “expectancy” to do some combination of Prevent/Detect/Respond to various threat sources.  Do I want to add more Prevent?  Bad idea!  Your risk is reduced at an insignificant level compared to the investment required to achieve that new level of prevention.  Do I want to add more training?  Good idea!  Training analysts in “detection” increases the risk reducing probability for this group of assets in an economically efficient manner.  Obviously, this is all probabilistic, but all decision making is, right?  I mean, this is why I’m NewSchool, I hope that someday we’ll reach this level of sophistication as an industry.

Anton Chuvakin’s been going old school.  Raising the specter of “risk-less” security via best practices and haunting me like the ghost of blog posts past.   Now my position around best practices in the past has been that they are, to use Jack Jones’ phrase, Infosec “shamansim”.  We do these things because our forefathers do them, because the tribe up the river does them, and/or because if we don’t the thunder gods (hackers) will get us, not because we have any formal evidence that they are “best”.    Now that said, these suggested practices I keep seeing are not completely uninformative, after all there’s a reason people suggest the things they suggest.  But being Friday and all, maybe we could talk a bit about why we don’t really have best practices, why Anton’s concept of risk-less security is a fallacy,  and what we call best practices?  Yeah, I think we could even argue that they’re unethical.

DO WE EVEN HAVE “BEST” PRACTICES?

First, why don’t we head over to that all-authoritative source, Wikipedia, for a definition of Best Practice.

A Best practice is a technique, method, process, activity, incentive or reward that is believed to be more effective at delivering a particular outcome than any other technique, method, process, etc. The idea is that with proper processes, checks, and testing, a desired outcome can be delivered with fewer problems and unforeseen complications. Best practices can also be defined as the most efficient (least amount of effort) and effective (best results) way of accomplishing a task, based on repeatable procedures that have proven themselves over time for large numbers of people.

Despite the need to improve on processes as times change and things evolve, best-practice is considered by some as a business buzzword used to describe the process of developing and following a standard way of doing things that multiple organizations can use for management, policy, and especially software systems.

As the term has become more popular, some organizations have begun using the term “best practices” to refer to what are in fact merely ‘rules’, causing a linguistic drift in which a new term such as “good ideas” is needed to refer to what would previously have been called “best practices.”

Now whether or not you agree that Wikipedia is useful, I think that passage serves our discussion today.  I have long said that if we were to use the word “best”, that would require us to know “not best”.  So considering:

Best practices can also be defined as the most efficient (least amount of effort) and effective (best results) way of accomplishing a task, based on repeatable procedures that have proven themselves over time for large numbers of people.

Maybe you could tell me about “efficient” and “effectiveness” – and how you would compare a “best practice” to an “almost best practice” without hand waving and resorting to a “just because I said so” argument.  I don’t think you could, our (the infosec industry)  measurement capabilities aren’t there yet, and so though while we have various reports and studies that suggest certain things are more effective than others, and we’re making great progress, Exhibit A today is:

Exhibit A – We don’t really have “best” practices – yet.

I could see saying “recommended” practices, “suggested” practices, or even based on some studies out there “essential” practices, sure.  But we don’t have the sophistication of “best/not best” and I would argue that we, as an industry, are guilty of

using the term “best practices” to refer to what are in fact merely ‘rules’, causing a linguistic drift in which a new term such as “good ideas” is needed to refer to what would previously have been called “best practices.”

CAN YOU HAVE RISK-LESS SECURITY?

I’m not sure of what desire is driving Anton towards risklessness, but it’s folly.  Here’s why:  as soon as you consider a control, you’re considering risk. Impact comes easily and instinctively, likelihood in a similarly quick and instantaneous fashion.  Otherwise, to use Peter Tippett’s examples, we’d all have titanium seatbelts and falling asteroid protectors on our cars.  Now the hitch is, we (human brains) are notoriously bad at likelihood without the methods of models and math.  But my point still stands – when we think security (and especially we, the people paid to think security) consider an asset we want to protect, we’re already doing a risk assessment, albeit in manner that just isn’t as structured as one that might use a formal model.

Exhibit B – So really, we’re *all* doing risk, and when we have these discussions of “use risk or don’t use risk” – all we’re really arguing about is how much cognitive bias we’re allowing into our assessment.

If you want to be on the side of the debate that suggests that lots of cognitive bias is a good thing, so be it.  There’s no rational way I can argue with you  – have a nice day, thanks and come again.  But if the role and purpose of risk modeling is to remove bias and provide a state of knowledge – then I have to ask you,

IS IT ETHICAL TO USE A RISK-LESS APPROACH?

Strong charge, I know.  And certainly an Incomplete thought that needs more discussion, but bear with me while I reason this out for us.  Take this quote from Donn Parker against a risk-based approach:

risk reduction justification makes it too easy to accept security vulnerabilities in exchange for other benefits.”

The problem with this statement is that the economic decision of butter vs. guns *is not ours to make*.  In all the CISSP study guides sitting on my library shelves, the premise is offered that the asset does not belong to us, security, but rather to the data owner.  So with all due respect, that’s not your call to make!  Not mine, not Donn’s, not yours.  It is the responsibility of the business owner.  Our responsibility is to do the best job we can informing them of the (wait for it) consequences of accepting (or not-accepting) a security vulnerability.

So if our job is to inform, not decide – and if risk analysis is the best way we have of reducing cognitive bias – then is resorting to a “do it because I say so” “best” practice approach ethical?  Because what you’re essentially doing is purposefully withholding information from the data owner in order to force them to accept your risk tolerance.  A “best” practice approach, without reference-able data or analysis to back it up is not just an act of hubris, it is deceit.

Exhibit C – If our job is to help the data owner make informed decisions, calling what we do in our industry a “best” practice is not entirely truthful, and not using a risk model that removes bias and expresses uncertainty is not doing the best job you can.

CAVEATS GALORE

1.)  There’s nothing wrong with real best practices.  In a sense, that’s what the New School is all about, inspiring us to arrive at things like (reasonably) measurable efficiency and effectiveness.  After all, a best practice could be said to be the current standing, well-tested hypothesis or theory.

2.)  There are plenty of times when we don’t have to go and do a formal risk assessment.  My goal here is not “spreadsheets for the masses”, it’s rational expressions of risk management and helping people make the best decision they can with the information available at the time of the decision.

3.)  Admittedly, 95% of the “likelihood” determinations out there in our industry, in the words of my 11 year old, make me want to throw a sidewalk pizza.  And I could see a response post where you might claim that bad likelihood models are just as unethical, and that there’s so much uncertainty in our estimates, we are many times doing a disservice.  Again, I’m not saying we’re in a perfect world yet, I’m suggesting that a good risk model removes bias and expresses uncertainty.

Similarly, the enemy here is not subjectivity – everything we do is subjective and/or relative.  The enemy here is not removing all the subjectivity you can and not expressing uncertainty in the results.  Our industry’s “best practices” don’t do that yet.  Let me encourage you to express the limitations in everything – it is the pathway of critical thinking we must travel.

4.)  I’m not accusing anyone, nor do I think anyone is really being purposefully unethical or deceitful.  I can say that Anton and Donn are really good, smart, and upstanding people in as much as I’ve had the pleasure of knowing them.  I’m just trying to caution us all about the consequences of what we do and how we approach our profession.