The New School of Information Security » Reports and Data

Time for an Award for Best Data?

adam — Wed, 01 Feb 2012 17:15:54 +0000

Yesterday, DAn Kaminsky said “There should be a yearly award for Best Security Data, for the best collection and disbursement of hard data and cogent analysis in infosec.” I think it’s a fascinating idea, but think that a yearly award may be premature. However, what I think is sorta irrelevant, absent data. So I’m looking for data on the question, do we have enough good data to issue an award yearly?

Please nominate in the comments.

Also, please discuss what the criteria should be.

Kudos to Ponemon

adam — Mon, 23 Jan 2012 15:59:03 +0000

In the past, we have has some decidedly critical words for the Ponemon Institute reports, such as “A critique of Ponemon Institute methodology for “churn”” or “Another critique of Ponemon’s method for estimating ‘cost of data breach’“. And to be honest, I’d become sufficiently frustrated that I’d focused my time on other things.

So I’d like to now draw attention to a post by Patrick Florer, “Some Thoughts about PERT and other distributions“, in which he says:

What follows are the results of an attempt to answer this question using a small data set extracted from a Ponemon Institute report called “Compliance Cost Associated with the Storage of Unstructured Information”, sponsored by Novell and published in May, 2011. I selected this report because, starting on page 14, all of the raw data are presented in tabular format. As an aside, this is the first report I have come across that publishes the raw data – please take note, Verizon, if you are reading this!

So I simply wanted to offer kudos to the Ponemon Institute for doing this.

I haven’t yet had a chance to dig into the report, but felt that given our past critiques I should take note of a very positive step.

Paper: “The Future of Work is Play”

adam — Thu, 01 Dec 2011 17:42:55 +0000

My colleague Ross Smith has just presented an important new paper, “The Future of Work is Play” at the IEEE International Games Innovation Conference. There’s a couple of very useful lessons in this paper. One is the title, and the mega-trends driving games into the workplace. Another is Ross’s lessons of when games work:

Over the last several years, Microsoft has employed dozens of games and game mechanics in its software development process. Forrester, Forbes and others have covered this work. Table 1 illustrates the areas where productivity games can be the most impactful. Focusing on either expanding skills in rile or “organizational citizenship behaviors” that require core skills &emdash; is the best way to ensure the success of a productivity game. Player motivations is a key component of the success of a productivity game.

Core Unique expanding skills

In role behavior Most Impact

Organizational Citizenship Behavior Most Impact

What this means is that if you try to produce a game that replicates or intrudes on either core work (say, writing code) or unique skills that someone already has (say, threat modeling) the game is likely to be less successful. But if you make a game to help people expand their skill (say, in threat modeling), it will be more impactful and accepted. Similarly, if you’re trying to get thousands of people to help check user interface translations for Windows, it helps to use a core skill, like reading another language, rather than a unique skill (again, let’s say threat modeling) that only a few people have.

This table is really useful guidance if you’re thinking of making a game.

Games, by the way, are tremendously New School. Games are New School because they’re a way to address the real human desires to do something (anything) more fun than deal with security stuff. By making it fun, we can entice people into enjoying the things we need them to do. You should consider if a game can address a problem you deal with, and if it’s in the area of expanding skills in a role or organizational citizenship behaviors that rely on core skills, you’re more likely to succeed.

(I’d link to the paper, but unfortunately, IEEE continues to lock up the scientific literature and impede the flow of progress, rather than charge a few dollars more for each conference to cover the costs of serving up the scientific literature.)

Big Brother Watch report on breaches

adam — Wed, 30 Nov 2011 16:09:06 +0000

Over at the Office of Inadequate Security, Dissent says everything you need to know about a new report from the UK’s Big Brother Watch:

Extrapolating from what we have seen in this country, what the ICO learns about is clearly only the tip of the iceberg there. I view the numbers in the BBW report as a significant underestimate of the number of breaches that actually occurred because not only are we not hearing from 9% of entities, but many authorities that did report probably did not detect or learn of all of the breaches they actually experienced. BBC notes, “For example, it does seem surprising that in 263 local authorities, not even a single mobile phone or memory stick was lost.” “Surprising” is a very diplomatic word. (“What They Didn’t Know: Big Brother Watch report on breaches highlights why we need mandatory disclosure“)

More on Authorization Persistence Threats

adam — Fri, 18 Nov 2011 16:16:03 +0000

Wade Baker has a quick response to my “Thoughts on the 2011 DBIR and APT,” including the data that I was unable to extract.

Thanks!

Block Social Media, Get Pwned

adam — Thu, 17 Nov 2011 16:15:58 +0000

At least, that’s the conclusion of a study from Telus and Rotman. (You might need this link instead)

A report in IT security issued jointly by Telus and the Rotman School of Management surveyed 649 firms and found companies that ban employees from using social media suffer 30 percent more computer security breaches than ones that allow free use of sites like Facebook and Twitter.

Counterintuitive? Maybe, but it makes perfect sense when you consider how hooked most of us are on social media, say the study’s authors.

Rotman professor Dr. Walid Hejazi says employees banned from social networks often download software onto company computers allowing them to circumvent firewalls and access forbidden sites. Those programs let employees to tweet on the job but also create security gaps hackers are happy to exploit. (“Being hacked? Your social media policy might be to blame“, Morgan Campbell, The Star)

A quick skim indicates that this study is based on a survey of Canadian companies which received 649 responses. Parts of the study are worrisome. (For example, their classification of breaches types shows 46% had “Virus/Worms/Spyware” but only 9% had “bots,” and 20% had “phishing/pharming” while only 5% had “social engineering attacks”) However, it seems plausible that organizations know that they’re hacked, and that organizations know if they have a social media policy, so the conclusion of a correlation or even causation may be reasonable. At the same time, it may be that there’s a causative effect of security conscious organizations having both better intrusion detection activity and social media policies, or organizations that are more likely to be hacked having more social media policies. I’m going to tentatively discount those hypotheses because the Verizon DBIR tells us that most organizations don’t detect their own hacks.

I also wanted to comment that a great many companies publicise their social media policies, and it’s probably possible to re-do this study with DatalossDB data.

I haven’t read the study in any detail (really!) but since it confirms my biases I decided to blog it early. Those biases include thinking that Angela Sasse’s “personal compliance budget” idea has a lot of explanatory power. Thanks to Bob Blakely for the pointer.

Thoughts on the 2011 DBIR and APT (Authorization Preservation Threats)

adam — Mon, 07 Nov 2011 15:35:15 +0000

So Verizon has recently released their 2011 DBIR. Or perhaps more accurately, I’ve managed to pop enough documents off my stack that my scribbled-on notes are at the top, and I wanted to share some with you. A lot have gone to the authors, in the spirit of questions only they can answer.

Here, I want to talk about one of two particularly New School takeaways from the report, which is figure 18:

Now, there’s two datapoints on there that thus jumped out at me as important. They’re “BRUTE” and “DFCRED”. What do BRUTE and DFCRED mean? BRUTE is “Brute force and dictionary attacks” and DFCRED is “Exploitation of default or guessable credentials.” Now, I’d guess that the difference between “guessable” and “dictionary” is the size of the dictionary needed, but I don’t want to quibble over taxonomies here. What I want you to focus on is how BRUTE was involved in 25% of breaches and on the order of 30% of records. DFCRED is involved in 35% of breaches and a bit under 30% of records.

Take a moment to think about that, and what you might do about it.

I’m not going to claim that changing passwords is free, or that password management is trivial. However, it seems that change your passwords is all it would take to substantially reduce the success of perhaps 30% or more of the breaches that Verizon studies. I’m being squishy because perhaps the attackers would have found another way, or perhaps DFCRED and BRUTE hit different customers, in which case it could be over 50% of attacks thwarted. I don’t want to attack anyone’s business here, but if you’re looking at any super-fancy technology before you’ve rolled out AD password policies and also mastered changing your passwords on the non-AD stuff, you’re ignoring the Authorization Preservation Threat.

And the data we have from the DBIR shows that Authorization Preservation Threats are common and impactful enough to trigger a Verizon investigation.

I’d love to know the numbers for the unions of those BRUTE and DFCRED, but I don’t think they can be derived from the DBIR as published. We’d need a table of breaches and their threat action types. [Update: Wade Baker has kindly published that on their blog!]

To be fair, Verizon’s analysts did understand this this-access control is one of their top recommendations, 30 pages later. I don’t think they stress it enough given the relative ease of implementation, nor do they tie it back sufficiently for my taste.

That’s ok-they’ve published data and methodology in sufficient detail for me to bring up the point and stress it for you. Because they do that, they allow others to interpret and build on their work. Different people have different perspectives that come from where and how they were raised, where they went to school, what jobs they have, what media they pay attention to. All of which combines into different ways of filtering and sorting through facts. Being able to bring those perspectives to bear on the same data helps us get more out of it than any single analyst, however smart. And so Verizon sharing their data is a big win.

Diginotar Quantitative Analysis (“Black Tulip”)

adam — Tue, 13 Sep 2011 15:12:05 +0000

Following the Diginotar breach, FOX-IT has released analysis and a nifty video showing OCSP requests.

As a result, lots of people are quoting a number of “300,000″.

Cem Paya has a good analysis of what the OCSP numbers mean, what biases might be introduced at “DigiNotar: surveying the damage with OCSP.”

To their credit, FoxIt tried to investigate the extent of the damage by monitoring OCSP logs for users checking on the status of the forged Google certificate. There is a neat YouTube video showing the geographic distribution of locations around the world over time. Unfortunately while this half-baked attempt at forensics makes for great visualization, it presents a very limited picture of impacted users.

Digitar and Fox-IT released enough that a dedicated secondary analyst like Cem can see methodological flaws in what they did. What else could we learn if we had more of the raw observations? When I read the report, I noticed the claim “A number of malicious/hacker software tools was found. These vary from commonly used tools such a the famous Cain & Abel tool to tailor made software.” This claim mixes analysis and observation. The observation is that there was software with which the analyst was not familiar. It may be that it was a perl script or other code that can be easily skimmed to see that it was “tailor made.” It may be that it was just something re-compiled to not match a hash. We don’t know. Similarly, the report claims (4.1) “In at least one script, fingerprints from the hacker are left on purpose, which were also found in the Comodo breach investigation of March 2011.” Really? On purpose? Perhaps the fingerprints were inserted as a matter of dis-information. Perhaps the Fox-IT analyst called the intruder on the phone, and he owned up to it. We don’t know.

I want to be clear that I don’t mean to be picking on Fox-IT here. My understanding is that the report they prepped came out incredibly quickly, and kudos to them for that. I’ve cherry picked two areas where I can ask for better editing, but I’m very aware that that editing comes at a cost in timeliness.

Cem’s article is very much worth reading, as is the Fox-IT report. But Cem’s analysis helps illustrate a theme of the New School, which is that we need diverse perspectives and analysis brought to bear on each report. The more data we see, the more we can learn from it. No single analysis will tell us everything we might learn. (I made a similar point here.)

I am left with a question for Cem, which I would have added to his post, but couldn’t comment there. My question is, having given all that thought to all the biases, what do you think is the probably true number (or range) of affected people?

Why Do Outsiders Detect Breaches?

adam — Wed, 20 Apr 2011 16:06:30 +0000

So I haven’t had a chance to really digest the new DBIR yet, but one bit jumped out at me: “86% were discovered by a third party.” I’d like to offer up an explanatory story of why might that be, and muse a little on what it might mean for the deployment of intrusion detection technologies and process.

One common element of third party connections is that they tend to be constrained in various ways including firewalls, structured database queries, and suspicious administrators looking to point fingers. They also, being on trust boundaries, may be better places to deploy and tune an IDS.

And it seems to work, given that 86% of breaches are found in these relatively constrained environments. So what’s the takeaway? Have more partners? Outsourcing is good for security? (I’m not sure if I’m being facetious here.)

It’s hard to deploy IDS within a company (as shown by the 14% of breaches detected internally). A big part of that is that in-company data flows get very complex very quickly. So what to do?

We could throw up our hands and give up, or we could look to see if similar conditions might exist internally at many large organizations. And I think they do. One property of big, complex systems is that they’re hard to manage. Because they’re hard to manage, groups inside a company form service level agreements with other groups to ensure that they have mutual commitments. So perhaps a good rule of thumb would be to deploy IDS near SLAs. (There’s a tie here to Gunnar Peterson’s rule to start from the overall IT budget.)

One of the points that Andrew and I made in the book is that data isn’t enough. We all benefit from different perspectives and interpretations of that data. What do you think? What should we learn from the fact that almost all breaches are currently detected by third parties?

Dark Reading Virtual Event & Evidence-Based Risk Management

alex — Thu, 03 Feb 2011 13:54:18 +0000

Hey, I know it’s late notice, but I’ll be speaking at 10:30 EST today on EBRM and the Verizon DBIR:

https://www.techwebonlineevents.com/ars/eventregistration.do?mode=eventreg&F=1002809&K=CAA1BC&tab=agenda

Alex

	Core	Unique	expanding skills
In role behavior			Most Impact
Organizational Citizenship Behavior	Most Impact