The New School of Information Security » adam

On Threat Modeling

adam — Mon, 06 Feb 2012 15:58:30 +0000

Alex recently asked for thoughts on Ian Grigg’s “Why Threat Modeling Fails in Practice.”

I’m having trouble responding to Ian, and have come to think that how Ian frames the problem is part of my problem in responding to him. So, as another Adam likes to say, “I reject your reality, and substitute my own.” Here you go:

“Experiences Threat Modeling at Microsoft” covers the trouble that threat modeling is an aspirational tabula rasa, and people project all sorts of requirements onto processes and methodologies.
However, I agree with Ian that there’s lots of “Trouble with Threat Modeling.”
See also my MSDN magazine articles “Uncover Security Design Flaws Using The STRIDE Approach” and “Reinvigorate your Threat Modeling Process” is about how I’m thinking about
threat modeling and some lessons learned. MSDN also published “Getting Started With The SDL Threat Modeling Tool.”

But that’s not my final answer. My final answer is your threat modeling fails because you’re not using Elevation of Privilege.

(Actually, I don’t think that’s why Ian’s threat modeling fails in practice. He’s a smart guy, and I think the issue seems to be one of expectations versus approach, and I think either could be usefully changed, depending on the context.)

Dear Verisign: Trust requires Transparency

adam — Fri, 03 Feb 2012 16:16:17 +0000

On their blog, Verisign made the following statement, which I’ll quote in full:

As disclosed in an SEC filing in October 2011, parts of Verisign’s non-production corporate network were penetrated. After a thorough analysis of the attacks, Verisign stated in 2011, and reaffirms, that we do not believe that the operational integrity of the Domain Name System (DNS) was compromised.

We have a number of security mechanisms deployed in our network to ensure the integrity of the zone files we publish. In 2005, Verisign engineered real-time validation systems that were designed to detect and mitigate both internal and external attacks that might attempt to compromise the integrity of the DNS.

All DNS zone files were and are protected by a series of integrity checks including real-time monitoring and validation. Verisign places the highest priority on security and the reliable operation of the DNS.

This does not suffice to restore my trust in a company to which we have delegated trust decisions across thousands of websites. Verisign concealed a breach from us, and possibly from its own management, according to Joseph Menn, who reports:

The 10-Q said that security staff responded to the attack soon afterward but failed to alert top management until September 2011. It says nothing about a continuing investigation [...]

Reasonable people can differ on what constitutes a thorough analysis. Reasonable people can differ on response activity. We can probably all learn a lot from what happened. Reasonable people can’t argue that Verisign has paid some PR cost, and that they’ll continue to pay it until those who are supposed to trust them are satisfied. That satisfaction requires more than the statements made above. I’m sure Verisign would prefer that the story go away, in which case they should release the report today (with whatever minor redactions are appropriate).

If Verisign has what they believe is a thorough analysis, they need to release as a step along the way to restoring trust in their ability to operate important parts of the internet infrastructure. And Verisign need to release real information soon, before the technical public come to see them as stonewalling.

Time for an Award for Best Data?

adam — Wed, 01 Feb 2012 17:15:54 +0000

Yesterday, DAn Kaminsky said “There should be a yearly award for Best Security Data, for the best collection and disbursement of hard data and cogent analysis in infosec.” I think it’s a fascinating idea, but think that a yearly award may be premature. However, what I think is sorta irrelevant, absent data. So I’m looking for data on the question, do we have enough good data to issue an award yearly?

Please nominate in the comments.

Also, please discuss what the criteria should be.

Sharing Research Data

adam — Mon, 30 Jan 2012 15:45:38 +0000

I wanted to share an article from the November issue of the Public Library of Science, both because it’s interesting reading and because of what it tells us about the state of security research. The paper is “Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results.” I’ll quote the full abstract, and encourage you to read the entire 6 page paper.

Background
The widespread reluctance to share published research data is often hypothesized to be due to the authors’ fear that reanalysis may expose errors in their work or may produce conclusions that contradict their own. However, these hypotheses have not previously been studied systematically.

Methods and Findings
We related the reluctance to share research data for reanalysis to 1148 statistically significant results reported in 49 papers published in two major psychology journals. We found the reluctance to share data to be associated with weaker evidence (against the null hypothesis of no effect) and a higher prevalence of apparent errors in the reporting of statistical results. The unwillingness to share data was particularly clear when reporting errors had a bearing on statistical significance.

Conclusions
Our findings on the basis of psychological papers suggest that statistical results are particularly hard to verify when reanalysis is more likely to lead to contrasting conclusions. This highlights the importance of establishing mandatory data archiving policies.

Despite the fact that the research was done on papers published in psychology journals, it can teach us a great deal about the state of security research.

First, the full paper is available for free online. Compare and contrast with too many venues in information security.

Second, the paper considers and tests alternative hypotheses:

Although our results are consistent with the notion that the reluctance to share data is generated by the author’s fear that reanalysis will expose errors and lead to opposing views on the results, our results are correlational in nature and so they are open to alternative interpretations. Although the two groups of papers are similar in terms of research fields and designs, it is possible that they differ in other regards. Notably, statistically rigorous researchers may archive their data better and may be more attentive towards statistical power than less statistically rigorous researchers. If so, more statistically rigorous researchers will more promptly share their data, conduct more powerful tests, and so report lower p-values. However, a check of the cell sizes in both categories of papers (see Text S2) did not suggest that statistical power was systematically higher in studies from which data were shared. [Ed: "Text S2" is supplemental data considering the discarded hypothesis.]

But most important, what does it say about the quality of the data we so avariciously hoard in information security? Could it have something to do with higher prevalence of apparent errors?

Probably not. It might surprise you to hear me saying that, but hear me out. We almost never have hypotheses to test, and so our ability to perform statistical re-analysis is almost irrelevant. We’re much for fond of saying things like “It calls the same DLLs as Stuxnet, so it’s clearly also by the Israelis.” Actually, there are several implied hypotheses in there:

No code by different authors calls the same DLL
No code calls any undocumented APIs
Stuxnet DLLs are not documented

Stuxnet being written by the Israelis is clearly not a hypothesis, but a fact, as documented by Nostradamus.

More seriously, read the paper, see how good science is done, and ask if anyone is holding us back but ourselves.

Thanks to Cormac Herley for the pointer.

Aviation Safety

adam — Wed, 25 Jan 2012 16:06:00 +0000

The past 10 years have been the best in the country’s aviation history with 153 fatalities. That’s two deaths for every 100 million passengers on commercial flights, according to an Associated Press analysis of government accident data.

The improvement is remarkable. Just a decade earlier, at the time the safest, passengers were 10 times as likely to die when flying on an American plane. The risk of death was even greater during the start of the jet age, with 1,696 people dying — 133 out of every 100 million passengers — from 1962 to 1971. The figures exclude acts of terrorism.

…
There are a number of reasons for the improvements.

The industry has learned from the past. New planes and engines are designed with prior mistakes in mind. Investigations of accidents have led to changes in procedures to ensure the same missteps don’t occur again.
Better sharing of information. New databases allow pilots, airlines, plane manufactures and regulators to track incidents and near misses. Computers pick up subtle trends. For instance, a particular runway might have a higher rate of aborted landings when there is fog. Regulators noticing this could improve lighting and add more time between landings.

(“It’s never been safer to fly; deaths at record low“, AP, link to Seattle PI version.)

Well, it seems there’s nothing for information security to learn here. Move along.

Kudos to Ponemon

adam — Mon, 23 Jan 2012 15:59:03 +0000

In the past, we have has some decidedly critical words for the Ponemon Institute reports, such as “A critique of Ponemon Institute methodology for “churn”” or “Another critique of Ponemon’s method for estimating ‘cost of data breach’“. And to be honest, I’d become sufficiently frustrated that I’d focused my time on other things.

So I’d like to now draw attention to a post by Patrick Florer, “Some Thoughts about PERT and other distributions“, in which he says:

What follows are the results of an attempt to answer this question using a small data set extracted from a Ponemon Institute report called “Compliance Cost Associated with the Storage of Unstructured Information”, sponsored by Novell and published in May, 2011. I selected this report because, starting on page 14, all of the raw data are presented in tabular format. As an aside, this is the first report I have come across that publishes the raw data – please take note, Verizon, if you are reading this!

So I simply wanted to offer kudos to the Ponemon Institute for doing this.

I haven’t yet had a chance to dig into the report, but felt that given our past critiques I should take note of a very positive step.

Please vote New School

adam — Thu, 12 Jan 2012 17:31:59 +0000

We’re honored to be nominated in three categories for the Security Bloggers Awards:

Most Educational
Most Entertaining
Hall of Fame

On behalf of all of us who blog here, we’re honored by the nomination, and would like to ask for your vote.

We’d also like to urge you to vote for our friends at Securosis for “Best Representing the Security Industry.” We don’t think Securosis actually is the best representative of the industry today. But I think they represent what we all ought to aspire to be, a empirical, business-aware industry. So please consider them as a part of the broad “New School” sort of slate. We’d also like to put a word in for the ThreatPost podcast as a great mix of technical and non-technical content, and for Veracode for best corporate blog. We’re suggesting Veracode in large part for Chris Eng’s empirical and side-splittingly funny thought leadership videos, but also for a general avoidance of FUD in their blogging.

But whomever you like, please take a moment to vote.

The New School of Software Engineering?

adam — Wed, 11 Jan 2012 16:33:27 +0000

This is a great video about how much of software engineering runs on folk knowledge about how software is built:

“Greg Wilson – What We Actually Know About Software Development, and Why We Believe It’s True”

There’s a very strong New School tie here. We need to study what’s being done and how well it works to figure out how to make better software more reliably.

Incidentally, at around 28 minutes in, Wilson mentions Nachi Nagappan‘s work on physical distance versus managerial distance, and then jumps to remote hires at a a startup. While I’m not sure of which paper Wilson is discussing, almost all of Nagappan’s work is done with Microsoft developers and products. As such, both have to be seen in the context of Microsoft’s deep and shared experience in shipping software. By definition, that shared experience doesn’t exist at a startup. And as to the managerial distance issue, it’s satirically discussed here. Assuming that his results generalize is a large jump, and one that I’m not sure I’d make.

New School Approaches to Passwords

adam — Tue, 10 Jan 2012 17:06:20 +0000

Adam Montville left a comment on my post, “Paper: The Security of Password Expiration“, and I wanted to expand on his question:

Passwords suck when they’re not properly cared for. We know this. Any other known form of
authentication we have is difficult because of the infrastructure required to pull it off. That
sucks too. Does this leave us at a stalemate where we need to get people to care about their
passwords?

I think the answer is “almost.” We need to agree that passwords suck when they’re not properly cared for, and that caring for them is hard. So we need to assume that passwords will tend to be poor, reused, etc, and develop methods to deal with that. Most of our mechanisms today punish users. We tell them to memorize 100 or more unique passwords, and then “security experts” abuse them for re-use or using a password management tool.

Cormac Herley has claimed that the password has a set of properties including being subject to memorization that make it impossible to replace, and we should accept that and start engineering for it. (“A Research Agenda Acknowledging the Persistence of Passwords” and “Passwords: If We’re So Smart Why Are We Still Using Them?“)

Similarly, Nate Lawson posted “On the evolving security of password schemes” which closes “most admins focus too much on increasing entropy of user choices and not enough on decreasing the attacker’s guess rate and implementing responses to limit their access when they do get a hit.” Indeed.

We need to observe the world, and ask how we can work within the constraints it presents regardless of if those constraints are economic, sociological or evolutionary.

Paper: The Security of Password Expiration

adam — Thu, 05 Jan 2012 16:19:14 +0000

The security of modern password expiration: an algorithmic framework and empirical analysis, by Yingian Zhang, Fabian Monrose and Michael Reiter. (ACM DOI link)

This paper presents the first large-scale study of the success of password expiration in meeting its intended purpose, namely revoking access to an account by an attacker who has captured the account’s password. Using a dataset of over 7700 accounts, we assess the extent to which passwords that users choose to replace expired ones pose an obstacle to the attacker’s continued access. We develop a framework by which an attacker can search for a user’s new password from an old one, and design an efficient algorithm to build an approximately optimal search strategy. We then use this strategy to measure the difficulty of breaking newly chosen passwords from old ones. We believe our study calls into question the merit of continuing the practice of password expiration.

This is the sort of work that we at the New School love. Take a best practice recommended by just about everyone for what seems like excellent reasons, and take notice of the fact that human beings are going to game your practice. Then get some actual data, and see how effective the practice is.

Unfortunately, we lack data on rates of compromise for organizations with different password change policies. So it’s hard to tell if password policies actually do any good, or which ones do good. However, we can guess that not making your default password “stratfor” is a good idea.

ACM gets a link because they allow you to post copies of your own papers, rather than inhibiting the progress of science by locking it all up.