The New School of Information Security

Dear Verisign: Trust requires Transparency

adam — Fri, 03 Feb 2012 16:16:17 +0000

On their blog, Verisign made the following statement, which I’ll quote in full:

As disclosed in an SEC filing in October 2011, parts of Verisign’s non-production corporate network were penetrated. After a thorough analysis of the attacks, Verisign stated in 2011, and reaffirms, that we do not believe that the operational integrity of the Domain Name System (DNS) was compromised.

We have a number of security mechanisms deployed in our network to ensure the integrity of the zone files we publish. In 2005, Verisign engineered real-time validation systems that were designed to detect and mitigate both internal and external attacks that might attempt to compromise the integrity of the DNS.

All DNS zone files were and are protected by a series of integrity checks including real-time monitoring and validation. Verisign places the highest priority on security and the reliable operation of the DNS.

This does not suffice to restore my trust in a company to which we have delegated trust decisions across thousands of websites. Verisign concealed a breach from us, and possibly from its own management, according to Joseph Menn, who reports:

The 10-Q said that security staff responded to the attack soon afterward but failed to alert top management until September 2011. It says nothing about a continuing investigation [...]

Reasonable people can differ on what constitutes a thorough analysis. Reasonable people can differ on response activity. We can probably all learn a lot from what happened. Reasonable people can’t argue that Verisign has paid some PR cost, and that they’ll continue to pay it until those who are supposed to trust them are satisfied. That satisfaction requires more than the statements made above. I’m sure Verisign would prefer that the story go away, in which case they should release the report today (with whatever minor redactions are appropriate).

If Verisign has what they believe is a thorough analysis, they need to release as a step along the way to restoring trust in their ability to operate important parts of the internet infrastructure. And Verisign need to release real information soon, before the technical public come to see them as stonewalling.

Threat Modeling Fails In Practice

alex — Thu, 02 Feb 2012 16:55:43 +0000

Would be interested in readers thoughts on Ian G’s post here:

https://financialcryptography.com/mt/archives/001357.html

Pulling A Stiennon: In The Cloud, The DMZ Is Dead

David Mortman — Wed, 01 Feb 2012 18:23:24 +0000

Calling something in the cloud a DMZ is just weird. Realistically, everything is a DMZ. After all, you are sharing data center space, and if your provider is using virtualization, hardware with all of their other customers. As such, each and every network segment you have is (or should be) isolated and have only a very small set of allowed ports/protocols/ips etc. So in a very real sense, in public cloud every network segment is a DMZ. And when everything is a DMZ, then calling anything a DMZ becomes pointless.

It’s better to call the segments by their function, e.g. web, app server, db, cache, mq whatever it is that the services in that security group are doing. It had the advantage of being easier to understand, closer to self-documenting and doesn’t imply a level of non-existent security like a term like DMZ does. Also by calling segments by their purpose, it points the security practitioner towards the right mindset of what types of traffic should or shouldn’t be allowed. All in all a very Jericho project kind of mentality.

[ETA: I had completely forgotten that Hoff covered this same issue in his Commode Computing talk last year. In particular see http://pic.twitter.com/wrx7F17R]

Time for an Award for Best Data?

adam — Wed, 01 Feb 2012 17:15:54 +0000

Yesterday, DAn Kaminsky said “There should be a yearly award for Best Security Data, for the best collection and disbursement of hard data and cogent analysis in infosec.” I think it’s a fascinating idea, but think that a yearly award may be premature. However, what I think is sorta irrelevant, absent data. So I’m looking for data on the question, do we have enough good data to issue an award yearly?

Please nominate in the comments.

Also, please discuss what the criteria should be.

Sharing Research Data

adam — Mon, 30 Jan 2012 15:45:38 +0000

I wanted to share an article from the November issue of the Public Library of Science, both because it’s interesting reading and because of what it tells us about the state of security research. The paper is “Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results.” I’ll quote the full abstract, and encourage you to read the entire 6 page paper.

Background
The widespread reluctance to share published research data is often hypothesized to be due to the authors’ fear that reanalysis may expose errors in their work or may produce conclusions that contradict their own. However, these hypotheses have not previously been studied systematically.

Methods and Findings
We related the reluctance to share research data for reanalysis to 1148 statistically significant results reported in 49 papers published in two major psychology journals. We found the reluctance to share data to be associated with weaker evidence (against the null hypothesis of no effect) and a higher prevalence of apparent errors in the reporting of statistical results. The unwillingness to share data was particularly clear when reporting errors had a bearing on statistical significance.

Conclusions
Our findings on the basis of psychological papers suggest that statistical results are particularly hard to verify when reanalysis is more likely to lead to contrasting conclusions. This highlights the importance of establishing mandatory data archiving policies.

Despite the fact that the research was done on papers published in psychology journals, it can teach us a great deal about the state of security research.

First, the full paper is available for free online. Compare and contrast with too many venues in information security.

Second, the paper considers and tests alternative hypotheses:

Although our results are consistent with the notion that the reluctance to share data is generated by the author’s fear that reanalysis will expose errors and lead to opposing views on the results, our results are correlational in nature and so they are open to alternative interpretations. Although the two groups of papers are similar in terms of research fields and designs, it is possible that they differ in other regards. Notably, statistically rigorous researchers may archive their data better and may be more attentive towards statistical power than less statistically rigorous researchers. If so, more statistically rigorous researchers will more promptly share their data, conduct more powerful tests, and so report lower p-values. However, a check of the cell sizes in both categories of papers (see Text S2) did not suggest that statistical power was systematically higher in studies from which data were shared. [Ed: "Text S2" is supplemental data considering the discarded hypothesis.]

But most important, what does it say about the quality of the data we so avariciously hoard in information security? Could it have something to do with higher prevalence of apparent errors?

Probably not. It might surprise you to hear me saying that, but hear me out. We almost never have hypotheses to test, and so our ability to perform statistical re-analysis is almost irrelevant. We’re much for fond of saying things like “It calls the same DLLs as Stuxnet, so it’s clearly also by the Israelis.” Actually, there are several implied hypotheses in there:

No code by different authors calls the same DLL
No code calls any undocumented APIs
Stuxnet DLLs are not documented

Stuxnet being written by the Israelis is clearly not a hypothesis, but a fact, as documented by Nostradamus.

More seriously, read the paper, see how good science is done, and ask if anyone is holding us back but ourselves.

Thanks to Cormac Herley for the pointer.

Aviation Safety

adam — Wed, 25 Jan 2012 16:06:00 +0000

The past 10 years have been the best in the country’s aviation history with 153 fatalities. That’s two deaths for every 100 million passengers on commercial flights, according to an Associated Press analysis of government accident data.

The improvement is remarkable. Just a decade earlier, at the time the safest, passengers were 10 times as likely to die when flying on an American plane. The risk of death was even greater during the start of the jet age, with 1,696 people dying — 133 out of every 100 million passengers — from 1962 to 1971. The figures exclude acts of terrorism.

…
There are a number of reasons for the improvements.

The industry has learned from the past. New planes and engines are designed with prior mistakes in mind. Investigations of accidents have led to changes in procedures to ensure the same missteps don’t occur again.
Better sharing of information. New databases allow pilots, airlines, plane manufactures and regulators to track incidents and near misses. Computers pick up subtle trends. For instance, a particular runway might have a higher rate of aborted landings when there is fog. Regulators noticing this could improve lighting and add more time between landings.

(“It’s never been safer to fly; deaths at record low“, AP, link to Seattle PI version.)

Well, it seems there’s nothing for information security to learn here. Move along.

Kudos to Ponemon

adam — Mon, 23 Jan 2012 15:59:03 +0000

In the past, we have has some decidedly critical words for the Ponemon Institute reports, such as “A critique of Ponemon Institute methodology for “churn”” or “Another critique of Ponemon’s method for estimating ‘cost of data breach’“. And to be honest, I’d become sufficiently frustrated that I’d focused my time on other things.

So I’d like to now draw attention to a post by Patrick Florer, “Some Thoughts about PERT and other distributions“, in which he says:

What follows are the results of an attempt to answer this question using a small data set extracted from a Ponemon Institute report called “Compliance Cost Associated with the Storage of Unstructured Information”, sponsored by Novell and published in May, 2011. I selected this report because, starting on page 14, all of the raw data are presented in tabular format. As an aside, this is the first report I have come across that publishes the raw data – please take note, Verizon, if you are reading this!

So I simply wanted to offer kudos to the Ponemon Institute for doing this.

I haven’t yet had a chance to dig into the report, but felt that given our past critiques I should take note of a very positive step.

Oracle’s 78 Patches This Quarter, Whatever…

David Mortman — Thu, 19 Jan 2012 14:49:08 +0000

There’s been a lot of noise of late because Oracle just released their latest round of patches and there are a total of 78 of them. There’s no doubt that that is a lot of patches. But in and of itself the number of patches is a terrible metric for how secure a product is. This is even more the case of companies that bundle all of their patches for all of their product lines at once. Most of the chatter I’ve seen, implies that all 78 are for the main Oracle database, but if you read their announcement, you’ll see the breakdown is as follows:

Oracle Database Server – 2 patches
Oracle Fusion Middleware – 11 patches
Oracle E-Business Suite – 3 patches
Oracle Supply Chain Products Suite – 1 patch
Oracle PeopleSoft – 6 patches
Oracle JD Edwards – 8 patches
Oracle Sun Products – 17 patches
Oracle Virtualization – 3 patches
Oracle MySQL – 27 patches

Fully 60% of the above patches are from OSS products. So which is more secure: open source or closed source. Or let’s compare Oracle DB vs MySQL: 2 versus 27 patches?

What do these numbers tell you? Absolutely nothing. Even with something like CVSS you still can’t tell which product is more secure. The whole thing is a load of malarkey. The product that is and will remain most secure is the one that you can manage and maintain the easiest for your organization.

Please Participate: Survey on Metrics

alex — Mon, 16 Jan 2012 17:24:24 +0000

I got an email from my friend John Johnson who is doing a survey about metrics. If you have some time, please respond…

————————————————————————————————————————————————

I am seeking feedback from others who may have experience developing and presenting security metrics to various stakeholders at their organization. I have a number of questions I’ve thought of, and put them into a simple survey form. I am looking for any examples of the good, bad and ugly involved in developing meaningful metrics. What has worked well and what has failed miserably? How have you packaged and presented the results in a meaningful way to your executives?

If you can spare a few minutes, please consider taking this survey. Even if you answer one question, it is helpful!

https://docs.google.com/spreadsheet/viewform?formkey=dGhDLXZHQVB5eEZoSy03aU5JQnZxV2c6MQ

You may also simply share an example, graphics or slides via email. I will be using your feedback to facilitate peer discussions and in a presentation aimed at educating security professionals on how they can improve their security metrics program.

Thanks in advance,

John

Continuous Deployment and Security

David Mortman — Mon, 16 Jan 2012 15:27:15 +0000

From an operations and security perspective, continuous deployment is either the best idea since sliced bread or the worst idea since organic spray pancakes in a can. It’s all of matter of execution. Continuos deployment is the logical extension of the Agile development methodology. Adam recently linked to an study that showed that a 25% increase in features lead to a 200% increase in code complexity, so by making change sets smaller we dramatically decrease the complexity in each release. This translates to a much lower chance of failure. Smaller change sets also mean that rolling back in the case of a failure state is also much easier. Finally, smaller change sets make identifying what broke unit and integration tests easier and far easier to code review which increases the chances of catching serious issues prior to deployment. All of this points to building systems that are more stable, more reliable, have less downtime and are easier to secure. This assumes, of course, that you are doing continuos deployment well.

In order for continuous deployment (and DevOps in general) to be successful there needs to be consistent process and automation. There are lots of other factors as well, such as qualified developers, proper monitoring, the right deployment tools but those are for another discussion.

Consistent processes are essential if you are to guarantee that the deployment happens the same way every time. To put it bluntly, when it comes to operations and security, variation is evil. Look to Gene Kim’s research (Visual Ops, Visual Ops Security) or more traditional manufacturing methodologies like Six-Sigma for a deep dive into why variation is so very very bad. The short version though is that in manufacturing, variation means products you can’t sell. In IT, variation means downtime, performance issues, and security issues. At the most basic level, if you are making changes and you are making changes to how you make the changes, you create a much harder situation from which to troubleshoot. This translates to longer incident response times and longer times to recovery which nobody wants. Especially in an online business.

The easiest way to keep deployment process consistent is to remove the human element as much as possible. In other words, automate as much it as possible. This has the added advantage of saving the humans for reviewing errors and identifying potential issues faster. It doesn’t matter which automation mechanism you use as long as it’s stable and supports your operating environment well. Ideally, it will either be the same system as currently being used the by the operations and applications teams (e.g. chef, puppet, cfengine) or be one that can integrated with those systems (e.g. hudson/jenkins).

With good check-in/build release messages, you even get automated logging for your change management systems and updates to your configuration management database (CMDB).