https://financialcryptography.com/mt/archives/001357.html

I am seeking feedback from others who may have experience developing and presenting security metrics to various stakeholders at their organization. I have a number of questions I’ve thought of, and put them into a simple survey form. I am  looking for any examples of the good, bad and ugly involved in developing meaningful metrics. What has worked well and what has failed miserably? How have you packaged and presented the results in a meaningful way to your executives?

If you can spare a few minutes, please consider taking this survey. Even if you answer one question, it is helpful!

https://docs.google.com/spreadsheet/viewform?formkey=dGhDLXZHQVB5eEZoSy03aU5JQnZxV2c6MQ

You may also simply share an example, graphics or slides via email. I will be using your feedback to facilitate peer discussions and in a presentation aimed at educating security professionals on how they can improve their security metrics program.

Thanks in advance,

John

Steve’s post contains excellent context, putting the issue in context of nuclear secrets, cryptography and software vulnerability disclosure. I want to follow up a bit on his closing:

The ultimate decision may rest on personal attitudes. To quote Fouchier one more time, “The only people who want to hold back are the biosecurity experts. They show zero tolerance to risk. The public health specialists do not have this zero tolerance. I have not spoken to a single public health specialist who was against publication.”

I think that personal preference is one way to think of this, and perhaps in fact, personal preference drives the choice of profession. But perhaps what’s really happening is that public health specialists are operating with a different set of drivers than “biosecurity experts.” In particular, given the very low incidence of ‘biosecurity incidents,’ perhaps ‘biosecurity experts’ are operating in a world where all threats exist only on paper (or in papers). In contrast, public health professionals have real epidemics and pandemics to deal with. They’re forced to deal with the propaganda of anti-vaccination nuts whose fear of autism is killing people with whooping cough and other diseases. They have to deal with contamination of the food supply. They can reasonably prioritize preventing salmonella or e.coli over theoretical terrorist threats.

However, this narrow focus on preventing all problems (in contrast to risk management, cost-benefit or other pragmatic approaches) is not unique to bio-security experts. The security professional, focused by definition on security, will naturally tend towards zero tolerance for risks.

An example, already reduced to absurdity, is visible in the TSA. Their goal is not balanced security, it’s a relentless and offensive pursuit of security at the expense of dignity, calm, and cupcakes. But we should not be surprised at their pursuit of the cupcake. It’s the natural result of having an agency focused entirely on security.

This is, by the way, relates to why CISOs should report into a functional area of the business, be it operations or IT, rather than reporting to the CEO. If the CISO is focused entirely on security, then those concerns need to be balanced with the overall operational picture by someone with accountability for delivering of a whole to the business, not treated as some special magic.

]]> http://newschoolsecurity.com/2012/01/steve-bellovins-lessons-from-suppressing-research/feed/ 0 http://newschoolsecurity.com/2011/11/its-time-to-learn-like-experts-by-jay-jacobs/ http://newschoolsecurity.com/2011/11/its-time-to-learn-like-experts-by-jay-jacobs/#comments Mon, 28 Nov 2011 15:26:03 +0000 adam http://newschoolsecurity.com/?p=2344 I want to call attention to a new, important and short article by Jay Jacobs.

This article is a call to action to break the reliance on unvalidated expert opinions by raising awareness of our decision environment and the development of context-specific feedback loops.

Everyone in the New School is a fan of feedback loops of one form or another. Hypothesis testing, learning, and calling out superstition are all forms of feedback loops.

One thing that Jay brings in that I hadn’t seen is the idea of kind and wicked learning environments. A kind environment is one in which you can quickly get good feedback on things experts agree will help you improve. (Did you fall off the bike?) An unkind environment is, amongst other things, one where feedback comes later, if at all. Jay has a table. It’s on page 2.

You should find Jay’s article here: “A Call to Arms: It’s Time to Learn Like Experts“, or his short blog here.

Let me say that again: the attacker IP address being revealed revived and revitalized the debate about PKI and certificate authorities. Without that, the motivators and even the truth of the claim of clinical Advanced Persistent Cyber Ninja Dudes would have been hard to contest. Without that, we might have believed for a few more years the bizarre hypothesis that Certification Authorities are a useful part of internet trust. With the IP address Moxie was able to test those ideas, and show exactly how flawed they are.

Before I say the rest of what I want to say, let me say that I like Moxie. I think he’s a good guy, does really good work, and I always enjoy talking with him. But Moxie isn’t the “sort of person” who’s going to “fit in” at a London meeting with the Prime Minister. He might not have an easy time getting “read in” for “information sharing programs” operated by people who work for three letter agencies and think that a background check every year is a normal way to live. But we need different perspectives, backgrounds and approaches to learn as much as we can from data. If we limit it to those who “fit in,” then we implicitly limit the perspectives, frames and orientations which are brought to bear.

But let me give benefit of the doubt to those information sharing folks. They deserve it. Many of them are quite smart and hard-working. Several of the ones I met with recently had really interesting things to say. A fellow named Paul had fascinating things to say about the economics of information sharing–things I hadn’t heard before. And folks like Mudge are getting read in. So perhaps Moxie could get access to those meetings and mailing lists. If he agreed to limit how he distributed information, he could have maybe had access to those 4 bytes of Internet Protocol address.

If we treat that IP address like a nugget of treasure, he’s unlikely to see it, and if he sees it, he may be unable to talk about it. Moxie was able to analyze the attack because the information was published, not shared. Moxie was able to publish share his analysis of the attack because the information was published, not shared. Moxie was able to tell a convincing story because the information was published, not shared. And I’m able to talk about and expand apon what he said because (wait for it!) the information was published, not shared.

We need to publish more data about what goes wrong, because when do, we can share new ideas, let them cross-fertilize and sometimes even converge into progress.

[Update: thanks for the correction, Nicko.]

]]> http://newschoolsecurity.com/2011/11/breach-disclosure-and-moxies-convergence/feed/ 1 http://newschoolsecurity.com/2011/10/eff-on-https/ http://newschoolsecurity.com/2011/10/eff-on-https/#comments Mon, 31 Oct 2011 14:41:28 +0000 Chandler http://newschoolsecurity.com/?p=2310 The Electronic Frontier Foundation has published a report on the State of HTTPS Security that promises to be the first in a series and is well worth reading on its own.

The TL;DR version:  HTTPS adoption is growing rapidly, but the current system, especially the Certificate Authorities, has much room for improvement before it actually delivers the level of security that HTTPS implies.

Also, if you’re on Firefox and don’t already use it, EFF’s HTTPS Everywhere Plug-In is now officially 1.0.

It’s awesome and it sucks.

IT’S AWESOME

That’s not damning with faint praise, rather, it’s acknowledging that it’s not really “risk” but is a useful tool if your goal is to be quick and dirty about vulnerability severity.

In other words, this is much better than CVSS, and should probably replace it immediately.

TILTING AFTER THE WRONG WINDMILLS

In fact, it’s a shame that Ben chose to compare this to OCTAVE, FAIR, SOMAP and others.  Because if he positioned this as “stop screwing around with CVSS” and “not really risk but a vuln rating” I would be telling everyone how much I liked it in that role.

In addition, if he positioned it with the Accounting/Audit Industrial Complex as good tool in the toolbox to compete with^H^H^H^H^H^H  augment their RCSA nonsense, I could probably welcome it there, as well (though not as an optimal solution).

The power of BRA is the fact that Ben chose to make things “binary”.  I can see this simple approach working well because it doesn’t allow you granularity – none of this arguing over “Moderate” or “Moderate-High” – just yes/no.

Also, Ben’s done a really good job thinking through what creates risk.  For the FAIR familiar there’s the concepts of TCap and Control/Resistance Strength.  I like that.

IT SUCKS

Speaking of subjectivity, I believe that Ben uses the power of binary choice to suggest that BRA “highlights” subjectivity.  Not to be a rude pedagogue, but it really doesn’t “highlight” subjectivity as much as it just doesn’t give you many choices as to where to “put” that subjective measurement.  Everything about it is still subjective (but that’s OK), and to reduce (or as I would rather “address properly”) that subjectivity would take more complexity than I believe Ben wanted to build (again, that’s OK).

As such at the end of the day, Ben’s right, it’s never going to be a replacement for what he calls “complex” analysis methodologies.  And because it doesn’t properly address subjectivity, BRA is not for formal risk or threat modeling.  I could never use it in my current capacity, as BRA just leaves a few too many questions unanswered.  I don’t have time for the arguing, I just want your SME estimate, throw that puppy into OpenPERT and be done with it.

Furthermore, it’s odd because even though BRA suggests that it is designed to   to “not ask anyone to guess on event frequency in the absence of statistical data (whatever that is)” it seems Ben’s intellectual honesty still could not let him escape the need to highlight it.  If you look at BRA the model, that occurrence thing there, yeah, that’s frequency.  It’s just a “binary” frequency determination which means….

BRA only talks about what’s possible.  

As a risk model, this is the point at which we reference the Tacoma Narrows suspension bridge that oscillated wildly in the wind.  Constructed nicely and all, but a small fundamental flaw in design renders it crazy bad for its purpose.

Also, impact is difficult for me to buy into because it uses asset value.  I hate to break it to you, but asset value mainly matters to threat motivation modeling.  The accounting value of the asset is RARELY the same as the losses we actually realize.

CHOOSE IT OR CHUCK IT?

So it wouldn’t be a review without such criticism.  This is one reason I hate reviewing things, because it is a critical process.  So please note that the above isn’t said with malice, it’s just an examination of the model itself.

In fact, as a tool, I wouldn’t dismiss it just yet.  If your security group isn’t formally into risk, is stuck doing too much with CVSS for too little return, I’d jump all over this.  If you have bigger fish to fry than an enterprise risk assessment but have the regulatory duty to create a risk register, BRA might just be the thing.  If you find yourself faced with an absurd RCSA from audit or something – I might whip out the sweet BRA iPad app and run a scenario or two through.   If I actually wanted a risk analysis, however, I would go elsewhere.

]]> http://newschoolsecurity.com/2011/10/some-thoughts-on-binary-risk-analysis/feed/ 3 http://newschoolsecurity.com/2011/08/change/ http://newschoolsecurity.com/2011/08/change/#comments Tue, 16 Aug 2011 13:44:46 +0000 alex http://newschoolsecurity.com/?p=2259 I’ve left Verizon.  A lot of folks have come up to me and asked, so I thought I’d indulge in a rather self-important blog-post and explain something:

It wasn’t about Verizon, but about the opportunity I’ve taken.

Wade, Chris, Hylender, Marc, Joe, Dave, Dr. Tippett & all the rest – they were all really, really great to me.  I love the RISK team.  Loved my job.  But the opportunity came up to do something, well, awesome.  So I left a great group and joined a financial institution as Director of Operational Risk.  If you ever get offered a chance to work for Wade at Verizon Terremark, I HIGHLY recommend it.

I’m looking forward to the change, and I’m looking forward to re-aligning my priorities.  Now that BlackHat & Defcon are over, now that Metricon is behind me, I’ve already started to scale back on SIRA management, reduced my role in the CSDM, and am in a refocusing phase.  I’ve missed blogging, and as I charge forward in “the real world” hope to share some scars and successes with you in this medium (data-driven, of course).

How I protect myself against phishing

I do a variety of things to protect myself from phishing attacks, including bookmarking my banking web sites, and setting up special email addresses that are only given to a single business. For example, Capital One (an Epsilon customer) might think that my email is cap1-814406d6fa52c5317aa@example.com. Now, any email that comes to that address has some special properties. I know that either it came from the expected sender, or there’s been a breach of confidentiality at the sender, the intervening network, or on my systems. In that sense, these addresses are honeytokens. In many instances, the entities I’m working with use opportunistic TLS for email, and so I can be confident that it’s not passive sniffing of the intervening network. Now, since I have technology that makes it easy to see if an email went to the right address, I can ignore most emails claiming to be from financial institutions. I save a great deal of time and energy that way. But for the emails that come into the special mailboxes, I can also save time by going with the assumption that they’re ok, because this defense tends to work.

How the breach hurts me

Since Epsilon was good enough to bring up the breach, and Capital One was good enough to contact their customers, I’m aware that my defenses are less strong than they otherwise would be. I have to expend more energy than I otherwise would have reading URLs in messages in that folder. If the breach had been concealed, then I would be naively vulnerable. I would be vulnerable because respectable experts hadn’t thought about this scenario, and had naively decided that I didn’t need to know about the incident.

The Limits of Expertise

This is one example of the limits of experts to understand the impact of breaches on consumers. There are doubtless others, and we should be willing to question the limits of our expertise to fully understand the impacts of breaches on everyone. We should also question our expertise to decide for them what’s best for others.

Breach Fatigue?

Now, some people will argue that there’s “breach fatigue”, and that that means we should select for others which incidents they’ll hear about and which they won’t. While I agree that there’s breach fatigue, that’s a weak argument in a free society. People should be able to learn about incidents which may have an effect on them, so that they (and I) can make good risk management decisions. We don’t argue against telling people that there’s lead paint on Chinese toys even though much of the damage will already have been done by the paint that’s flaked off. We don’t argue against telling the public about stock trades by insiders even though only a few experts look at it. We as a free society encourage the free flow of information, knowing that it will be put to a plethora of uses.

This is just one of the many reasons why I support broad breach notification.

There are some technical details after the break.The break>

So, how this works in practice. The actual procmail I use looks a lot like this. Domains and addresses have been altered.

:0:
* ^To: cap1-814406d6fa52c5317aa@example.com
capital-one

:0:
* ^To: boa-ed218d18fbf844d677970@example.com
bank-of-america
# I don't think Bank of America uses Epsilon; I want to illustrate the one-off nature of the addresses

So can you do this? If you have access to a domain and procmail, it’s easy. Setting those up is a fair amount of work. You could do something similar with “+ addressing” which some mail providers support, but some web developers break your ability to enter a + in an email address, mistakenly thinking it blocks SQL injection. You could also use a unique address (sdhjfdslfh237232@yahoo is still available!) and check each regularly, but that’s probably more work–one of the nice things about the procmail solution is that it integrates seamlessly into my personal email flow. [Update: If you don't have a domain, see Kurt's comment. I wasn't aware that businesses that did this exist. Note that using them like this adds a party to the trust list.]

Now, I could do more. I could check DKIM signatures before depositing the mail, which would break the ability of the Epsilon attackers to fake me out, and maybe I will. I could do other consistency checking a la tofu. But I wouldn’t have thought about it without knowing about the breach. And in fact, I think this method works incredibly well without that. As long as we have breach notification.

]]> http://newschoolsecurity.com/2011/06/how-the-epsilon-breach-hurts-consumers/feed/ 3