Pie charts are not always wrong

In a comment, Wade says “I’ll be the contrarian here and take the position that using pie charts is not always bad.” And he’s right. Pie charts are not always bad. There are times when they’re ok. As Wade says “If you have 3-4 datapoints, a pie can effectively convey what one is intending to present.” Which is true. But in every case I’ve seen, those situations are as well served with a small bar graph.

What’s the least contrived situation in which a pie chart is better than a bar graph or table? (Pac man and pies are two obvious examples.)

Symantec State of Security 2010 Report Out

http://www.symantec.com/content/en/us/about/presskits/SES_report_Feb2010.pdf

Thanks to big yellow for not making us register!  Oh, and Adam thanks you for not using pie charts…

The Visual Display of Quantitative Information

In Verizon’s post, “A Comparison of [Verizon's] DBIR with UK breach report,” we see:

pie-charts-suck.jpg

Quick: which is larger, the grey slice on top, or the grey slice on the bottom? And ought grey be used for “sophisticated” or “moderate”?


I’m confident that both organizations are focused on accurate reporting. I am optimistic that this small example in the utlity of pie charts will inform report writers. The report writers and their graphics departments, loving their customers, will move to bar charts to help them compare numbers between sources.

I’m confident that not using pie charts is a best practice.

Elsewhere: “The only time it makes sense to use a pie chart.”

And elsewhere: “The Visual Display of Quantitative Information, 2nd edition

Adam & Andy Jaquith: A conversation

In December, Andy Jaquith and I had a fun conversation about info security with Bill Brenner listening in. The transcript is at “Meeting of the Minds,” and the audio is here.

Measuring the unmeasurable — inspiration from baseball

The New School approach to information security promotes the idea that we can make better security decisions if we can measure the effectiveness of alternatives.  Critics argue that so much of information security is unmeasurable, especially factors that shape risk, that quantitative approaches are futile.  In my opinion, that is just a critique of our current methods and instruments, not any proof of ultimate feasability.  What we need is major innovations in metrics, instrumentation, and such.

We can take inspiration from other fields.  Consider this innovation in statistical value management in baseball, a.k.a. the ”Moneyball” approach:

Evaluating fielding is baseball’s hardest math. There are just too many unknowns in a play. How much ground did Jeter cover? How fast was the ball moving? In essence: How unlikely was it that he’d catch the ball?   [...]

Sportvision’s FieldFX camera system records the action while object-recognition software identifies each fielder and runner, as well as the ball. After a play, the system spits out data for every movement: the trajectory of the ball, how far the fielder ran, and so on. “After an amazing catch by an outfielder, we can compare his speed and route to the ball with our database and show the TV audience that this player performed so well that 80 percent of the league couldn’t have made that catch,” says Ryan Zander, Sportvision’s manager of baseball products. That information, he says, will allow a much more quantitative measure of exactly what is an error.

Happy Valentine’s Day!

They say that Y equals m-x plus b
(well, when you remove the uncertainty).
So let me reveal a secret confession:
You’re the solution to my least squares obsession.

stolen from the applied statistics blog

Open Security Foundation Looking for Advisors

Open Security Foundation – Advisory Board – Call for Nominations:

The Open Security Foundation (OSF) is an internationally recognized 501(c)(3) non-profit public organization seeking senior leaders capable of providing broad-based perspective on information security, business management and fundraising to volunteer for an Advisory Board. The Advisory Board will provide insight and guidance when developing future plans, an open forum for reviewing community feedback and a broader view when prioritizing potential new services.

I figure readers of this blog should be interested in helping drive open data sources.

Best Practices for Defeating the term “Best Practices”

I don’t like the term “Best Practices.” Andrew and I railed against it in the book (pages 36-38). I’ve made comments like “torture is a best practice,” “New best practice: think” and Alex has asked “Are Security “Best Practices” Unethical?

But people keep using it. Worse, my co-workers are now using it just to watch get me spun up. My continued snark is clearly a Best Practice because I keep doing it despite evidence that it doesn’t work.

I’d love to hear your experiences. What are proven or effective practices for getting people to stop using the term?

Podcast on ISM3

Last week, I spoke at the Open Group meeting here in Seattle, and then recorded a podcast with Dana Gardner, Jim Hietala and Vicente Aceituno about ISM3 Brings Greater Standardization to Security Measurement Across Enterprise IT (audio) or you can read the transcript.

It was fun, and the podcast is short and to the point. Take a listen!

Does It Matter If The APT Is “New”?

As best as I can describe the characteristics of the threat agents that would fit the label of APT, that threat community is very, very real.  It’s been around forever (someone mentioned first use of the term being 1993 or something) – we dealt with threat agents you would describe as “APT” at MicroSovled when I was there in 2001-2005.  We dealt with it as a firewall vendor at Progressive Systems in 1998.  This isn’t a “is the APT real?” blogpost.

That said, I wanted to talk about why there should be still more discussion around the APT.  Hogfly at the Forensic Incident Response blog asks:

“What should matter is how successful they have been. What should matter is defending ourselves. What should matter is how and where we share this information. What should matter is taking this information to those with the ability to do something about it. What should matter is taking the fight to the enemy.

So I ask again, does it matter if this threat is new?”

My response is that it actually matters very much.

We are hearing a new label.  Whether the label originated from “the cool kids” or not, it’s being co-opted by marketing.  And right now, we’re sort of in this important window of trying to get some understanding, some significant amount of intersubjectivity about what the APT is and what it means to a broader audience.  Once that’s established, then we can try to understand what to do.  But why does it matter if the threat is new or old?

There is a significant increase in the use of the term.  When it’s a BusinessWeek cover story (2008, btw), it gets seen by people.  What we need to understand is if this “new” visibility is the result of either a change in the threat landscape or a change in the marketing landscape.

IS APT A SHIFT IN FREQUENCY, A SHIFT IN CAPABILITY, OR A SHIFT IN BOTH FREQUENCY AND CAPABILITY?

If it is a change in the threat landscape, we need to understand what aspect of the landscape is changing.  The shift could be said to be one of a few scenarios:

1.)  More attacks on the same targets by the same actors. That is, is the government, defense industrial base, or other targets attractive to certain nation-states are experiencing a new amount of threat events.

2.) More attacks on new targets by the same actors. That is, are the nation-state actors finding new targets?  If so, are their targets of choice changing from organizations that are antagonistic to the policy desires of the sponsor state (certainly the Mandiant report reads like the Chinese are after anyone who threatens their political stability), to other targets – like retailers or hospitals (has, as Mandiant says, the APT become *everyone’s* problem)?

3.)  More attacks on the same targets by new actors. That is, it’s not just the usual suspects.  If *this* is the case, then we’re seeing a fundamental shift in the capabilities of threats.  That is, bad guys who used to be dumb just got a lot smarter thanks to the dissemination of skills/resources (sharing of technique, new access to advanced toolsets, etc) and they are going after all those people who were worrying about the APT in 2003.

4.)  More attacks on new targets by new actors. That is, the bad guys who used to be dumb just got a lot smarter and are now trying to use their new smarts against victims who heretofore had not had to worry about the APT.

Finally, the other option is that there is no shift in frequency or capability, but there is a shift in marketing budgets.  I tried to run a google trend on “Advanced Persistent Threat” but got:

Your terms – “Advanced Persistent Threat” – do not have enough search volume to show graphs.

And “APT” trend search was clouded by other things that shared the same TLA.

WHAT DO YOU THINK?

I’m not sure what we’re seeing.  I was personally disappointed by the Mandiant report’s lack of demographics and frequency information.  I’m ready to believe that we’re seeing a fundamental shift in distributions concerning the threat agents, but there wasn’t anything in the report to support that notion.  I will leave you with a couple of items from the Verizon Report, though, and I’ll let you draw your own conclusions, given that the Verizon data set isn’t heavy on what we might call the Defense Industrial Base – those folks already live and breathe this stuff  – and this data is from 2008.

SOURCE OF ATTACKING IP

TARGETED VS. OPPORTUNISTIC ATTACKS

TREND IN USE OF CUSTOMIZED MALWARE

TIME TO DISCOVERY