The New School of Information Security » breaches

How to mess up your breach disclosure

adam — Fri, 30 Mar 2012 15:57:46 +0000

Congratulations to Visa and Mastercard, the latest companies to not notify consumers in a prompt and clear manner, thus inspiring a shrug and a sigh from consumers.

No, wait, there isn’t a clear statement, but there is rampant speculation and breathless commentary.

It’s always nice to see clear reminders that the way to get people excited about a breach is to dribble out the information. For what little the public knows, to help Brian Krebs piece together the story and decide how the public will come to understand it because Visa and Mastercard aren’t talking, see MasterCard, VISA Warn of Processor Breach.

Why Breach Disclosures are Expensive

adam — Tue, 07 Feb 2012 17:11:42 +0000

Mr. Tripathi went to work assembling a crisis team of lawyers and customers and a chief security officer. They hired a private investigator to scour local pawnshops and Craigslist for the stolen laptop. The biggest headache, he says, was deciphering how much about the breach his nonprofit needed to disclose…Mr. Tripathi said he quickly discovered just how many ways there were to count to 500. The law requires disclosure only in cases that “pose a significant risk of financial, reputational or other harm to the individual affected.” His team spent hours poring over a backup of the stolen laptop files.
(“Digital Data on Patients Raises Risk of Breaches“, Nicole Perlroth, The New York Times, Dec 18 2011)

This is the effect of trigger provisions: it’s the biggest headache in dealing with a breach. We shouldn’t be burdening businesses with the decision about what a significant risk entails, exposing them to the liability of making a wrong call, or risking that their decisions will be biased.

Dear Verisign: Trust requires Transparency

adam — Fri, 03 Feb 2012 16:16:17 +0000

On their blog, Verisign made the following statement, which I’ll quote in full:

As disclosed in an SEC filing in October 2011, parts of Verisign’s non-production corporate network were penetrated. After a thorough analysis of the attacks, Verisign stated in 2011, and reaffirms, that we do not believe that the operational integrity of the Domain Name System (DNS) was compromised.

We have a number of security mechanisms deployed in our network to ensure the integrity of the zone files we publish. In 2005, Verisign engineered real-time validation systems that were designed to detect and mitigate both internal and external attacks that might attempt to compromise the integrity of the DNS.

All DNS zone files were and are protected by a series of integrity checks including real-time monitoring and validation. Verisign places the highest priority on security and the reliable operation of the DNS.

This does not suffice to restore my trust in a company to which we have delegated trust decisions across thousands of websites. Verisign concealed a breach from us, and possibly from its own management, according to Joseph Menn, who reports:

The 10-Q said that security staff responded to the attack soon afterward but failed to alert top management until September 2011. It says nothing about a continuing investigation [...]

Reasonable people can differ on what constitutes a thorough analysis. Reasonable people can differ on response activity. We can probably all learn a lot from what happened. Reasonable people can’t argue that Verisign has paid some PR cost, and that they’ll continue to pay it until those who are supposed to trust them are satisfied. That satisfaction requires more than the statements made above. I’m sure Verisign would prefer that the story go away, in which case they should release the report today (with whatever minor redactions are appropriate).

If Verisign has what they believe is a thorough analysis, they need to release as a step along the way to restoring trust in their ability to operate important parts of the internet infrastructure. And Verisign need to release real information soon, before the technical public come to see them as stonewalling.

[Update: Welcome, Schneier blog readers! I wanted to clarify the status: we have a very data-free set of assertions from someone claiming to be a Symantec employee. We do not yet have a detailed report on the investigation that addresses who knew what when, and how they knew it.]

The Diginotar Tautology Club

adam — Fri, 23 Sep 2011 15:09:06 +0000

I often say that breaches don’t drive companies out of business. Some people are asking me to eat crow because Vasco is closing its subsidiary Diginotar after the subsidiary was severely breached, failed to notify their reliant parties, mislead people when they did, and then allowed perhaps hundreds of thousands of people to fall victim to a man in the middle attack. I think Diginotar was an exception that proves the rule.

Statements about Diginotar going out of business are tautological. They take a single incident, selected because the company is going out of business, and then generalize from that. Unfortunately, Diginotar is the only CA that has gone out of business, and so anyone generalizing from it is starting from a set of businesses that have gone out of business. If you’d like to make a case for taking lessons from Diginotar, you must address why Comodo hasn’t gone belly up after *4* breaches (as counted by Moxie in his BlackHat talk).

It would probably also be helpful to comment on how Diginotar’s revenue rate of 200,000 euros in 2011 might contribute to it’s corporate parent deciding that damage control is the most economical choice, and what lessons other businesses can take.

To be entirely fair, I don’t know that Diginotar’s costs were above 200,000 euros per year, but a quick LinkedIn search shows 31 results, most of whom have not yet updated their profiles.

So take what lessons you want from Diginotar, but please don’t generalize from a set of one.

I’ll suggest that “be profitable” is an excellent generalization from businesses that have been breached and survived.

[Update: @cryptoki pointed me to the acquisition press release, which indicates 45 employees, and "DigiNotar reported revenues of approximately Euro 4.5 million for the year 2009. We do not expect 2010 to be materially different from 2009. DigiNotar’s audited statutory report for 2009 showed an operating loss of approximately Euro 280.000, but an after-tax profit of approximately Euro 380.000." I don't know how to reconcile the press statements of January 11th and August 30th.]

The Rules of Breach Disclosure

adam — Wed, 07 Sep 2011 15:50:49 +0000

There’s an interesting article over at CIO Insight:

The disclosure of an email-only data theft may have changed the rules of the game forever. A number of substantial companies may have inadvertently taken legislating out of the hands of the federal and state governments. New industry pressure will be applied going forward for the loss of fairly innocuous data. This change in practice has the potential to affect every CIO who collects “contact” information from consumers, maybe even from employees in an otherwise purely commercial context. (“Breach Notification: Time for a Wake Up Call“, Mark McCreary of Fox Rothschild LLP)

My perspective is that breach disclosure now hurts far less than it did a mere five years ago, and spending substantial time on analysis of “do we disclose” is returning less and less value. As companies disclose, we’re getting more and more data that CIOs can use to improve IT operations. We can, in a very real way, start to learn from each other’s mistakes.

Over the next few years, this perspective will trickle both upwards and downwards. CEOs will be confused by the desire to hide a breach, knowing that the coverup can be worse than the crime. And security professionals will be less and less able to keep saying that one breach can destroy your company in the face of overwhelming evidence to the contrary.

As the understanding spreads, so will data. We’ll see an explosion of ways to talk about issues, ways to report on them and analyze them. In a few years, we’ll see an article titled “Breach Analysis: Read it with your coffee” because daily analysis of breaches will be part of a CIO’s job.

Thanks to the Office of Inadequate Security for the pointer.

Breach Harm: Should Arizona be required to notify?

adam — Tue, 28 Jun 2011 14:49:16 +0000

Over at the Office of Inadequate Security, Pogo was writing about the Lulzsec hacking of Arizona State Police. Her article is “A breach that crosses the line?”

I’ve been blogging for years about the dangers of breaches. I am concerned about dissidents who might be jailed or killed for their political views, abortion doctors whose lives are endangered from fringe elements, women who have tried to escape abusive spouses, porn actors whose families may be harassed by the publication of their names and addresses, confidential informants and law enforcement officers, and immigrants whose personal information was illegally revealed to law enforcement and to media by the actions of Utah state employees. All of those people have been put at risk of physical harm as a result of data breaches.

To date, what we know was taken from Arizona’s (apparently) insufficiently secured systems was names and addresses of people who have good reason to think they’re in danger from the release of that information.

I want to talk about four major risks here: The risk of harm, the risk of attributing all that risk to Lulzsec, the incentive to cover-up, and the risk of believing our analyses are complete.

The first risk, the risk of harm, Pogo covers fairly well. I have a cousin who works in a correctional facility. Their house, their phone, their cable, all these things are listed in the wife’s name, and I understand the fear of knowing that a real criminal thinks you’re at fault and knows where your family lives. I bring this up because it’s my family too, and that’s important because I’m about to discuss the apportionment of blame, and want to be clear that I’m doing so with some skin in the game.

The second risk is the risk of attributing all of the responsibility to Lulzsec. Some of the fault here is that of the State of Arizona Department of Public Safety (AZDPS). AZDPS made a decision to collect information. They had a responsibility to protect it. AZDPS also made a decision to store that information in electronic form. AZDPS made a decision to store that electronic information in an internet accessible fashion. AZDPS made decisions about computer security which, in hindsight, may be being reconsidered. However elite the ninjas of Lulzsec may or may not have been, however many lazer-eyed sharks they might have employed, if the information was only stored on paper in a locked room in Arizona, it would have been far more secure. And if Lulzsec could break in, potentially others have already broken in and stolen the data for purposes far more dangerous than embarrassing AZDPS. AZDPS is not unique in this set of choices. The organization reaps lots of benefits in putting the data online. Many of those benefits, such as speed and efficiency, are probably shared with employees, customers or citizens. All that said, Lulzsec did increase the risk by making the data widely available to anyone. (They also marginally decreased the risk by making people aware it’s out there, but the net risk is still increased.)

The third risk is the risk of cover-up. AZDPS is one of many organizations that collects information today. Like most of those organizations, AZDPS makes some investments in security to protect the data. I suspect that they make more investments than many others, since they know about the sensitivity of it and the many motivated attackers. Interestingly, their policy states that “Security methods and measures have been integrated into the design, implementation and day-to-day practices of the entire Azdps.gov web portal.” (AZDPS Privacy Policy (as of January 4, 2010, via the WayBack Machine)) which strikes me as a mature statement compared to the common “we follow industry-leading best practices in buying a firewall.” Most organizations that are hacked are not hacked by Lulzsec, and so may choose to cover up. AZDPS should investigate what went wrong, and share their analysis so others can learn from them.

The final risk is the risk of believing our analysis is complete. Much like I pointed out in “How the Epsilon Breach Hurts Consumers,” it’s easy to come to an analysis which misses important elements because the investigators have a defined scope. They are more likely to talk to those close to the system, and thus will be influenced by their perspectives and orientation. By sharing information about the breaches, different perspectives can emerge from a chaotic discussion. This is a perspective deeply influenced by Hayek. Unlike markets, information security lacks a pricing mechanism to help us bring all of the perspectives into a single sharp focus. It’s hard to add security to see what people will pay, and we lack good information about the inputs that led to breaches or other outcomes. Without that information, it’s hard to know what security is cost-effective, or appropriate in light of the duties that an information collector takes on by collecting data.

So to bring this together around those risks, the people whose data was exposed (first risk) were exposed in part because most organizations never issue a good report on what went wrong (the third risk) and so the choices made in collecting and storing data are made in an information vacuum (the second risk).

And so the Arizona DPS should take seriously their public safety mission. They should perform a deep investigation of what went wrong, and they should share it with the citizens of Arizona and people around the world. If they do so, and their counterparts do so, we’ll all be able to learn from each other’s mistakes, and we’ll all be able to, in that hated phrase “do more with less.”

That’s how public entities, operating with data about citizens, should be operating, and in my personal opinion, ought to be required to operate.

Representative Bono-Mack on the Sony Hack

adam — Wed, 11 May 2011 15:11:06 +0000

There’s a very interesting discussion on C-SPAN about the consumer’s right to know about breaches and how the individual is best positioned to decide how to react. “Representative Bono Mack Gives Details on Proposed Data Theft Bill.”

I’m glad to see how the debate is maturing, and how no one bothered with some of the silly arguments we’ve heard in the past.

What does Coviello’s RSA breach letter mean?

adam — Mon, 21 Mar 2011 14:51:25 +0000

After spending a while crowing about the ChoicePoint breach, I decided that laughing about breaches doesn’t help us as much as analyzing them. In the wake of RSA’s recent breach, we should give them time to figure out what happened, and look forward to them fulfilling their commitment to share their experiences.

Right now we don’t know a lot, and this pair of sentences is getting a lot of attention:

Some of that information is specifically related to RSA’s SecurID two-factor authentication products. While at this time we are confident that the information extracted does not enable a successful direct attack on any of our RSA SecurID customers, this information could potentially be used to reduce the effectiveness of a current two-factor authentication implementation as part of a broader attack.

With the exception of RSA and its employees, I may be one of the best positioned people to talk about their protocols, because a long time ago, I reverse engineered their system. And when I did, I discovered that “The protocol used by Security Dynamics has substantial flaws which appear to be exploitable and reduce the security of a system using Security Dynamics software to that of a username and password.” It’s important to note that that’s from a 1996 paper, and the flaws I discovered have been corrected.

I’ve been trying to keep up with the actual facts revealed, and I’ve read a lot of analysis on what happened. In particular, Steve Bellovin’s technical analysis is quite good, and I’d like to add a little nuance and color. Bellovin writes: “Is the risk that the attackers have now learned H? If that’s a problem, H was a bad choice to start with.” In conversations after I wrote my 1996 paper, it was clear to me that John Brainard and his engineering colleagues knew that. (Their marketing department felt differently.) RSA has lots of cryptographers who still know it.

The nuance I’d like to point out is that many prominent cryptographers had reviewed their system before I noticed the key management error. So it’s possible that that lesson leads to the statement that the information could be used. That is, the crypto or implementation, however aware of Kerkhoffs’ Principle, could still contain flaws.

If someone had compromised the database of secrets that enable synchronization, then that would “enable a successful direct attack on” one or more customers. So speculation that that’s the compromise cannot be correct without the CEO of a publicly traded company lying in statements submitted to the SEC. That seems unlikely.

But there’s another layer of nuance, which we can see if we read the advice RSA has given their customers. When I read that list, it becomes apparent that the APT used a variety of social engineering attacks. So it’s possible that amongst what was stolen was a database of contacts who run SecurId deployments. That knowledge “could potentially be used to reduce the effectiveness of a current two-factor authentication implementation as part of a broader attack”

My opinion is that social engineers using the contacts database in some way is more likely than a cryptanalytic attack, and a cryptanalytic attack is more likely than a compromise of a secrets database. But we don’t know. Speculating like mad isn’t helping. Maybe I shouldn’t even post this, but the leaps of logic out there provoke some skeptical thinking.

[update: some great comments coming in, don't skip them.]

[Update 2: Between Nicko's comment on the new letter, and Paul Kocher's analysis in his Threatpost podcast I'm not sure that this analysis is still valid.]

Another critique of Ponemon’s method for estimating ‘cost of data breach’

Russell — Wed, 26 Jan 2011 01:55:14 +0000

Adam just posted is general critiques of the annual US Cost of Data Breach Study. I agree with his critique about survey methods, but I have more fundamental objections to their methods used to estimate ‘indirect costs’ due to lost customers (‘abnormal churn’) and the cost of replacing them (‘customer acquisition costs’).

A noble effort, but…

Before I start chopping it up, let me say that I think their annual survey is a good effort and it’s positive that they can get sponsorship and also readership for the results. I think they have good intentions and try to give a fair, balanced, and reasonable estimate. Our field would be better off if there were similar data gathering efforts in other areas of InfoSec. I also don’t believe that any of the errors are due to intentions to ‘spin’ or mislead. It looks like they didn’t have sufficient expertise on their team in business finance, marketing analysis, and economics.

But I see some serious problems with their methods. This is a big deal since ‘indirect costs’ make up a majority (68%) of their estimate of total costs.

Problem #1: A fog of buzz words

If their data and analysis were bulletproof, then maybe we could forgive sloppy use of terms. But it isn’t bulletproof and their use of terms is actually misleading because it gives the impression that the method is well established and well executed when it really isn’t. Furthermore, it’s a sign that whoever is doing this part of the analysis doesn’t know what they are talking about. Examples:

“The survey design relied upon a shadow costing method used in applied economic research.” (p. 36) — There is no such method as ‘shadow costing’. Do a web search if you doubt me. The only examples of ‘shadow costing’ are economic studies that use ‘shadow prices‘ multiplied by input quantities to derive ‘shadow costs’ for certain manufacturing or service process. Having just completed a Mathematical Economics class last semester, I can assure you that the Ponemon method has nothing to do with shadow prices or shadow costs.
“Utilizing activity-based costing…” (p. 3) and “The diagram below illustrates the activity-based costing schema…” (p. 36) — They do not use activity-based costing. Activity-based costing (ABC) is a way of allocating overhead costs by measuring some ‘activity’ in operations that are thought to drive those overhead costs. The ratio of ‘activity’ for each business unit to the total is used to allocate the overhead cost to that business unit (or product line or customer segment or what ever). You can read more about ABC here and here. How big a flub is this? Big. It’s like labeling a signature-based AV software as an ‘expert system’. Anyone who uttered such a statement would immediately be dismissed by security experts. What ever the Ponemon method is, it’s not ABC. Just because costs are related to activities does not mean you are doing activity-based costing.
“…most companies experience opportunity costs associated with a breach incident...” — No, these aren’t ‘opportunity costs’. The term ‘opportunity cost‘ has very specific meaning in microeconomics. Basically, an opportunity cost is the cost of giving up your next-best alternative when you make a decision. They misuse the term here to refer to costs that expected in the future, i.e. beyond the historical frame of the breach incident and post-incident remediation. Their use of the term here is just sloppy and could easily mislead someone who doesn’t know economics.

Problem #2: Mixing accounting costs with economic costs

This is a subtle but fundamental problem, and it’s why people get degrees in accounting and economics. Accounting costs are those that appear in a financial statement somewhere and follow specific costing rules, e.g. GAAP. They have already occurred (historical costs) or they are forecasted to occur (pro forma costs). In the Ponemon method, they list four categories of accounting costs:

Detection or discovery
Escalation
Notification
Ex-post response

Now it’s probably true that most organizations do not have explicit accounts for these costs so they have to be derived from other accounting costs. But it’s pretty easy to slice and dice accounting data (i.e. general ledger entries) to get decent estimates of these costs. It’s also possible estimate costs by using per-resource costs (labor cost per hour) multiplied by the usage of those resources (hours to resolve an incident). In the Ponemon survey, they ask their point-of-contact for their estimate of these costs. That’s probably OK, given that the point-of-contact is a privacy/security person directly involved in the incident.

But then they mix in future economic costs (what they mislabel as ‘opportunity costs’):

Turnover intentions of existing customers
Diminished new customer acquisition

(Leave aside for a moment that they are asking about “intentions” of customers to defect. Adam discussed this in his post.)

These are both economic costs. (See this slide deck, slide #2. The wikipedia article on this topic is not good.) Basically, economic costs are all in the future. There is no such thing as ‘historical economic costs’. Only cash flows count in economic costs — no ‘intangibles’, no depreciation, no ‘good will’. Those can only be included in the form of future cash flows discounted for time and risk (and uncertainty). Economic costs include opportunity costs (see above), which are the cash flows associated with the next-best alternative. However, opportunity costs will never appear on a financial statement, now or in any future. Economic costs are incurred when the commitment is made, not when they are recognized in the accounting system.

Most important: All cash flows are discounted for the time value of money and the riskiness of the cash flow. This feature is essential for rational deicsion-making over time and over risky alternatives, but it also guarantees that no estimate of economic costs will ever equal the corresponding accounting costs because accounting systems to not adjust for the time value of money or risk. Finally, a full estimate of economic costs includes the present value of ‘real options’ and should be adjusted for risk (i.e. derating by using the costs of insuring against unexpected/extreme events, cost of lowered credit rating, etc.).

The element of their method that specifically invokes economic cost is ‘lifetime value of customers’ (LTV). In their method, the cost associated with lost customers is estimated by multiplying the % of breached customer records that will defect (‘abnormal customer churn rate’) multiplied by LTV. (LTV originated in direct marketing in the 1980s. Wikipedia has a decent article that explains it. Here’s a good demonstration.) LTV is a net present value, discounted by the cost of capital associated with the riskiness of the cash flow. It’s an economic profit, not an accounting profit.

Putting this all together, either the costing method should use only accounting costs (historical and/or pro forma) or it should only use economic costs (prospective discounted cash flows, risk-adjusted). Otherwise, the numbers don’t add up, literally.

Problem #3 : Decision polices matter

(i.e cheap short-sighted bastards can have lower costs than prudent socially responsible managers)
Here’s another problem with mixing accounting costs with economic costs. Let me illustrate this with a story. There are two companies — Cheap Bastard, Inc. (CBI) and Nice Guys R Us (NGRS). CBI has decision policies to spend as little as possible on InfoSec, especially in incident detection and incident response. They push all liability onto their customers, suppliers, and contractors. They systematically downplay evidence of breaches, and downplay the severity or costs of breaches. They avoid forensic analysis if they can get away with it. And so on.

In contrast, NGRS puts a lot of attention on pro-active security and detection and goes out of it’s way to mitigate the costs of insecurity on it’s ecosystem. They are especially eager to spend money post-breach to restore public trust and to learn from the event to get at root causes.

How would CBI and NGRS show up in the Ponemon survey? My guess is that NGRS would have cost per record of 2X or 3X greater than CBI, primarily because CBI will have much lower accounting costs (as covered by the survey) by decision policy. It’s also likely because CBI can ‘safely’ ignore the probable future costs of their rapacious behavior (i.e. class action lawsuits, regulatory penalties, even larger security breaches). I put ‘safely’ in quotes because such corporate behavior is only safe until you get caught or get screwed.

I don’t see a way around this if you only use accounting costs. If you use a sufficiently broad framework for economic costs, then you stand a chance of understanding the ‘total costs’ that exposes the riskiness of CBI’s decision policy.

Problem #4 : Do respondents really know anything about customer LTV or ‘churn’ intentions?

I’m surprised that no one has brought this up before. As someone who has calculated and published LTV metrics for a business unit, I can say with some confidence that almost no one who didn’t read those reports would have been able to guess the LTV of customers, including accounting people who knew about the cost and revenue categories but never put them together into LTV. SWAGs (as in def. 5) were could be off by an order of magnitude.

My opinion is that asking a privacy/security/incident response specialist to estimate LTV is fundamentally flawed unless that person has access to their own company’s management reports that include LTV. It might be possible to elicit useful estimates from them after their estimates are calibrated through exercises, including exercises that estimate the weighted average cost of capital, average lifetime of a customer, acquisition costs, retention costs, etc.

Same goes for ‘churn’ rate (percentage of customers who leave because their records were breached). To estimate ‘abnormal churn’ due to the breach, the point-of-contact would need to know something about ‘normal churn rate’ and, as Adam says in his post, the variability of churn rate. If churn rate varies widely from year to year, then a small increase in churn due to a data breach would be washed out by the other factors driving variability.

It would be much more useful to find out if the company increased their marketing budget as a direct consequence of a given data breach. If they did, then this would be credible evidence that the number and value of lost customers was great enough for the company to change it’s spending decisions.

Problem #5: Leaving out significant cost categories

This problem may be bigger than all the others combined. If they left out major categories of cost, then their estimate of cost per breach could be off by 50% or more.

To answer this question, you first need to decide between estimating accounting costs vs. estimating economic costs. Every economist and every B-school professor will advise you to estimate economic costs. It may be useful to analyze historical accounting costs as a way to estimate future economic costs, but that is a separate exercise.

As an economic cost analysis, it might be best to frame the decisions this way:

Given a breach of customer data of size X records (same size as historical breach), how would the firm’s economic costs change vs. no breach?

I’ll point out two categories of cost that this analysis would include that are currently excluded in the Ponemon survey.

Cost of additional spending on security
If a firm incurs incremental spending on security due to a breach, shouldn’t those costs be included in the ‘cost of a data breach’? This goes back to my story about fictional companies CBI and NGRU, above. If CBI is likely to spend more to fix their crappy security in the future if they experience a large breach today, then they will be forced to ‘pay the piper’ in economic terms and their decision policy of spending as little as possible won’t help them avoid the full costs of data breaches. This will also capture the cost of half-measures, since trying to get off cheap on the security upgrade will still show up as higher expected costs for future breaches.

Of course, this raises sensitive political issues with respondents to the survey. They may be reluctant to answer questions about actual spending on improvements to security or, even more, to speculate about possible future costs. For example, what if a company’s outsourcing strategy is hopelessly insecure and the firm is forced to reverse those decisions and insource those processes. What if a company is forced to exit a line of business because the security risks and costs are too high? What if a data breach leads to process changes that diminish or eliminate their key competitive advantage?

Factoring these costs could increase the cost of a breach by 0.5X to 10X. I would make it much harder to do cross-company and cross-industry comparisons. But wouldn’t it make the true economic costs of data breaches more relevant to management decision-making?

Social costs
The other ‘elephant in the room’ is the cost to consumers or employees for having their private data breached. Add these all up and you get ‘social cost’, or appropriately adjusted, ’social welfare‘. I understand that the Ponemon survey is estimating only costs that are incurred by a single organization that experiences the breach, not by any other stakeholders in that firm’s ecosystem.

There are plenty of studies on the direct and indirect costs of identity theft and the perceived costs of breaches of privacy. Drawing on these studies might make it possible to estimate the social cost of a breach. Then the estimation question is “What portion of social cost will the firm have to bear?”

The answer to this question depends on firm policy (see CBI and NGRU story, above), the legal system, the regulatory system, and also the legislative system. Basically, if a firm or collection of firms consistently and egregiously impose large costs on their customers or employees, then one or more of these other social/political mechanisms might kick in to impose an ‘equity remedy’.

The most immediate remedy, from the American firm’s point of view, is a class action lawsuit. Of course, estimating the likelihood of getting sued, the damages sought, and the likelihood of losing such a suit is risky business . But just because it’s difficult to estimate with precision should it be excluded?

Again, including this cost category might increase the cost per breach by 2X to 10X in some cases. But it might also shift management attention to crucial questions such as “What is our role in our value network regarding information security and risk?”

Problem #6: Unsupportable inferences

Given that their survey method is not statistically robust (see p. 33), they do not have sufficient confidence to make the inferences summarized on p. 28. I won’t go through these one by one, but anyone who has done statistical sampling and inference knows how sample size and variability affect confidence intervals. If the difference in question does not exceed the confidence interval, then you cannot support the inference from the data. The best they can do is say, “we say X% of companies report Y, vs. A% of companies reporting B. This suggests that…”. All such suggestions would then need to be subjected to additional tests.

Problem #7: Is ‘Cost per Record’ the best measure?

It appears that only a few costs truly vary by the number of records breached. These include costs of ‘notification’ and some of ‘ex-post costs’. But ‘discovery’, ‘escalation’, and ‘indirect costs’ are mostly independent of size of breach measured by number of records. Some might be fixed costs that are independent of the size of breach. Some might be increasing functions, perhaps relative to some threshold of that defines ‘big’ or ‘material’ (to use the accountant’s term).

This problem may not be significant compared to the others. I just think it needs to be justified by comparing it to alternative formulations.

Summary

The summary result ($204 per record) reported in the Ponemon survey is not reliable. No one should rely on the absolute value of this measure. Some of the relative measures might be informative, especially the direct costs that the point-of-contact respondents are qualified to answer. Trend analysis might be somewhat informative. None of the recommendations reported (i.e. the value of hiring outside IT security consultants) can be supported by statistically significant inferences.

To get a reliable measure of Cost of a Data Breach will require substantial revision to the survey instrument, sampling method, analysis methods, and reliability controls. I’m guessing that this is beyond the appetite of PGP, the sponsor of the survey.

Call to action

What would it take to launch a Version 2.0 of this study with more robust methods and a stronger team of experts to execute it and analyze the results? There’s no mystery about how to do Version 2.0. The only obstical is resources and commitment.

“Cost of a Near-miss Data Breach“>

Visualization for Gunnar’s “Heartland Revisited”

alex — Tue, 16 Nov 2010 14:31:27 +0000

You may have heard me say in the past that one of the more interesting aspects of security breaches, for me at least, is the concept of reputation damage. Maybe that’s because I heard so many sales tactics tied to defacement in the 90′s, maybe because it’s so hard to actually quantify brand equity and impact to brand equity from a data breach.

Either way, Gunnar’s post on “Heartland Revisited” is great analysis. I’d like to point you there, and add two things. First, its my personal pet hypothesis that “reputation” only really matters in B2B cases where there are individuals who are responsible for choosing the breached vendor. Nobody wants to be the guy that “hired those screwups”, and if you are, you pretty much automatically have to consider firing them.

Second, I thought I’d add a bit of a visualization, tracking the stock prices from just before the incident until now. By clicking on the image below to see the full graph, you’ll see that Heartland had been a leader among those four (at least using this particular metric), dropped significantly with the data breach and, as per Gunnar’s analysis, is now still trying to recover (be that from the breach or other factors or what have you – not making any inference there).

Again, I’m not trying to draw any conclusions from this, saying “See! Reputation matters!” or even claiming that Heartland is an exception to Betsy Nichols excellent work, but I do think this is interesting, even if just as a casual observation.