Employees Say Company Left Data Vulnerable

by adam on October 7, 2014

There’s a recurring theme in data breach stories:

The risks were clear to computer experts inside $organization: The organization, they warned for years, might be easy prey for hackers.

But despite alarms as far back as 2008, $organization was slow to raise its defenses, according to former employees.

The particular quote is from “Ex-Employees Say Home Depot Left Data Vulnerable,” but you can find similar statements about healthcare.gov, Target, and most other breaches. It’s worth taking apart these claims a little bit, and asking what we can do about them.

This is a longish blog post, so the summary is: these claims are true, irrelevant, and a distraction from engineering more secure organizations.

I told you so?

First, these claims are true. Doubtless, in every organization of any size, there were people advocating all sorts of improvements, not all of which were funded. Employees who weren’t successful at driving effective change complain that “when they sought new software and training, managers came back with the same response: ‘We sell hammers.’” The “I told you so” isn’t limited to employees, there’s a long list of experts who are willing to wax philisophic about the mote in their neighbors eyes. This often comes in the form of “of course you should have done X.” For example, the Home Depot article includes a quote from Gartner, “Scanning is the easiest part of compliance…There are a lot of services that do this. They hardly cost any money.” I’ll get to that claim later in this article. First, let’s consider the budget items actually enumerated in the article.

Potential Spending

In the New York Times article on Home Depot, I see at least four programs listed as if they’re trivial, cheap, and would have prevented the breach:

  1. Anti-virus
  2. Threat intelligence
  3. Continuous (network) anomaly detection
  4. Vulnerability scanning

Let’s discuss each in turn.

(1) The claims that even modern, updated anti-virus is trivially bypassed by malware employed by criminals are so common I’m not going to look for a link.

(2) Threat intelligence (and “sharing”) usually means a feed of “observables” or “indicators of compromise.” These usually include hashes of files dropped by intruders, IP addresses and domain names for the “command and control” servers or fake emails which are sent, either containing an exploit, a trojan horse, or a phishing link. This can be useful if your attackers don’t bother to change such things between attacks. The current state of these feeds and their use is such that many attackers don’t really bother to make such changes.
(See also my previous comments on “Don’t share, publish:” we spend so much time on rules for sharing that we don’t share.) However, before saying “everyone should sign up for such services,” “they’ll be a silver bullet,” we should consider what the attackers will do, which is to buy the polymorphism services that more common malware has been using for years. So it is unlikely that threat intelligence would prevent this breach.

(3) Continuous anomaly detection generally only works if you have excellent change management processes, careful network segmentation, and a mostly static business environment. In almost any real network, the level of alarms from such systems are high, and the value of the alarms, incredibly low. (This is a result of the organizations making the systems not wanting to be accused of negligence because their system didn’t “sound the alarm,” and so they alarm on everything.) Most organizations who field such things end up ignoring the alarms, dropping the maintenance contracts, and leaving the systems in place to satisfy the PCI auditors.

(4) Vulnerability scanning may be cheap, but like anomaly detectors, they are motivated to “sound the alarm” on everything. Most alarms are not push-button remediation. Even if that feature is offered, there’s a need to test the remediation to see if it breaks anything, to queue it in the aforementioned change management, and to work across some team boundary so the operations team takes action. None of which falls under the rhetoric of “hardly cost any money.”

The Key Question: How to do better?

Any organization exists to deliver something, and usually that something is not cyber security. In Home Depot’s case, they exist to deliver building supplies at low cost. So how should Home Depot’s management make decisions about where to invest?


Security decisions are like a lot of other decisions in business. There’s insufficient information, people who may be acting deceitful, and the stakes are high. But unlike a lot of other decisions, figuring out if you made the right one is hard. Managers make a lot of decisions, and the relationship between those decisions and the security outcomes is hard to understand.

The trouble is, in security, we like to hide our decisions, and hide the outcomes of our decisions. As a result, we never learn. And employees keep saying “I told you so” about controls that may or may not help. As I said at BSides Las Vegas, companies are already paying the PR price, they need to move to talking about what happened.

With that information, we can do better at evaluating controls, and moving from opinions about what works (with the attendant “I told you so”) to evidence about effective investments in security.

2 comments

As is my custom to read Schneier’s security blog whenever I go on the Internet, his latest comments had a link to your blog. Now I have a new security related site to frequent.

As for your comments, absolutely spot-on! I just left a position where I warned and begged and pleaded for years to implement policy changes and hardware upgrades to protect the network that I was given charge over. After 10 years, I gave up. I am much happier now. 🙂

The only concern I retain for that place, is one that knows, that if someone really goes after them, they won’t even know it.

It is my opinion that the greatest enemy of secure networks… is management.

Live long and prosper.

by Arthur Dent on November 5, 2014 at 8:20 pm. Reply #

There are two primary ways to understand and invest in cyber risk reduction: measure risk and manage uncertainty.

For measuring risk in order to manage investments in controls (people, process, and technology), look first to OpenGroup FAIR and two dominant control categories: avoidance controls and response controls. For the four that NYT listed (in your post above), I see threat intelligence as a response control (although you and the NYT make the mistake of seeing it as a protective control, which it is not). Threat intelligence, especially via the ISACs and ISAOs is to reduce the damage (sic, loss magnitude) to the industry over time. There are other ways to invest in response controls, though, such as threat hunters tied to technology tools (e.g., GRR Rapid Response). Avoidance controls are best built into the existing and future infrastructures with the OWASP Proactive Controls, especially appdevs leveraging AppSensor. If you can create a value chain between all of these people, processes, and tools — then you can balance your investments more wisely.

Managing uncertainty requires heuristics, but not necessarily computational or statistical ones. The rule-of thumb scenario that fits Home Depot, Target, JPM, and most others is that managing third-party integrations is key. For Home Depot, I might recommend two-party integrity and a No-Dark Corners approach to the physical POS terminals. Target and JPM both needed better web application security practices in place — for all of their assets as well as third-party assets. One could only wonder if each of these targeted webapp assets had been scanned by Netsparker Cloud or Tinfoil Security (and not whatever they had been scanned with, i.e., common or well-known PCI DSS ASVs or Gartner-recommend appsec tools). BitSight Technologies also comes to mind here, especially for Target’s nightmare scenario (where knowledge that the breached third-party had recent contact with botnet C2).

For heuristics, it’s primarily about gathering information about your staff and assets (internal intelligence) and your enemy’s weapons (technical intelligence, not threat intelligence) in order to provide satisfactory equations for understanding risks and managing investments.

by Andre Gironda on November 30, 2015 at 4:13 pm. Reply #

Leave your comment

Not published.

If you have one.