Surfer v. Vending Machine Repairman: Conditional Probability [guest post]

[#1]

This guest-post is brilliant.

Simply put, it's better than my original piece, and a fantastic-must-read if you're in enterprise security and are getting senior executives asking about the 'big, bad, current events' that are taking place around them. Heath's masterful analysis, related to real-world enterprise experience is something I recommend you all read, print, and read again later when you're having "one of those days"...

------------------------------------------------------------------------

The key point of Raf’s last post, Sharks v. Vending Machines was that we should focus on our basic defenses, before becoming distracted by the most recent cyber-monster. Raf supported this conclusion by invoking the parable of the vending machine and the shark presented by Freakonomics. The parable of the vending machine and the shark warns us not to let the vividness of the shark attack (APTs) increase our perception of the likelihood of the shark attack. We are more likely to be killed by a vending machine (normal, everyday stuff like patching).

Let’s accept for now that the pursuit of foundational security is good. The parable above begs the next question: when deciding how to invest the budget, doesn’t it matter whether or not I’m a surfer or a vending machine repairman? How do we make tradeoffs? This is also a question Raf highlighted when addressing “conditional probability,” or prior probability. This struck me as a nice launching point. What tools do we have in our field to help us determine if we are a surfer, a vending machine repairman, or just a guy looking for peanut M&Ms? Then, how should we practically use the information that is available? How do we bring these things to life in real-time analysis?

First, how do we determine if we are a likely target in the short-term? When I get a question from senior management on the latest external threat, the first thing I do is determine what analysis is already available to tell the story. I pull the DBIR to remind myself of where my organization and the external threat meet in the breach universe. This helps me determine if I should expect to be in a special sub-population. I then look at the Data Loss Database , Ponemon’s Cost of a Data Breach report, and the HITRUST Healthcare Data Breach Trends report. Internally, I scan our security incident database and our analytic repository. I give a phone call to security engineers and security incident managers to make sure I have all the current data available. I explicitly ask myself, “what’s new about this current event, and should any new evidence change my prior judgments?”

I do try to incorporate the concept of prior probability into our analysis any way I can and still make my deadline. For example, the HITRUST breach report tells me that of the intentional breaches they studied, 11 percent were from corporate espionage. The problem is that these statistics are separated by two pages of distracting anecdotes from the prior probability: only 11 percent of the studied breaches were intentional. So 11 percent of the 11 percent that were intentional is just over 1 percent of all breaches they studied.  Prior probability would dictate that we have to be careful about how we characterize the risk of phishing based on this report.  We can further nuance our understanding of the phishing threat. The HITRUST study is focused on breaches that involve over 500 individuals. The truth is that we probably have little insight from this source into the focused phishing attacks that need the information of only one highly placed manager. This is a case were the value of the data can matter as much as the volume. We would characterize our source accordingly.

I use all the information I have in a matter of hours to start crafting a multi-source analytic statement about our attractiveness to an external threat. So what do we do with this conclusion?

We do NOT tell the board about the vending machine versus the shark parable. We don’t scoff at any supposed sheep-like behavior. We don’t bemoan all that’s wrong with the media these days. What we do is milk that puppy. We milk their interest, but we milk responsibly. These are some of the few situations where they are coming to us! We use their interest and respond as quickly as possible in a way that addresses their concern while directing their energy in the most productive direction. Here are a couple examples.

The LinkedIn password breach in June got many senior managers’ attention because it was a social media platform that they have in common. Also, the media extensively covered the breach. These anecdotal factors increased their concern beyond where it probably should have been. Initial reporting indicated the breach involved roughly 4 percent of LinkedIn users. Managers were asking questions in many of the meetings my CISO attends about how the event impacted us. I scrambled to get some background data and explain the improbability that the attack directly put our sensitive data at risk.

Then, my CISO said to me, “It’s not only about the possibility of our passwords being leaked, it’s also about phishing. It’s about using all that good social networking data to build context for more targeted phishing attacks.”

Our response to leadership went something like this: “Mr. Senior Manager, although it is unlikely that our sensitive data will be directly impacted from the compromised passwords at LinkedIn, it is sound advice to encourage managers to reset their LinkedIn password. Also, information on LinkedIn could be used to develop better phishing attacks with refined social networking data. We highlighted the growing trend of targeted phishing in regards to a high-profile attack at RSA last year. We have a tool that simulates a phishing attack, and would like to launch a focused phishing exercise to raise our resiliency. Would you endorse our campaign?” Check.

What about the RSA breach? It was a shark tale of APT proportions. This was a strategic attack most likely designed to collect government and other proprietary information. The attackers probably required multiple pieces of information to exploit the vulnerability. My organization was not a likely target. Our real time message to management’s immediate concern went something like this: “Mr. Senior Manager, the compromise at RSA is remarkable in its target selection and political implications. Until we learn more about their breach we must assume that our two-factor authentication is probably less secure. RSA representatives assure us that compromising their solution would take additional pieces of information. We are working directly with them to assess any likely impact to us, but so far their solution remains an industry standard. We are in the midst of a policy review of our deployment of two-factor authentication for remote users. We believe that we should be more focused in our use of this technology, better manage our PINs and passwords for some users, and that RSA tokens are probably inappropriate as a single fix to other authentication issues. We rewrote our policy in a way that will provide us adequate protection, also reduce our dependence on this solution, and strengthen other forms of authentication. Would you like to expedite the approval of this policy revision?” Check.

I have a lingering sense we can push the idea of assessing our conditional probability much further in real time analysis.  One approach would be to invest in tools in advance to use quickly during an emerging situation. I would love an interactive, DBIR-like database. We also have to continue to push the shark and vending machine parable deeper into our internal risk assessments (intermediate-term analysis) on the threat event frequency side. The FAIR framework’s “contact” and “action” factors for threats address this very well. I understand the contact factor is like the potential of a shark coming in contact with a surfer in water near Seal Rock in South Africa vs. a vending machine repairman. Finally, information sharing is vital. We have to find credible and low-cost ways to share our experience.

These are not stories about pointing out to our managers the statistical insignificance of their concerns. This is a story about judo. We accept interest in any external threat and we make sure energy is directed to a core issue that really matters. The very fact that we took the time to respond thoroughly and quickly helps our cause. The more interest in information security the better, whether it started from a shark tale or a trip to the vending machine. It’s how we respond that matters.

---- About the guest-blogger

“Heath Nieddu is a Senior Information Security Analyst at Providence Health & Services, where he tries to keep his CISO and Director entertained with policy-relevant analysis and metrics. His comments are his own and do not represent the views of his employer.” 

---------------------------------

If you've got something to share with the world and would like to guest-blog here, please feel free to reach out and let me know. I'm happy to have you join in on a current topic, or start your own.