Using security metrics to measure human awareness

Free tools offer security practitioners a way to measure the effectiveness of awareness programs

It's been said that security is hard to measure. Producing measurable results around a lack of problems or incidents is challenging. But the field of security metrics has evolved considerably in recent years, giving security managers more resources to make the case for investing in security programs and technologies.

Now the SANS Institute, through their Securing the Human Program, is offering a set of free metric tools designed to give security leaders the ability to track and measure the impact of their own security awareness programs.

[Security Metrics: Critical Issues]

According to Lance Spitzner, training director for the program, the tools can be used to improve training, demonstrate return on investment, or compare an organization's human risk to other organizations in an industry. All resources are free, developed by the community for the community, said Spitzner.

The tools include:

Metrics Matrix -- A spreadsheet that identifies and documents different options for measuring a security awareness program. It includes metrics for both measuring impact (change in behavior) and for tracking compliance.

Measuring Human Risk Survey -- The newest addition to the tools that is still in development, the twenty-five question survey helps determine the human risk in an organization. Each question and its respective answers have different levels of risk associated with them. Depending on how employees respond, answers can be totaled to determine a quantitative value of your human risk.

Phishing Assessments Planning Package -- Phishing assessments are not only a simple and effective way to measure the impact of your awareness program, but a very powerful way to reinforce key training concepts. This package helps you step by step plan, build and implement a successful phishing assessment program, including several templates, said Spitzner.

CSO spoke with Spitzner about using the metric tools.

CSO: What was the mission in creating these metric-gathering tools?

Spitzner: The tools were developed out of need by the security awareness community. I run a private mail list of about 200 professionals who are all involved in, or lead the security awareness program for their organization. People post what they are looking for, and then, we as a group develop resources that help solve that problem.

One of the first challenges we solved was creating the Security Awareness Maturity Model that helps identify how mature your awareness program is and then how you want to build on that. As a group we then developed the Security Awareness Roadmap that explains in detail how to reach each maturity level. There was a repeated request and need for metrics.

What are the challenges of using security awareness metrics?

As always there are several challenges with metrics, security awareness metrics are no different. A couple of points to keep in mind:

  • Ultimately, metrics are a tool used to measure the effectiveness of your security awareness program and how to improve it. Sometimes organizations get so caught up in their metrics that the metrics become more important then the program itself, they forget about what their ultimate goal is. As such the best approach is to focus only on a few, very good metrics.
  • Unfortunately good metrics are hard. They have to be easy to measure (preferably automated), they have to be measured consistently (in other words even if different people measure they get the same result) and they have to something you can take action on. Classic example of a bad metric is the top ten most infected countries. What value does that metric have? What action are you supposed to take based on that?

This is one of the reasons we developed the security awareness metrics matrix, it has a list of over 15 metrics organizations can choose from, depending on which metric has the most value to them.

What is different about awareness metrics from other types of security metrics?

You are attempting to measure the human element, specifically peoples' behaviors and awareness. Technology is bits and bytes, which can be easier to measure (number of attacks detected, number of ports scans blocked by the firewall, etc). The other challenge is root cause analysis. Quite often incidents are caused by humans but organizations do not realize it because they never do a root cause analysis.

The classic example is infected systems. If your security team did a root cause analysis of infected systems, they would most likely discover that the vast majority of infections are not a technical issue but a human issue. Unfortunately many organizations fail to do any type of root cause analysis of incidents, thus hiding the fact that the human is most often the issue, not technology.

Why are you passionate about this topic?

Because I passionately believe this is where we can have the greatest impact. In the past 15 years I've been in information security our community has focused almost entirely on using technology to secure technology, and we have gotten very good at it. As a result, most operating systems have become very difficult to hack into, except that we have done nothing to secure the human element, what I like to call the HumanOS. They HumanOS has never been trained, as result they have no firewall, they have all their services on by default, there is no patching. All the classic mistakes we made 15 years ago are still happening with people today. This is why the human has become the primary attack vector. By investing some basic resources into people you can have a tremendous impact in reducing risk, just like we have in other operating systems today.

What types of folks do you envision using the SANS tools and what kind of benefit do you hope it will provide?

Absolutely any organization can benefit from our free resources, and not just organizations by ordinary people in their personal lives. Think about it, about 70-80 percent of any security awareness program applies both to the organization and employees personal life; topics such as email, mobile devices, social networking or passwords. Our approach is not to just make people aware and change their behaviors at work, but change those same behaviors at home as they face the same risks. As such, security becomes part of their DNA.

Feedback on how to improve the awareness metric resources can be sent to community@securingthehuman.org.

Show Comments