Cybersecurity has become a full-fledged item on the agenda of organizations worldwide and everyone is concerned about information security, data breaches, malware, and cyberattacks. But when it comes to improving an organization’s security, how do you actually measure it? How can you quantify the current state of security and track improvements? Any worthwhile cybersecurity program needs carefully defined performance indicators that provide meaningful and comparable values – your cybersecurity metrics. This article shows how to define useful cybersecurity metrics, examines the benefits they can bring, and suggests a starter set of metrics for web application security.

Measuring Your Organization’s Security Posture

The Chief Information Security Officer, or CISO, is fast becoming an obligatory and important role in large organizations, often reporting directly to the CEO rather than the CSO. One of a CISO’s main tasks is providing C-suite management with information about the current cybersecurity status, trends, and requirements to support decision making. To deliver clear and actionable reports, you need meaningful performance indicators that can be compared across various dimensions. Cybersecurity metrics provide such key performance indicators (KPIs) for the organization’s cybersecurity posture.

Every organization has its own unique operational and technical needs, so there is no one-size-fits-all set of cybersecurity metrics. A security metrics program to develop and maintain a suitable set of indicators should be a fundamental part of any cybersecurity program and – more broadly – any risk management program that includes cybersecurity.

The Benefits of Cybersecurity Metrics

Preparing a tailored set of cybersecurity metrics requires a lot of time and effort, so let’s start by looking at the advantages they can bring. Because they are high-level indicators, cybersecurity metrics are most useful in the strategic role. Here’s how they can help in the boardroom: 

• Communicate security performance: Selecting clear and informative metrics helps to demonstrate the current cybersecurity status and developments, ensuring that business leaders understand the significance and implications of cybersecurity processes.
• Support effective decision making: By providing concrete talking points and reference values, metrics help security professionals inform and influence decision-making processes.
• Justify security resource allocation requests: Armed with real numbers, you can present a clear and compelling case when additional resources are needed in the IT budget to improve cybersecurity.
• Demonstrate compliance: Specific cybersecurity controls are increasingly mandated by corporate policies, industry standards, and regulatory requirements. Metrics can help to monitor and demonstrate cybersecurity compliance and assist with risk management.
• Compare performance to competitors: With the right set of metrics, you can benchmark your organization against its peers to compare cybersecurity spending and results and gain a competitive edge.

Apart from the strategic role, metrics also bring practical benefits on the operational and organizational level:

• Diagnose problems: By monitoring vital metrics at regular intervals, you can quickly identify and address global issues that might not be apparent based on just operational data.
• Measure the effectiveness of security controls: Cybersecurity controls and initiatives are only as good as their results, and with the right metrics, you can monitor performance and follow up as necessary.
• Drive internal performance, awareness, and engagement: Metrics can also be used to set specific goals that focus the security team’s attention on important areas and increase diligence.

7 Key Features of a Good Cybersecurity Metric

With so many information sources and data points already available and so many more easily added, selecting the right metrics is vital to ensure they are relevant and actionable. Here are 7 key requirements for any effective security metric:

1. Necessary: If your metric does not correspond to a business requirement, you don’t need it. Metrics should exist only to support decision making.
2. Relevant: Each metric should provide clear and actionable information that is relevant to a specific person or role.
3. Quantifiable: Metrics should provide specific values, not qualitative descriptions. This lets you directly compare results across periods and dimensions.
4. Consistently measured: To ensure reliable and comparable results, each metric should always be measured in the same way.
5. Repeatable: Only repeatable information security processes should be measured.
6. Economical: Gathering and processing the raw data should require minimal time and effort.
7. Expressed in a specific unit: The unit of measure is as important as the value itself. Multiple units are even better, as they allow analysis across multiple dimensions.

Developing Cybersecurity Metrics for Web Applications

When it comes to web application security, your core cybersecurity metrics will likely focus on vulnerabilities and availability. Chances are you already have a lot of data sources for both these areas, including logs, dashboards, and reports. For maximum benefit, data gathering should be automated as much as possible, and leading vulnerability scanners provide enterprise-class visibility and reporting features to help you. Here’s a starter set of cybersecurity metrics for web applications:

• The total number of reported cybersecurity incidents: This is the basic indicator of the overall state of cybersecurity during a certain period. To keep values directly comparable across periods, you may need to divide them by a variable factor, for example providing the total number of incidents per application or installation. Adding severity information will allow you to drill deeper into the data.
• Changes in the number of reported cybersecurity incidents: Expressed as a percentage, this metric gives a quick overview of cyberthreat developments across specific dimensions. You can use this to compare periods, business units, or applications, and again you can add a severity classification to provide additional intelligence.
• Time to identify: This is the classic vulnerability management metric, usually known as the Mean Time to Identify (MTTI). Of course, this value should be as low as possible, and regular vulnerability scans can help bring this down.
• Time to resolve: Alongside MTTI, Mean Time To Contain (MTTC) this is another mainstay of cybersecurity KPIs, and one of the most important metrics. It reflects the incident response time and patching effectiveness. Since major attacks on organizations usually exploit well-known but unpatched vulnerabilities, minimizing this value can help you avoid data breaches and reduce your overall cyberrisk.
• Cost per incident: This is probably the hardest value to calculate but also the most important piece of board-level intelligence. Apart from the direct costs of incident remediation, recovery, and investigation, you will also need to include indirect costs, such as the cost of downtime and lost business opportunities.
• Application uptime: While uptime can be affected by operational factors as well as cybersecurity incidents, it provides a solid measure of overall application availability and resilience. Just as importantly, every minute of downtime brings tangible losses, so this metric has a crucial financial aspect as well.

Each of these indicators should be customized to suit your environment and business needs and can be extended across various dimensions for more complex analysis. For example, you might take the total number of incidents per application per month and extend it by adding “per severity” or “per location”. Of course, beyond these core metrics, you will also need to add your own to build a set of cybersecurity performance indicators that provides the best results for your organization.

Using Invicti’s Reporting Features in Cybersecurity Metrics

To measure web application security, you need to start with detailed and reliable vulnerability data, so let’s see how Invicti’s scan reports can be used for this purpose. In addition to providing built-in reports and report templates, Invicti Enterprise can also generate customizable statistical reports to provide data for cybersecurity metrics. After specifying the required time period, website group, vulnerability classes, and other parameters, you can generate custom reports including:

• Issues per Period
• Vulnerable Website Groups per Period
• Vulnerable Websites per Period
• Issue Trend per Period

Depending on your requirements, you might use these results as standalone metrics or incorporate them into more general cybersecurity reports. By combining custom reports with with issue resolution data from your vulnerability management processes, you can get a complete picture of your vulnerability status in terms of numbers, severities, trends, and time to fix, obtaining much of the essential information for your web security metrics program.