Developing Realistic Benchmarks to Track and Improve Cybersecurity Incident Response

It’s clear that for any organization looking to have a mature and resilient cybersecurity posture, detection and response capabilities are a must. However, what’s not clear is what effective detection and response looks like. What are the metrics, KPIs, and goals a department should reach towards? 

Industry guidance is limited largely because of the complex and always-changing nature of any given company’s environment and infrastructure. However, that doesn’t mean an organization can’t establish their own benchmarks to get an understanding of their current cybersecurity posture to improve their overall detection and response performance. 

In this article, we’ll go over what industry guidance is worth following, how to establish detection and response key performance indicators (KPIs) and what process, tools, and solutions can help get you there. 

How to Benchmark Your Detection and Response Capabilities 

To get a better sense of how to measure your detection and response capabilities, Forrester Principal Analyst, Allie Mellen, recently wrote a blog titled “An Actual Complete List of SOC Metrics (And Your Path to DYI),” where she shared the granular metrics that make up detection quality alongside response quality. These include:

Detection Quality 

• Mean Time to Detect 
• Detection Firing Frequency 
• Missed Attacker Activity (Lack of Detection) 
• Detection Accuracy 
• Attacker Dwell Time  

Response Quality 

• Mean Time to Triage 
• Mean Time to Investigate 
• Mean Time to Contain 
• Security or IT tool Pivots  
• Incident Response Process Adherence 
• Frequency of Playbook Execution 
• Frequency of Playbook Change Requests 
• First-call Resolution (Repeat Events) 
• Mean Time to Remediate 

While it’s a lengthy list of metrics, it’s important to focus on the ones that will be most impactful, especially when you’re first starting to benchmark your department’s performance. Among these metrics, there are a few standouts. 

“Mean time to detect (MTTD), mean time to respond (MTTR), false positive rate, detection coverage (or percentage of threats detected), and detection accuracy are most important,” says Josh Armstrong, a manager at Bitdefender’s global security operations center (SOC), adding that tuning out false positives will lead to more quality detections that will ultimately help prevent an attack quicker. 

Optimizing your detection metrics will have downstream effects on improving your response efficacy as well, which is why it’s important to prioritize detection first. 

What Good Detection Quality Looks Like 

It’s difficult to have standardized benchmarks when it comes to metrics like MTTD and MTTR, largely because of the numerous variables and complexity for any given organization. What’s more helpful to organizations, especially those who are beginning their path towards maturity, is to establish benchmarks based on their internal capabilities. 

“Getting a baseline of reference can be done by analyzing historical data, industry reports, and even collaborating with peers in the cybersecurity community,” Armstrong says. “Then you can adjust your benchmarks based on your specific risk profile and current security posture.” 

Improving Your Detection Quality Beyond Average Performance 

The 2023 SANS Survey on incident response is a good representation of what average looks like regarding detection and response. They found that over 50% of organizations surveyed had a MTTD of less than 5 hours, with 25% of organizations having a MTTD of 60 minutes or less.  

However, their other detection metrics weren’t as strong. Across all organizations, the percentage of incident detection dropped, from 88.9% in 2019 to 80.6% in 2023 and the percentage of false positives increased from 67.5% to 77.2% in the same time frame. 

This is a good reference point to start with. If you want to be in the upper quartile of MTTD, aim for a sub-60-minute metric and prioritize reducing your false positive rate. If you find that you’re not quite there yet, put together a plan for improvement (see below) and have monthly check-ins to measure progress. 

How Your Department Can Make Substantial Improvements to Detection Metrics 

To improve on your MTTD, MTTR, and the other KPIs driving the performance of your goals, you’ll have to rely on tools and processes for tangible improvement.  

Armstrong recommends the following to improve metrics in-house:

• Investing in industry standard comprehensive analyst training specific to detection and response. 
• Enhancing your overall threat hunting capabilities. 
• Conducting ongoing regular red team exercises. 
• Assessing and refining your detection rules based on feedback and past performance. 
• Looking for opportunities to leverage automation throughout the incident response process for quicker triage and response. 
• Fostering both collaboration and knowledge sharing within your SOC to enhance overall effectiveness.  

This advice's core is to find ways to save your team time and effort and improve detection and response performance.  

Tools should be looked at the same way. Principal Product Marketing Manager, Bitdefender Cristian Iordache recommends looking at tools that leverage automation as much as possible and empower teams’ efforts by minimizing the time they spend on analysis and manual tasks. 

He recommends native extended detection and response (XDR) solutions to unify and accelerate threat investigation and response, especially for midsized organizations. While open XDR, security information and event management (SIEM) and Security Orchestration Automation and Response (SOAR) tools can aggregate and analyze large volumes of data, they are more suited for large and experienced teams that can develop and maintain integrations, detection rules and automation playbooks. “Some tools enable build-it-yourself automation – advanced teams can use it, but it takes expertise and effort compared to an XDR solution that should automate many of the key investigation and response processes out of the box,” Iordache says. 

How Native XDR Can Help Improve Detection and Response Metrics 

Native XDR unifies and streamlines security incident analysis and response processes across the organization. It assembles a complete picture of every security incident by automatically correlating and contextualizing alerts from sources such as endpoints, network, identity services, productivity apps, email, mobile devices, or cloud workloads.

The use of native sensors from the same vendor eliminates the need to build and maintain custom integrations, delivers a human-readable synopsis of incidents in real-time and improves detection fidelity, reducing noise and alert fatigue.  

“Most SIEM tools would require you to build detection rules and custom integrations,” Iordache says. “And you’d still have much more to do manually to arrive at an accurate full picture of an incident. XDR assembles a full incident report via different signals in real time, accelerating the detection and response process.” 

Effective Detection and Response Requires Organizational-Wide Effort 

To truly improve your detection and response metrics, it’s important to have a holistic strategy that isn’t just looking to improve on what’s possible today but also adapts and reacts to potential changes.  

“New threats and evolving threat tactics can impact your detection and response quality,” Armstrong says, “It’s important to implement continuous monitoring and ongoing threat intelligence as part of your overall strategy.” 

This is in addition to establishing and maintaining an effective baseline (or working towards one) as that will let you adapt to new threats even faster. Ultimately, the core elements of effective detection and response should be applied across the board for overall cyber resiliency. Know and understand your internal environment, identify your threats and risks, benchmark your current performance, and set goals. Then you’re able to home in on the specific processes and technology required and be able to utilize these tools even more effectively. 

It’s a comprehensive approach, but a necessary one to have tangible impacts on your organization's security and know that you’re both protecting your company and ready to handle an incident when it occurs.