How can the MITRE Engenuity ATT&CK® EVALUATIONS help you reduce cybersecurity operational cost

Security operations are expensive! Hiring, training, and keeping a team of security analysts demands significant resources from any organization wanting to run security operations in-house. There are many studies which show the urgent need to improve security operations center (SOC) analysts’ productivity.

For example, a pre-pandemic study conducted by Ponemon Institute showed security teams spend approximately 25% of their time chasing false positives. The Covid-19 pandemic has made things worse by increasing the number and aggressiveness of the cyber threats affecting both individuals and businesses. Security alert fatigue not only is raising the direct cost of running security operations but is also generating indirect costs that can be significant to organizations. Long and inefficient security investigations are increasing the response time to cyber threats, increasing attacker dwell time, and enabling the negative consequences of attacks to pile up dramatically in many cases. Aside from that, a busy security operations team will manage to triage only about 70% of total alerts with around 30% remaining untouched (and, thus, uninvestigated) as the volume is simply too high for teams to keep up. This significant blind spot leaves the organizations vulnerable to attacks for extended periods of time.

Coping with alert fatigue

To solve the problem, one could try to add resources to the SOC. However, this is obviously impractical both from a cost perspective and because of the difficulty finding trained security professionals to help reduce unaddressed security alerts. Therefore, the main solution remaining is a technical one: reduce the number of total alerts and/or reduce the time required to triage with the help of technology. Ironically, security technology   is also the root cause of the problem. The limited capability of most security architectures, especially Endpoint Detection and Response (EDR), to make sense of detected activities and clearly distinguish attack tactics and techniques, is a major factor contributing to the security alert overload   analysts must deal with. The problem is obvious; however, the solution is not because of tradeoffs required between detection sensitivity, security insights and accuracy.

The sweet spot between detection, insights, and accuracy

Choosing the optimal EDR solution that delivers aggressive threat detection, provides security insights consistently, while also keeping the false positives to a minimum, is tough for security decision makers. Extended trials are necessary, and most organizations lack the resources, methodology and experience to perform accurate product comparisons. Fortunately, the MITRE Engenuity ATT&CK® Evaluations, can be used with confidence to complement and even replace extended in-house product trials and comparisons. Here is how.

As covered in a previous post, MITRE uses a methodology for testing EDR solutions that is unique among cybersecurity industry tests. It not only tests the solution’s ability to detect and block cyber threats, but meticulously emulates the full behavior of sophisticated attacks (APT3 or APT29 in previous rounds and Carbanak/FIN7 in most recent tests). This allows the MITRE ATT&CK Evaluation to reveal, in detail, capabilities of each solution to detect, analyze, provide telemetry and visibility on all phases/sub-phases of a cyber-attack.

There are two key metrics associated with alert fatigue to focus on in the evaluation results: Total Number of Detections and Total Number of Analytics (or Analytic Detections). The combination of these two metrics provides highly valuable insights for maximizing detection sensitivity, accuracy, and alert validation. The Detection Count reveals the raw ability of the EDR solution to detect potentially unwanted activities. The more detections, the stronger the EDR’s ability to detect threats, but also, a higher probability to generate false positives. MITRE Analytic Detections tests address false positives    by correlating EDR detections with specific security tactics or techniques. This metric focuses on security insights and is critical for security analysts. An alert that has security context and insights already included is faster to triage and easier to solve. Skipping some more technical details, the ratio between the ability to clearly identify attack techniques and raw detection power is a very important indicator for accuracy. The higher the percentage of attack steps and sub-steps that can be revealed in detail, the higher the chances of a solution to produce no-nonsense alerts and less overall noise.

Total Detections, Analytics Insights and Detection Accuracy are key metrics that an organization should leverage when evaluating the potential cost impact of an EDR solution on security operations. The higher the analytics coverage, the more support this will offer to security analysts for triaging and resolving incident alerts.

To learn more about the key metrics included in the 2021 MITRE Engenuity ATT&CK® EVALUATION and how use them to support cybersecurity decisions, join our Live Webinar on May 12, 3PM GMT. Dragos Gavrilut, director of Bitdefender Cyber Threat Intelligence Lab, will share insights on what makes the MITRE Evaluation valuable and how to use the results to improve your organization’s cyber resilience.