As cyberattacks grow in volume and sophistication, and the cybersecurity skills gap widens, many organizations are turning to Managed Detection and Response (MDR) services. But finding the right MDR provider can be a real challenge. How do you know if they can keep up with today’s sophisticated cyber threats? That’s where MITRE evaluations come in, helping organizations make smarter choices by providing valuable insights into Managed Services from eleven different vendors. Read on as we review the key factors that can help you make the best decision for your cybersecurity needs.

Inside The MITRE ATT&CK® Evaluations for Managed Services

This year’s MITRE Engenuity ATT&CK® Evaluations used a multi-threaded attacks to evaluate each participating vendor. The first mimicked the attack tactics and techniques of the cybercriminal group menuPass. They are known for targeting various industries globally, with a focus on stealing sensitive information like intellectual property. The attacks are known for exploiting living-off-the-land techniques to avoid detection and leveraging third-party relationships for stealing credentials.

The second utilized the BlackCat ransomware written in the RUST language. The ransomware is operating system agnostic, capable of targeting Windows and Linux systems across multiple industries. BlackCat is designed to disrupt system defenses, encrypt data, and obstruct the recovery processes. Both scenarios represent adequate examples of the types of attacks that target modern businesses.

The Results

While achieving a strong performance in MITRE MDR evaluations is certainly a point of pride for us, a single score can't capture the entire story. These evaluations are valuable because they delve into a range of interconnected metrics, providing a more nuanced picture of a vendor's capabilities. However, it's important to consider the data in context, as some vendors might choose to focus on specific metrics to their advantage.

Here, we'll unpack the key metrics from our MITRE Managed Services evaluation and explain what they mean for you. We'll also explore some of the qualitative aspects, like reporting style, that can be gleaned from the vendor communications provided by MITRE. This approach will help you understand how our performance translates to your specific security needs.

FIGURE 1: The table shows the results for all vendors across the evaluated categories.

Seeing It All

The MITRE evaluation assesses a vendor's MDR solution across a series of 43 sub-steps, representing various stages within attacker tactics and techniques. There are three key levels measured for each sub-step:

1. Visibility: This determines if the vendor's solution can collect sufficient data to identify that a particular sub-step happened. It's essentially a test of the platform's ability to see the attacker activity.
   • 100% coverage: We achieved a perfect score in Visibility, indicating our solution can effectively collect data to identify all 43 sub-steps within the attacker tactics and techniques. This is a strong showing, demonstrating our platform's ability to "see" attacker activity across the entire evaluation spectrum.
2. Reported (not actionable): Here, the evaluation goes beyond simply detecting the activity. It checks if the vendor can not only identify the sub-step but also report it. However, this report may lack specific details or context, making it difficult to take immediate action.
   • 95% coverage: Our score here is also impressive, exceeding the average of 80% coverage. This means our solution can not only identify most of the sub-steps (41 out of 43) but also report them. While these reports might lack specific details, it highlights our strength in detecting suspicious activity.
3. Reported (actionable): This is the ideal scenario. The vendor not only detects and reports the sub-step but also provides additional information like timestamps, locations, users involved, and the nature of the activity. This richer context allows for a more informed and effective response.
   • 93% coverage: We are proud to announce that we achieved the highest score in "Reported - actionable" compared to the average of 65%. This score signifies our exceptional ability to not only detect and report sub-steps but also provide the most valuable context, including timestamps, locations, users involved, and the nature of the activity. This allows your security team to take swift and decisive action to mitigate threats.

Having established our strong foundation in detecting attacker activity, let's now explore how efficiently we translate detection into action. This is where Mean Time to Detect (MTTD) comes into play.

Swift Response, Informed Decisions

Mean Time to Detect (MTTD) measures the average time it takes for a security provider to identify and alert of potential attacker activity. A lower MTTD generally indicates faster detection and response capabilities. Bitdefender’s averaged 24 minutes MTTD, significantly faster than the average response time of 42 minutes.

Our focus lies in striking a balance between timely detection and minimizing unnecessary noise. We prioritize delivering high-fidelity alerts that provide actionable insights, allowing your security team to respond efficiently to genuine threats. It’s important to consider MTTD in conjunction with other metrics, particularly the volume of alerts generated – or noise.

Minimizing Noise for Maximum Efficiency

A critical aspect of any MDR solution is its ability to distinguish between genuine threats and irrelevant noise. Security teams are often bombarded with an overwhelming number of alerts, making it difficult to focus on the most critical issues.

In the MITRE MDR evaluations, the Bitdefender MDR team prioritized a balance between minimizing noise and maintaining high alert fidelity. While some vendors generated alert volumes in the hundreds or even thousands, Bitdefender MDR produced a significantly lower number of alerts compared to the industry average (130 emails and 389 console alerts).

Here's what this translates to for you:

• Reduced Alert Fatigue: Our solution helps security teams avoid information overload by presenting a smaller volume of alerts (54 emails and 28 console alerts). This allows them to focus their attention on the most important threats.
• Prioritizing High-Severity Events: Our focus on fidelity ensures that a significant portion of our alerts (77% of emails) are classified as critical or high severity, ensuring security teams can prioritize the most impactful threats.
• Actionable Insights: Quantity doesn't equate to quality. We prioritize providing clear and concise information within each alert, empowering security teams to take decisive steps to mitigate identified threats.

Conclusion

The MITRE MDR evaluation showcases Bitdefender MDR's strengths: exceptional threat detection, actionable insights (highest among participants for "Reported-actionable"), and a commitment to minimizing alert fatigue. This translates to a powerful solution that empowers security teams to focus on what matters most – effectively responding to genuine threats and keeping organizations secure.

FIGURE 2: Bitdefender MDR achieved the highest actionability score while keeping noise to a minimum

Want to learn more? Dive deeper with the experts themselves! Join our upcoming webinar featuring Bitdefender's SOC analysts and security researchers on Wednesday, June 26th, 2024 at 10:00 am EST. They'll be unpacking the MITRE MDR evaluations, discussing our results in detail, and answering any questions you have about our approach to MDR. This is a technical deep-dive (not a marketing event) and a chance to learn directly from the front lines of threat detection and response.