The results of the latest round of the MITRE Engenuity ATT&CK® Evaluations are out. This time the evaluation was modeled against the Wizard Spider cybercrime and the Sandworm nation-state groups. ESET is a pioneer of research on Sandworm, with some of the most significant discoveries made about this threat group under our belt – but more on that later.

First, let’s take a brief look at the evaluation methodology and, most importantly, how ESET Inspect (formerly ESET Enterprise Inspector) fared.

Evaluation overview

As stressed by MITRE Engenuity, “these evaluations are not a competitive analysis” and thus “there are no winners”. The evaluations are a professionally executed, transparent, and objective snapshot of an endpoint threat detection tool’s capabilities to detect some of the adversarial behaviors demonstrated by the threat groups selected as the theme of the current round. At the same time, there are certain areas – crucial in real-world defense – that are out of the scope of the evaluations.

Some key parameters that the evaluations do not consider include performance and resource requirements, alerting strategy, noisiness (alert fatigue – any product could obtain a very high score on most of these results by producing alerts on every action recorded in the test environment), integration with endpoint security software, and ease of use.

The reasoning behind this is that organizations, security operations center (SOC) teams, and security engineers all have different levels of maturity and different regulations to comply with, along with a host of other sector-, company-, and site-specific needs. Hence, not all the metrics given in the ATT&CK Evaluations have the same level of importance to each evaluator.

To analyze the evaluation results properly, it’s important to understand the methodology and a few key terms. The detection scenarios consisted of 19 steps (10 for Wizard Spider and 9 for Sandworm) spanning a spectrum of tactics listed in the ATT&CK framework, from initial access to lateral movement, collection, exfiltration, and so on. These steps are then broken down to a more granular level – a total of 109 sub-steps. ESET Inspect, which supports Linux machines, was not yet released at the time of the evaluation, so Linux-related steps and sub-steps were out of scope. That means 15 steps and 90 sub-steps were evaluated in ESET’s case. The MITRE Engenuity team recorded the responses and level of visibility at each sub-step for each participating solution.
 
The results were then combined into various metrics, essentially based on the solution’s capability to see the behaviors of the emulated attack (Telemetry category) or to provide more detailed analytical data (General, Tactic, and Technique categories). For more details, read MITRE Engenuity’s documentation on detection categories.

Figure 1. Detection categories in the Wizard Spider and Sandworm Evaluation (image source: MITRE Engenuity)

ESET’s evaluation results

The results are publicly available here.

Out of the 15 applicable steps in the detection evaluation, ESET Inspect detected all steps (100%). Figures 2 and 3 illustrate the different types of detection per step.

Wizard Spider Scenario

Figure 2. Distribution of detection type by step in the Wizard Spider scenario (image source: MITRE Engenuity)

Breaking the attack emulation down to a more granular level, out of the 90 applicable sub-steps in the emulation, ESET Inspect detected 75 sub-steps (83%). Figures 4 and 5 illustrate the different types of detection per sub-step.

Wizard Spider Scenario

As the results indicate, ESET Inspect provides defenders excellent visibility of the attacker’s actions on the compromised system throughout all attack stages.

A key metric that is important for SOC analysts to understand what’s happening in their environment is analytics – additional context – for example, why the attacker executed the specific action on the system. ESET Inspect provided this extra information for 69 of the detected sub-steps (92%).

Note that ESET did not participate in the Linux part of the evaluation as the new version of ESET Inspect with Linux support was publicly launched only on March 30, 2022, completing our coverage of all major platforms alongside Windows and macOS.

Linux detections aside (note that ESET’s ecosystem does provide endpoint protection for Linux – but this was outside the scope of this evaluation), ESET Inspect did not identify 15 out of the 90 sub-steps.

Nearly all of those “misses” are due to ESET Inspect not monitoring certain API calls. API monitoring is a tricky business due to an unfavorable signal-to-noise ratio. Considering the enormous number of API calls present in a system, monitoring all of them is neither feasible nor desirable as this would be an enormous hog on resources.

To provide an example, one of the missed sub-steps (10.A.3) pertains to detecting the CreateToolhelp32Snapshot API call, which is commonly used in legitimate applications for process enumeration. This sub-step precedes an attempt to inject malicious code into a process. ESET Inspect takes the more efficient strategy of detecting this process injection, putting focus on the less frequent and more suspicious action.

This is in no way trying to say that API monitoring doesn’t have its place on a defender’s checklist – ESET is continuously evaluating scenarios where it makes sense to stay vigilant and add detection capabilities – yet in some cases this provides very little additional benefit at a very high cost.

The key principle when designing an effective extended detection and response (XDR) solution – and this applies to endpoint security software as well – is balance. In theory, it’s easy to create a solution that achieves 100% detections – simply detect everything. Of course, such a solution would be next to useless and is precisely the reason why traditional endpoint protection tests have always included a metric for false positives – a true comparative test cannot be done without testing for false positives. This is also the reason why security analytics platforms typically suffer from a high false positive rate and have heavy resource demands. Such platforms will have to contend with XDR solutions to remain relevant.

Yes, the situation is a bit different with XDR compared to endpoint protection (because you can monitor or detect without alerting) but the principles still apply: too many detections create too much noise, leading to alert fatigue. This causes an increased workload for SOC analysts, who have to sift through a large number of detections, leading to the exact opposite of the desired effect: it would distract them from genuine high-severity alerts. In addition to the increased human workload, too many lower importance detections also increase costs due to higher performance and data storage requirements.

The essential role of a good XDR solution is not necessarily to alert the analysts to every single procedure carried out during an attack (or sub-step in the ATT&CK Evaluation). Rather it should alert them that an attack took place (or is ongoing) … and afterward, support the investigation by providing the capability to navigate transparently through detailed and logically structured evidence of what happened in the environment and when. This is a functionality that we continue to put great emphasis on in developing ESET Inspect.

Figure 6 shows the detection of sub-step 19.A.6 – an attempt to spread NotPetya from an infected machine via WMI. Figure 7 depicts the attempt detected from the other side – the targeted machine.

In addition to alerting the SOC analysts to the malicious activity, additional contextual information is provided, including detailed command line parameters executed by the adversary, and the execution chain and process tree – highlighting further related events that were suspicious or clearly malicious.

An explanation of the observed behavior is provided, along with a link to the MITRE ATT&CK knowledge base, and the typical causes for this type of behavior, both malicious and benign. This is especially helpful in ambiguous cases where potentially dangerous events are used for legitimate purposes due to the organization’s specific internal processes, which are for the SOC analyst to investigate and distinguish.

Recommended actions are also provided, as well as tools to mitigate the threat by actions such as terminating the process or isolating the host, which may be done within ESET Inspect.

Wizard Spider and Sandworm

ESET has done extensive research on both groups that were the inspiration for this evaluation round.

Wizard Spider is a financially motivated group behind TrickBot and the infamous Emotet malware, which has often led to the deployment of ransomware such as Ryuk. These have been some of the most active botnets and thus in the sights of ESET’s automated Botnet Tracking service for some time. We have been closely monitoring the development of these threats and our systems have uncovered (and blocked) new variants, configurations, and command and control domains even before they were used in an attack against potential victims.

Our effective defense against Emotet apparently caused significant frustration to Wizard Spider. In addition to implementing protection for users of our security solutions, our extensive threat intelligence on this group helped in the coordinated TrickBot takedown effort that crippled its activity for several months.

Sandworm is an advanced persistent threat (APT) group notorious for its high-impact attacks against Ukraine and other countries. The US Department of Justice and the UK National Cyber Security Centre attributed the group to the Russian GRU. ESET’s research has played a pivotal role in uncovering the group’s activities.

ESET first identified the work of Sandworm, along with the references in its code to Dune from which its name was inspired, as that of the BlackEnergy subgroup in a Virus Bulletin 2014 talk. We have tracked this group’s activity from its inception (around the time of the beginning of the war in Donbas and the occupation of Crimea in 2014), later uncovering both of its attacks against the Ukrainian power grid – the BlackEnergy-facilitated attack in 2015 and the Industroyer attack in 2016.

Most pertinent to this evaluation round were our discoveries around NotPetya in 2017, as the second detection scenario specifically emulates this faux ransomware. ESET researchers were the ones who linked NotPetya to TeleBots, a subgroup of Sandworm, and uncovered patient zero of what eventually became the most costly cyberattack in history: the supply-chain compromise of M.E.Doc.

For a more detailed overview and timeline of Sandworm’s attacks in Ukraine and around the world, and for information on the recent cyberattacks we detected around the 2022 Russian military invasion of Ukraine, refer to our blogpost, podcast and webinar.

Conclusion

We are happy to see that the rigorous MITRE ATT&CK Evaluation demonstrated the qualities of our XDR-enabling technology and validated the vision and roadmap we have for ESET Inspect looking forward.
 
It’s important to keep in mind that the development of a high quality XDR solution cannot be a static undertaking. As adversary groups change and improve their techniques, so must XDR and endpoint protection platforms keep pace to continue protecting organizations from real-world threats.
 
And that’s exactly the case with ESET Inspect: it is not a solution whose development is disconnected from active threat research. No, it’s our experts who track the world’s most dangerous APT groups and cybercriminals who also ensure ESET Inspect’s rules are effective and capable of detecting malicious activity on targeted systems.

The threat intelligence that is a product of our research is used to improve our security solutions and is also offered to customers as part of our premium threat intelligence offering in the form of private reports and data feeds covering technical, tactical and strategic threat intelligence. This is on top of our publicly available research, for example, on the subject of the Ukraine crisis.  
 
ESET Inspect is just one of the components in our comprehensive cybersecurity portfolio, designed to deliver reliable protection against cyberattacks. ESET Inspect is an integral part of ESET’s multi-layered security ecosystem, which includes strong endpoint security, cloud-based protection, machine learning-based detection technologies, and ESET LiveGrid® telemetry coming from a user base of tens of millions of endpoints (which, among other benefits, allows ESET Inspect to factor into its decisions the reputation of binaries and processes).
 
ESET believes this unified approach to delivering security solutions is absolutely crucial, because while it’s important to have great visibility into an attack executed in your network, it is much more important to be able to spot and recognize it among a myriad of events, or even better, to prevent it from happening at all.

For more information about ESET’s participation in the ATT&CK Evaluations, visit our page here.

Obviously, visibility is not the only metric. The additional metrics considered in the ATT&CK Evaluations largely center around the context that accompanies the detection of specific adversarial techniques. These have a varying level of importance to each organization, depending on its level of maturity, the different regulations it needs to comply with, and a host of other sector-, company-, and site-specific needs. Yet there are other metrics, such as performance and resource requirements, alert fatigue, integration with other security software, and ease of use, that are unconsidered in the evaluations.

This rich palette of metrics in part reflects the success of EDR as a go-to tool for security defenders. Even organizations without dedicated security teams have gained interest in this type of defense tool. Yet the market is pushing EDR to become even better. Not infrequently, security defenders have been juggling between an EDR solution to hunt down threats and a security information and event management (SIEM) solution to sift through big data sets as complementary activities to defend their networks — the strategy being that what is lacking in one tool can be made up for by the other.

Some might think that if only a synthesis could be achieved between EDR and a security analytics platform, security defenders would have the ultimate tool in their hands. However, Forrester analysts see the situation differently. EDR needs to evolve into extended detection and response (XDR) by learning to ingest data feeds sourced beyond endpoints for analysis and response. Budding XDR solutions can then enter into a toe-to-toe competition with security analytics platforms in a race that appears destined for collision.

On the other side, to remain competitive, security analytics platforms need to be equipped with EDR features like higher-quality alerts and threat response capabilities and use resources more efficiently. The evolved solution that demonstrates better overall security is likely to become the increasingly favored tool for defenders. Our bet is that XDR has the winning odds because of its stronger foundation on EDR solutions that are designed for higher-quality detections with rich context, even if partially limited in overall visibility.

What does XDR mean for ESET?

ESET adopts Forrester’s understanding of XDR as the next generation of EDR that moves beyond endpoints to collect data also from network devices, email servers, cloud-based services, and other sources and allows security defenders to discover and respond to more threats. In order to achieve this increased level of data collection and response capability, XDR can use two integration strategies: hybrid and native.

A hybrid strategy focuses on integrations with third-party tools. A native strategy focuses on integrations with tools from the same vendor. Currently, no vendor offers a complete XDR solution, making it an ideal yet to be achieved.

ESET is committed to building out a full XDR solution by iteratively improving ESET Inspect (formerly ESET Enterprise Inspector) and continuing to add integrations both with ESET’s ecosystem of security tools and with third-party tools. Previously, as ESET Enterprise Inspector, ESET’s EDR solution was positioned for enterprise security teams. Now, as ESET Inspect, it has evolved into the XDR-enabling component of the ESET PROTECT platform, ready for both large enterprise and smaller security teams.

Before looking at the ESET PROTECT platform, let’s consider ESET Inspect a little more closely.

ESET Inspect: Enabling the pursuit of XDR

More than being a mere name change, ESET Inspect is a new version with increased visibility and remediation capabilities. ESET Inspect carries over the familiar features of previous versions, such as the referencing to the MITRE ATT&CK knowledge base in its rule set, a REST API for integration with security orchestration, automation, and response (SOAR), SIEM, ticketing and other similar tools, and the ability to run a remote PowerShell session on Windows machines for precise response and remediation options.

In the latest version of the product — version 1.7 — several major Linux distributions are now supported. This adds to the already existing support for Windows and macOS and marks an important milestone on the road to a complete XDR solution. ESET Inspect’s coverage for Linux machines works in concert with ESET Endpoint Antivirus for Linux and ESET Server Security for Linux (from versions 9.0) to support the following distributions:

ESET Inspect also features an expanded rule set currently consisting of almost 1,000 rules. The rules are used to analyze events happening on endpoints, firewall and network data, user account actions, and other data for potentially malicious behaviors to be investigated by security engineers.

ESET researchers have enhanced a selection of rules with automated responses, for instance, to isolate a computer or block an executable. The use of an automated response in specific rules is made visible in the “Active Rule Actions” column of the “Detection rules” view:

The continued growth of ESET Inspect’s rule set and its integration with ESET’s other security products is a good example of building a native XDR solution. For ESET, the core of this integration lies in ESET PROTECT, a management console for IT admins to deploy and configure ESET security products. The ESET PROTECT platform packages endpoint security, server security, encryption, cloud-based, and detection and response solutions with the console to provide businesses a scalable prevention, detection, and response capability.

Thus, positioning ESET Inspect more closely to ESET PROTECT helps create a more integrated experience for security defenders using ESET’s security ecosystem. As mentioned above, one of the key goals of an XDR solution is to incorporate threat data beyond just endpoints, meaning that continuing to extend ESET Inspect’s coverage to more platforms and data sources that are already integrated with ESET PROTECT puts it firmly on the road to that goal.

Lifting ESET PROTECT Enterprise toward XDR

ESET is building an XDR solution on the ESET PROTECT Enterprise platform, which packages ESET Inspect along with endpoint and server protection, cloud-based threat detection, and encryption to provide a comprehensive security capability.

XDR demands more data for ingestion, but this entails a need for greater database storage capacity that can hit performance hard if not adequately handled. Investing in more hardware to buttress on-premises database solutions is always possible, even if lacking some flexibility. This is why thinking ahead about the database needs for ESET PROTECT and the database needs for ESET Inspect is crucial.

With a cloud deployment, however, scaling up to meet this data storage demand is much more flexible as there is no need to buy new hardware. The cost lies rather in renting more storage space and possibly more powerful machines — a change that can be executed rapidly. The flexibility and scalability offered by the cloud are precisely the qualities required of an XDR solution, as Forrester points out.

Formerly, ESET PROTECT and ESET Inspect were only available for on-premises deployment, which could require considerable hardware investment, server maintenance costs, and on-site staff. While on-premises deployment remains an option, especially for organizations more desirous of keeping their data close to hand, both these solutions are also available in the cloud — a new capability for ESET Inspect since version 1.7 — and thus answer XDR’s call for cloud-native support.

Managed detection and response

In some cases, especially for smaller organizations or those whose core focus is outside of security, extracting value from an XDR solution can be too heavy of a lift to make unassisted. Similarly, managing ESET PROTECT Enterprise requires either full-time security staff or a managed detection and response (MDR) provider to take over some, if not all, of the daily security work.

For some time, the manufacturers and developers of complex products have recognized the need to increase the value of their products with a service offering. This is because at times customers struggled to see the value of their purchase when they did not have the expertise to use the product well, often misconfiguring it or failing to use all its features. More importantly, offering services reflects the type of relationship that providers wish to have with customers — for ESET that means engaging in a security partnership with customers.

ESET’s revamped MDR service assists security teams to set up ESET’s products with the best configuration and optimizations possible for their networks. The service also includes periodic health checks to address any poor security practices or misconfigurations that may have crept in over time. Critically, organizations can request the help of ESET security engineers for on-demand or proactive threat hunting with ESET Inspect.

Interested organizations (in select countries) can explore their options by reaching out via ESET’s MDR service web page.

Conclusion

This article started by discussing visibility and all the other metrics that organizations need to look at when considering a detection and response solution. For a snapshot of the depth of visibility that ESET Inspect can provide threat hunters, read the blogpost: Hunting down Sandworm and Wizard Spider: How ESET fared in the ATT&CK® Evaluations.

Designing a detection and response solution that balances all these metrics is by no means an easy task. The brief journey through XDR taken here has revealed a complexity of issues in developing native and hybrid integrations for EDR solutions and in scaling up data collection via cloud deployments, all while making detections of high quality and rich with context and maintaining high system performance.

While there are certainly mature organizations ready to wade through this complexity, the tough reality for most organizations is that they may have only one or two IT administrators … and they are managing security part time, along with their many other responsibilities. This can make the ATT&CK Evaluations, which concentrates on sophisticated threat groups, or even a discussion about XDR appear as somewhat of a niche focus.

However, both the evaluations and the pursuit of XDR speak to the gamut of security practices — from basic configurations and policies all the way to advanced fine-tuning and optimization — that should be in place to protect against even the most common threats. Furthermore, XDR’s demand for the cloud opens the door to improved security even for organizations without dedicated security teams. Such organizations may discover that their security practice falls short of industry standards when engaging in this discussion. At least in this way — by beginning the game of catch-up on their security — the ATT&CK Evaluations and the pursuit of XDR can provide actionable insight even to less mature organizations.

ESET releases a slew of updates enhancing the preventive defenses of businesses

In early 2021, ESET released ESET PROTECT, a new generation of our remote management tool enabling IT admins to securely deploy and manage ESET security products. Now, in 2022, ESET’s line of business products is undergoing a wide-ranging refresh that will touch nearly all business customers.

The overarching goal behind the bevy of upgrades is to sharpen customers’ tools to prevent cybercriminals from playing havoc with their networks. Although it is not possible to prevent all attacks on a network, by reducing the attack surface along with the employment of preventive measures such as speedy patching, careful system configuration, fastidious monitoring, and periodic health checks, it is possible either to entirely remove the ability for some types of attacks to be attempted or to quickly detect the red flags that an attack has started so as to shut it down.

Among the host of changes, four are key in assisting IT admins to better manage the security risks in their environments: auto-updates, the ESET PROTECT dashboard for ESET LiveGuard Advanced, management of Brute-Force Attack Protection, and web control for Android devices.

1. Auto-updates for patch management
A new auto-update mechanism to keep ESET Endpoint Security and ESET Endpoint Antivirus for Windows always up-to-date is now in place. Starting from version 9, auto-update is enabled by default with the possibility for IT admins to turn off this feature entirely or only up to a specific version via the ESET PROTECT console.

The introduction of the auto-update mechanism aims to lighten the burden on IT admins’ shoulders of keeping security software up-to-date in a timely manner, especially when new protective features become available and bugs need to be fixed.  

2. An inside view into ESET LiveGuard Advanced
ESET PROTECT now offers a dashboard for ESET LiveGuard Advanced (formerly called ESET Dynamic Threat Defense). ESET LiveGuard Advanced is a cloud-based technology that analyzes suspicious files submitted by endpoints for new or previously unseen threats, especially new versions of ransomware. By running the submitted samples in the cloud, it is possible to use heavyweight machine learning models and robust detection algorithms that can rapidly analyze and score the samples as malicious, suspicious, or clean.

The ESET LiveGuard Advanced dashboard gives customers a broad view into the detections seen by the technology globally and more detailed statistics for their own networks:

3. Brute-force attack protection for remote workers
To keep pace with the new Brute-Force Attack Protection in ESET endpoint security products for Windows, IT admins can now manage this module from the ESET PROTECT console. Introduced to better defend businesses in the remote work era, this detection layer blocks external IP addresses that exhibit the characteristics of an oncoming brute-force attack on remote desktop protocol (RDP) logins.

With the ubiquitous availability of RDP on Windows machines and the need to work remotely becoming an overnight need in 2020, many businesses turned on RDP to give employees access to company resources from home. In the hope of breaching organizations worldwide who may have failed to secure their use of RDP sufficiently, cybercriminals have been scanning the internet and attempting billions of brute-force attacks as seen in ESET telemetry alone.

4. Protecting Windows on Arm
With the expected growth of ARM64 hardware reaching employees’ hands, ESET Endpoint Security and ESET Endpoint Antivirus now offer ARM64 builds from version 9. Traditionally, ARM-powered processors dominated the mobile device market due to unique hardware characteristics that made them amenable to hand-sized form factors, fanless designs, and long battery life. Taking these ideas to laptop and 2-in-1 platforms has opened a whole new world of useful devices that extend the familiar experience we know so well from mobile devices along with the need to secure them.

Protecting more than just Windows
For Android users, a new web control feature that allows IT admins to regulate employees’ access to websites from their mobile devices is now available via ESET PROTECT Cloud. Using built-in categories and custom rules, admins can blacklist, whitelist, or warn about URLs that lead to sites with harmful content or that can negatively impact employee productivity.

For Mac users, ESET has released ESET Endpoint Antivirus version 7, which is a new generation of the product built with a microservices architecture that replaces the previous monolithic one. As a result, the product has better performance because it uses services only as needed and has an improved multithreaded design for faster scanning on multiprocessors; is more stable because failed components can automatically restart instead of causing the entire application to crash; has better security isolation because components can run with non-privileged user rights; and is compatible with Apple M1 devices via Rosetta 2.

Currently, version 7 of this product can be managed via ESET PROTECT. The graphical user interface on the endpoint will become available for full management of the product in a future release.

Future plans
Finally, while ESET remains committed to protection for macOS devices, work remains to be accomplished in reengineering native builds for Apple M1 devices and in creating the additional protective modules that will integrate into the new microservices architecture to make up the full feature set of ESET Endpoint Security for macOS.