TL;DR
This paper surveys automated attack investigation systems designed to help enterprises respond to sophisticated cyber threats. It highlights the challenges these systems face, such as alert fatigue and data volume, and categorizes existing approaches based on their data sources, investigation types, and system architectures. The authors also identify critical open problems, including the need for real-time analysis, reduced analyst workload, and more realistic evaluation methods.
What Problem Does This Paper Address?
Modern enterprises are constantly under siege from sophisticated adversaries who employ multi-stage attacks to infiltrate networks, establish persistence, and exfiltrate sensitive data or demand ransom. While traditional security systems like EDR (Endpoint Detection and Response) and SIEM (Security Information and Event Management) can detect suspicious activities, they generate an overwhelming volume of alerts. This "alert fatigue" forces human analysts to sift through a haystack of false positives, increasing the risk of missing genuine threats. The sheer scale and complexity of enterprise networks, coupled with noisy and incomplete data, make automated investigation a significant challenge. This paper addresses the need for systems that can automatically investigate these complex attacks, reduce the burden on human analysts, and provide actionable insights.
Methods
The paper categorizes automated attack investigation systems based on several parameters:
- Detection System Type: Online (real-time detection) vs. Offline (analyzing collected logs).
- Data Source: Provenance-based (tracking OS-level events like file access, process creation) vs. other data sources (e.g., network traffic, login events).
- System Type: Single-host vs. Multi-host (crucial for detecting lateral movement).
- Provenance Granularity: Fine-grained (mapping byte-to-byte flow) vs. Coarse-grained (tracking information flow at a higher level).
Key techniques employed by these systems include:
- Causality Graph Construction: Building a graph that links related events to reconstruct the attack timeline and identify root causes.
- Root Cause and Scope Analysis: Using backward tracing to find the origin of an attack and forward tracing to understand its extent.
- Anomaly Detection: Employing heuristics (topological graph features, machine learning models) to identify suspicious events or paths.
- Mitigating Dependency Explosion: Techniques to manage the exponential growth of dependencies in large systems, such as partitioning processes or using tag propagation.
- Prioritizing Alerts: Developing algorithms to rank alerts based on severity, rareness, and fanout scores to help analysts focus on critical threats.
- Interpretable Results: Presenting findings in a way that is understandable and actionable for human analysts.
- Mitigating Storage: Strategies for compressing and managing large volumes of log data without losing critical forensic information.
Datasets
The paper discusses datasets used by the surveyed systems, noting a significant reliance on:
- Enterprise Data: Systems often partner with enterprises to use their real data (e.g., Dropbox, Microsoft, Symantec, NEC Labs America).
- Public Datasets: Datasets like DARPA TC, ATLAS, and DEPIMPACT are used.
- Simulated Attacks and Red-Team Testing: The majority of systems are evaluated using simulated attacks or red-team exercises due to the difficulty of obtaining real-world attack data.
The authors highlight that only one system (Latte) used a real-world instance of a lateral movement attack in its dataset for evaluation, and only one system (NodLink) was reimplemented in an online setup to capture real-world threats. This points to a significant gap in realistic evaluation benchmarks.
Key Insights and Ideas
- The "Needle in a Haystack" Problem is Real: The sheer volume of alerts from traditional security tools makes manual investigation incredibly challenging and prone to errors.
- Causality is Key: Reconstructing the sequence of events and understanding their causal relationships is fundamental to effective attack investigation.
- No Silver Bullet: Different systems excel at different aspects of attack investigation. There's no single solution that perfectly addresses all challenges.
- The Human Element Remains Crucial: Despite automation, human analysts are still essential for triaging, validating, and responding to threats. Systems need to support, not replace, them.
- Real-World Data is Scarce and Sensitive: The lack of readily available, realistic attack data is a major bottleneck for developing and evaluating robust investigation systems.
- Interpretability Matters: Even with advanced detection, if the results aren't understandable to an analyst, their utility is limited.
Key Results
The paper doesn't present a single set of quantitative results from a unified experiment. Instead, it synthesizes findings from the surveyed literature:
- Categorization of 11 Systems: The paper provides a detailed classification of 11 systems based on their capabilities in mitigating dependency explosion, prioritizing alerts, providing interpretable results, and mitigating storage. Table 2 summarizes these capabilities, showing that no single system excels in all areas.
- Common Challenges: The paper identifies common challenges faced by these systems, including high alert reporting rates, the time-consuming nature of manual investigation, the difficulty of obtaining complete and tamper-evident data, and the need for interpretable outputs.
- Identification of Open Problems: The authors highlight four key areas for future research: pushing towards real-time analysis, reducing analyst workload, addressing unrealistic assumptions, and incorporating more realistic evaluations.
Limitations and Future Work
Limitations:
- Data Scarcity: The reliance on simulated attacks and red-team testing means that the realism of evaluations is often questionable. Obtaining and sharing real-world attack data is a significant hurdle due to privacy and proprietary concerns.
- Unrealistic Assumptions: Many systems assume tamper-evident logs and complete data availability, which is not always the case in real-world scenarios.
- Analyst Workload: While many systems aim to improve alert quality, few explicitly consider the analyst's time budget or the interpretability of their findings as primary design parameters.
- Generalizability: Some systems' assumptions about enterprise structure (e.g., functional units) may limit their applicability across different organizations.
Future Work:
- Real-time Analysis: Developing systems that can detect and investigate attacks in real-time, rather than relying solely on offline analysis.
- Reducing Analyst Workload: Designing systems that actively minimize the number of alerts and present information in a highly interpretable and actionable format.
- Realistic Evaluations: Creating and utilizing more realistic datasets and benchmarks that better reflect the complexities of real-world attacks.
- Adaptability: Building solutions that can integrate with existing EDR systems and adapt to diverse enterprise environments.
- Addressing Data Gaps: Developing methods to handle incomplete or potentially tampered log data.
Summary updated