Sentient: Detecting APTs Via Capturing Indirect Dependencies and Behavioral Logic

Wenhao Yan, Ning An, Wei Qiao, Weichang Wu, Bo Jiang, Yuling Liu, Zhigang Lü, Junrong Liu

TL;DR

Sentient is a new anomaly detection framework that uncovers stealthy Advanced Persistent Threats (APTs) by looking beyond immediate neighbors in system logs. By combining a Graph Transformer with a bidirectional Mamba2 architecture, it captures long-distance dependencies and the "behavioral logic" of attackers, reducing false positive rates by an average of 44% compared to current state-of-the-art methods.

What Problem Does This Paper Address?

Advanced Persistent Threats (APTs) are the "ninjas" of the cyber world. They move slowly, stay dormant for months, and meticulously blend in with legitimate system activity. Security researchers often use provenance graphs—visual maps of how processes, files, and network sockets interact—to track these attacks.

However, existing graph-based detection tools have three major blind spots:

  1. Missing Indirect Dependencies: Most tools use Graph Neural Networks (GNNs) that only "see" immediate neighbors. If an attacker triggers a chain of events across distant parts of the system, the model might miss the connection.
  2. Noisy Scenarios: In a real attack, an infected process still performs thousands of normal tasks. This "noise" often drowns out the tiny signal of the exploit.
  3. Missing Behavioral Logic: Individual actions (like writing to a log) might look benign in isolation. Their malicious intent only becomes clear when you see the logical sequence of events leading up to and following them.

Methods

Sentient introduces a multi-stage pipeline designed to understand the "intent" behind system behaviors:

  • Graph Construction with Positional Awareness: The system converts audit logs into a graph. It uses Word2Vec to capture the semantic meaning of file names and processes, and Laplacian eigenvectors (a mathematical way to describe a node's location in a network) to give the model a sense of the graph's overall structure.
  • Pre-training with Graph Transformers: Unlike standard GNNs, the Graph Transformer uses an attention mechanism that allows every node to "attend" to every other node. This ensures that indirect dependencies—even those many hops away—are captured.
  • Intent Analysis Module (IAM): This is the heart of the paper. Sentient uses Bi-Mamba2 (a bidirectional version of the state-of-the-art State Space Model) to analyze sequences of behaviors. Mamba2 is exceptionally good at processing long sequences efficiently. By looking at these sequences both forward and backward, Sentient learns the "logic" of how system events relate to one another.
  • Self-Supervised Learning: Crucially, Sentient is trained only on benign data. It learns what "normal" looks like and flags attacks as deviations (reconstruction errors) from these established patterns. This means it can potentially detect "zero-day" attacks it has never seen before.

Datasets

The authors evaluated Sentient on three widely recognized benchmarks in the APT detection community:

  • Streamspot: A dataset containing five benign scenarios and one complex malicious scenario.
  • Unicorn Wget Dataset: Simulated attacks designed to test a system's ability to handle high-volume logs.
  • DARPA-E3 (Cadets, Theia, and Trace): The gold standard for APT research, featuring real-world "Red Team" attacks conducted on an enterprise network.

Key Insights and Ideas

  • Global vs. Local Context: The paper argues that "spatial" adjacency in a graph isn't enough. An attacker might use a browser to download a script, which then uses a system utility to escalate privileges. These events are logically linked but might be "spatially" far apart in a massive provenance graph.
  • Mamba2 for Security: While Transformers are great for global structure, Mamba2 proves to be highly effective for the "temporal" or sequential logic of system calls. It offers a more efficient way to model long-term behavioral dependencies than traditional RNNs or LSTMs.
  • Denoising via Intent: By clustering behaviors with similar "intent embeddings," Sentient can filter out the background noise of legitimate tasks, making the actual attack story much easier for a human analyst to read.

Key Results

Sentient consistently outperformed six state-of-the-art methods (including Unicorn, Flash, and Threatrace):

  • Accuracy & Recall: It achieved 96% accuracy and 99% recall across the DARPA datasets.
  • False Positive Reduction: The most significant result was a 44% average reduction in False Positive Rate (FPR). In cybersecurity, high FPRs lead to "alert fatigue," where analysts start ignoring warnings because so many are false alarms.
  • Efficiency: Despite its complexity, Sentient is fast. It can process a full day's worth of enterprise logs in just 63.6 seconds, making it viable for real-world deployment.
  • Robustness: The system remained effective even when subjected to "mimicry attacks," where attackers try to hide their actions by padding them with benign-looking events.

Limitations and Future Work

While Sentient is powerful, the authors and the methodology suggest a few areas for further exploration:

  • Concept Drift: The model is trained on benign data. If a company updates its software or changes its workflow, "normal" behavior changes. The paper doesn't explicitly detail how the model should be updated over time to prevent a spike in false positives.
  • Initial Training Requirements: While benign data is "easy" to get, the model still requires a clean period of log collection to establish a baseline. If a system is already compromised during the training phase, the model might learn to accept malicious behavior as normal.
  • Clustering Granularity: The attack investigation relies on clustering behaviors. Finding the "sweet spot" for cluster size is vital; too many clusters overwhelm the analyst, while too few might oversimplify a complex multi-stage attack.

Summary updated

Abstract

Advanced Persistent Threats (APTs) are difficult to detect due to their complexity and stealthiness. To mitigate such attacks, many approaches model entities and their relationship using provenance graphs to detect the stealthy and persistent characteristics of APTs. However, existing detection methods suffer from the flaws of missing indirect dependencies, noisy complex scenarios, and missing behavioral logical associations, which make it difficult to detect complex scenarios and effectively identify stealthy threats. In this paper, we propose Sentient, an APT detection method that combines pre-training and intent analysis. It employs a graph transformer to learn structural and semantic information from provenance graphs to avoid missing indirect dependencies. We mitigate scenario noise by combining global and local information. Additionally, we design an Intent Analysis Module (IAM) to associate logical relationships between behaviors. Sentient is trained solely on easily obtainable benign data to detect malicious behaviors that deviate from benign behavioral patterns. We evaluated Sentient on three widely-used datasets covering real-world attacks and simulated attacks. Notably, compared to six state-of-the-art methods, Sentient achieved an average reduction of 44% in false positive rate(FPR) for detection.
BibTeX
@inproceedings{yan2026sentient,
  author = {Wenhao Yan and Ning An and Wei Qiao and Weichang Wu and Bo Jiang and Yuling Liu and Zhigang Lü and Junrong Liu},
  title = {{Sentient: Detecting APTs Via Capturing Indirect Dependencies and Behavioral Logic}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  url = {https://openreview.net/forum?id=MbTmEgfDhu},
}
Figure