As Large Language Model (LLM) agents move from simple chatbots to sophisticated assistants with persistent memory, they face a subtle but dangerous threat: memory drift. This occurs when an adversarial or misleading record in an agent's memory bank silently hijacks its execution path, leading to harmful outcomes like data leaks or task failure.
A new paper from researchers at Zhejiang University, Peking University, and Alibaba Group introduces DriftGuard, a framework designed to detect and neutralize these drift-inducing memories in real-time.
TL;DR
DriftGuard protects LLM agents from "memory drift" by using offline counterfactual simulations to train a lightweight online model that identifies which retrieved memories will causally disrupt an agent's behavior. By decoupling expensive causal attribution from real-time serving, it accurately flags and neutralizes harmful memories with minimal latency overhead.
What Problem Does This Paper Address?
Modern LLM agents rely on persistent memory to maintain context across multi-step tasks. However, this memory is a massive attack surface. Adversaries can plant "poisoned" memories—via tool calls, web content, or dormant "sleeper" payloads—that only activate under specific conditions.
The authors identify two "fundamental challenges" in stopping this:
- Content vs. Causality: You can't find a "bad" memory just by looking at it. A memory that looks suspicious (high perplexity) might be harmless, while a perfectly normal-looking one might be the actual "true causal" trigger for a hijack.
- The Latency Bottleneck: The gold standard for proving causality is "counterfactual attribution"—running the agent's task multiple times, each time removing one candidate memory to see if the behavior changes. This is prohibitively expensive for real-time applications.
Methods
DriftGuard solves this by decoupling offline causal supervision from online estimation.
Stage 1: Offline Causal Metric Construction
The researchers first build a dataset by performing repeated, paired counterfactual rollouts. For a given task, they run the agent normally, then run it again after "neutralizing" (rewriting to be benign) a single retrieved memory.
They use these runs to calculate a Memory Drift Score (MDS), a four-dimensional vector capturing:
- Action Drift: Did the next tool call or argument change? (See Figure 3).
- Attack Removal: Did neutralizing the memory stop a successful attack?
- Violation Removal: Did it prevent a policy violation?
- Task Recovery: Did it allow a previously failing benign task to succeed? (See Figure 4).
Stage 2: Online Risk Model (ORM)
Using this MDS-labeled dataset, they train an Online Risk Model. This is a distilled model (based on Qwen2.5-7B) that learns to predict the MDS components directly from "pre-action" information (the task query, execution history, and the candidate memory itself).
At runtime, the ORM scores every retrieved memory in a single forward pass. If any memory's risk score exceeds a threshold, DriftGuard neutralizes it before the agent ever sees it, preventing the drift before it happens.
Datasets
The authors evaluated DriftGuard across several rigorous benchmarks:
- Offline Detection: MINJA (clinical tool use), AgentPoison (autonomous driving), and MPBench (covering six attack classes).
- Online Defense: Mem2ActBench (multi-turn tool use), STATE-Bench (long-horizon workflows), and DBBench (database operations).
Key Insights and Ideas
- Causality over Content: The core "aha" moment is that safety shouldn't just be about filtering "bad" content, but about understanding the causal impact of that content on the agent's specific task.
- Amortized Intervention: By moving counterfactuals offline, DriftGuard "amortizes" the cost of causal reasoning into a fast, learnable risk estimator.
- Decomposed Supervision: Rather than a single "is this memory bad?" score, the 4D MDS provides much richer supervisory signals, making the model more robust across different types of attacks (e.g., direct overrides vs. stealthy sleeper memories).
Key Results
- Superior Detection: DriftGuard significantly outperformed frontier LLM rankers (like GPT-4 and Claude 3.5). On the difficult MPBench, DriftGuard achieved an AUPRC of 0.2415, outperforming Claude Opus 5 (Few-shot) by 86.2%.
- Effective Defense: In online settings, it drastically reduced the Attack Success Rate (ASR). For example, against "Sleeper Memory" attacks on Mem2ActBench, ASR dropped from 77.41% (no defense) to just 3.25%.
- High Efficiency: DriftGuard is roughly 20x faster than performing actual online counterfactual replays and adds only about 1.5 seconds of latency per task.
Limitations and Future Work
The authors acknowledge a few key limitations:
- Single-Memory Focus: The current offline supervision only intervenes on one memory at a time, potentially missing "collusive" attacks where multiple memories work together to cause drift.
- Data Intensity: Building the offline supervision dataset still requires significant compute for the thousands of counterfactual rollouts.
- Domain Specificity: While effective on tool-using agents, more work is needed to see how well these causal metrics generalize to broader, less structured agentic tasks.
Summary updated