Securing Autonomous Agents Against Persistent Memory Poisoning

This direction is split into other directions: persistent memory poisoning. Its last version stays readable and citable here.

Version 1 of 1119 papers237 conceptsfrozen 2026-10-04

Traditional Large Language Model (LLM) security has focused on the "stateless" interaction: a user provides a prompt, and the model generates a response. In this paradigm, a "prompt injection" is a transient event. If you clear the chat, the threat is gone.

However, as we move toward autonomous agents, this assumption of statelessness is breaking. Modern agents use long-term memory, vector databases, and persistent logs to maintain context across multiple sessions. Research across 62 papers reveals that this persistent state creates a massive, unprotected attack surface. Attackers are shifting from trying to "trick" the model in the moment to "reprogramming" the agent's long-term behavior through Persistent Memory Poisoning.

The Vulnerability of Trusted History

The core reason these attacks work is a fundamental design choice in agentic architectures: agents are designed to trust their own history. To maintain consistency, an agent retrieves past interactions from a database and treats them as authoritative "prior reasoning" or "ground truth."

Pramod Viswanath and colleagues demonstrate this in Context manipulation attacks : Web agents are susceptible to corrupted memory, introducing a technique called Plan Injection. By corrupting the high-level task plan stored in an agent's external memory, an attacker can redirect the agent's reasoning without ever touching the initial user prompt. This method is roughly three times more effective than standard prompt injection because it bypasses input filters entirely; the malicious instruction is retrieved from the agent's own "trusted" storage.

This vulnerability is particularly acute in financial ecosystems. In AI Agents in Cryptoland: Practical Attacks and No Silver Bullet, researchers Pramod Viswanath, Prateek Mittal, and S. Ashwin Hebbar show that poisoning an agent’s long-term memory can trigger unauthorized cryptocurrency transfers that persist across different platforms like Discord and X.

Delayed Exploitation and Sleeper Cells

Because memory persists, an attack no longer needs to be immediate. Hang Su and others describe this in A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents as delayed exploitation. An attacker can plant "sleeper cell" instructions—seemingly benign information that is stored in the agent's memory—which only trigger a malicious action when a specific future condition is met.

This creates a "logic-layer" threat. As identified in Logic layer Prompt Control Injection (LPCI): A Novel Security Vulnerability Class in Agentic Systems, these payloads can remain dormant in vector stores for weeks, bypassing real-time safety filters that only inspect the current turn of conversation.

The Self-Reinforcing Error Cycle

A particularly stealthy mechanism is the recursive reliance on historical logs. In Memory Injection Attacks on LLM Agents via Query-Only Interaction, Shen Dong and colleagues introduce MINJA, showing that agents often treat their own past outputs as "gold standard" demonstrations for future reasoning.

If an attacker successfully injects a single malicious record into the history, the agent may "learn" from that record, using it as a template for how to handle similar queries in the future. Qianshan Wei notes in A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory that this creates a self-reinforcing error cycle: the agent's mistake is stored as a valid precedent, making the agent progressively more susceptible to the same adversarial manipulation over time.

From Single Users to Multi-Agent Infections

The risk scales exponentially when agents interact with each other or shared environments. Pramod Viswanath and Prateek Mittal introduce the concept of Cross-User Poisoning (CUP) in MURMUR, where malicious rules injected into a shared Slack or Discord history are blindly applied by the agent to other, unsuspecting users.

Even more alarming is the concept of Prompt Infection proposed by Mo Tiwari in Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems. Here, a malicious prompt acts like a virus, containing self-replicating instructions that force one agent to "infect" the memory of another agent, allowing a compromise to spread silently across an entire network of AI systems.

Emerging Defenses: Isolation and Verification

Standard prompt filters are insufficient for stateful agents because they cannot distinguish between a legitimate historical context and a poisoned one. New research suggests several shifts in defense strategy:

  1. Isolated Planning: Shanghao Shi and colleagues propose Tool-Guard in Think Twice Before You Act, which uses "isolated planning" to quarantine suspicious tools and prevent poisoned descriptions from influencing the agent's overall trajectory.
  2. Zero-Trust Architectures: Ken Huang argues in Fortifying the Agentic Web for a move toward Zero-Trust for agentic memory, where retrieved context is never implicitly trusted and must be dynamically monitored for behavioral anomalies.
  3. Verify-then-Freeze: In VerificAgent, Yash Jain suggests a pipeline where memory entries are fact-checked and then "frozen" (made read-only) during deployment to prevent adversarial injections from permanently altering the agent's procedural knowledge.

As agents become more capable, the "inverse scaling law" identified in AgentDojo by Edoardo Debenedetti and Marc Fischer suggests they will only become more vulnerable to these attacks. Securing the "brain" of the agent is no longer enough; we must secure the "memory" that informs its every move.

Papers in this version

Showing 60 of 119 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_securing_autonomous_agents_against_persi_32724783,
  title = {Securing Autonomous Agents Against Persistent Memory Poisoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/securing-autonomous-agents-against-persistent-memory-poisoning/rdr_3272478389dc4c578f0d77809f2fa2f9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_securing_autonomous_agents_against_persi_32724783_v1,
  title = {Securing Autonomous Agents Against Persistent Memory Poisoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/securing-autonomous-agents-against-persistent-memory-poisoning/rdr_3272478389dc4c578f0d77809f2fa2f9/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}