AdMem: Expectation-Grounded Procedural Memory for Long-Horizon Task-Solving Agents

Runzhe Wang, Huilin Lu, Shengjie Liu, Li Dong, Zhihuai ZhuarXiv 2026 · arXiv

TL;DR

AdMem introduces a "forgetting and consolidation" policy for LLM agents, proving that naively accumulating past experiences can actually make an agent perform worse than having no memory at all. By requiring agents to state their expectations before acting and using a critic to score the results, AdMem creates a high-utility procedural memory that significantly outperforms memoryless baselines across complex, long-horizon tasks.

What Problem Does This Paper Address?

In the quest to build autonomous agents, researchers have long assumed that more memory is better. However, the authors uncover an "uncomfortable observation": simply storing every past interaction (workflow induction) often leads to "memory interference," where an agent gets distracted by irrelevant or flat-out wrong past experiences. On the AgentBoard benchmark, a standard memory-augmented agent actually performed worse than a memoryless one in seven out of eight domains.

The core challenges are credit assignment (knowing exactly which action led to a success or failure) and memory management (knowing what to keep, what to merge, and what to delete). Without these, an agent’s context window becomes a junkyard of "hallucinated" best practices that don't generalize.

Methods

AdMem (Expectation-Grounded Procedural Memory) introduces a structured life cycle for memory entries using three concurrent modules:

  1. The Actor: An LLM that interacts with the environment. Crucially, before taking any action, it must commit to an expected outcome.
  2. The Critic: A second LLM instance that watches the "realized outcome." It compares what happened to what the Actor expected, assigning a binary reward and writing a "reflection" (procedural entry). This allows for per-action credit assignment without needing a reward signal from the environment itself.
  3. Long-Term Memory (LTM) Store: Instead of just using vector similarity for retrieval, AdMem assigns each entry a scalar utility score. This utility is updated online using a noisy-OR reward model—a probabilistic approach that estimates how likely a specific memory was to have caused a successful action.

The system also implements a "Short-Term Memory" stack, similar to a program's call stack. When an agent starts a sub-goal, it pushes a new frame; when finished, it pops and summarizes that frame, preventing the context window from exploding during long tasks.

Datasets

The authors evaluated AdMem using AgentBoard, a comprehensive analytical benchmark for multi-turn LLM agents. They tested across 707 tasks in eight distinct domains:

  • Embodied/Grid-world: AlfWorld, BabyAI.
  • Interactive Fiction: Jericho.
  • Symbolic Planning: PDDL.
  • Tool Use: Tool-query (Academic and Weather).
  • Web/Science: WebShop, ScienceWorld.

Key Insights and Ideas

  • Forgetting is a Feature, Not a Bug: The most striking insight is that a memory store without an eviction policy is a liability. AdMem’s ability to "forget" low-utility entries is what prevents the performance collapse seen in other systems.
  • Expectation-Grounded Credit: By judging an action against a pre-stated goal rather than an after-the-fact success, the system avoids "lucky" successes and penalizes "unlucky" failures more accurately.
  • Human-Readable "Auditable Surface": All memories are stored as plain text. This allowed the researchers to diagnose exactly why the agent failed (e.g., reading a bad memory entry) and suggests that future agents could be audited by humans for safety.
  • The "Negative Half" Result: The paper highlights that "Workflow Induction" (AWM) solved 76 fewer tasks than a memoryless agent. This serves as a major warning to the field against naive RAG (Retrieval-Augmented Generation) for procedural tasks.

Key Results

AdMem demonstrated a clear advantage in domains with transferable structure:

  • Overall Success: Across 707 tasks, AdMem solved 441, compared to 388 for the memoryless ReAct baseline and only 312 for the workflow-induction (AWM) baseline.
  • PDDL (Planning): AdMem solved 76.7% of tasks vs. ReAct’s 33.3%, showing that procedural memory is incredibly powerful when action schemas are consistent.
  • Jericho: In this complex text-adventure domain, AdMem improved its performance over multiple "epochs" (repeated attempts), whereas simpler memory systems actually got worse as they accumulated more "junk" data.

Limitations and Future Work

  • Computational Cost: AdMem is not "cheap." It requires roughly 3x the LLM calls and 2x the wall-clock time of a vanilla agent due to the concurrent Critic and planning steps.
  • Security Risks: Because the memory is human-readable and derived from environment observations, it is vulnerable to indirect prompt injection. An adversary could potentially "poison" the agent's memory by placing specific text in the environment that the Critic then saves as a "successful" procedure.
  • Backbone Dependency: The study primarily used Claude Haiku 4.5. While the architecture is model-agnostic, performance might vary significantly with smaller or less capable models.
  • Future Directions: The authors suggest exploring more robust defenses against memory poisoning and refining the "noisy-OR" model to handle conflicting or overlapping memory entries more gracefully.

Summary updated

Abstract

An agent that carries memory across tasks is only as good as the entries it keeps. We begin from an observation that is easy to reproduce and uncomfortable for the field: naively accumulating cross-task experience can make an agent \emph{worse} than one with no memory at all. On AgentBoard, a workflow-induction memory solves fewer tasks than a memoryless ReAct agent in seven of eight domains, and a per-step procedural store with no surrounding structure loses half of ReAct's completions on Jericho. We argue that the missing ingredients are per-action credit assignment and an explicit life cycle for stored entries, and present AdMem, which supplies both. Before each action the actor commits to an expected outcome; a concurrently running critic scores the realized outcome against that expectation and writes a procedural entry carrying a binary reward and a reflection, so credit lands on the action that earned it and no environment reward is required. Each entry then carries a scalar utility, updated online under a noisy-OR reward model, that ranks retrieval jointly with context similarity and drives eviction and merging, giving the store a forgetting and consolidation policy instead of unbounded growth. Across 707 tasks in eight AgentBoard domains AdMem solves 441 against 388 for ReAct and 312 for the workflow-induction baseline --- a pooled gain significant under a two-sided Fisher exact test, concentrated in the domains that supply transferable cross-task structure. All memory stays human-readable text, which is what makes the failures above diagnosable --- and, as we discuss, also makes the store an auditable but attackable surface.
BibTeX
@misc{wang2026admem,
  author = {Runzhe Wang and Huilin Lu and Shengjie Liu and Li Dong and Zhihuai Zhu},
  title = {{AdMem: Expectation-Grounded Procedural Memory for Long-Horizon Task-Solving Agents}},
  year = {2026},
  eprint = {2611.20605},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20605},
}
Figure