A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents

Wei-Chieh Huang, Weizhi Zhang, Yueqing Liang, Yuanchen Bei, Yankai Chen, Tao Feng, Xinyu Pan, Zhen Tan, Yu Wang, Tianxin Wei, Shanglin Wu, Ruiyao Xu, Liangwei Yang, Rui Yang, Wooseong Yang, Chin-Yuan Yeh, Hanrong Zhang, Haozhen Zhang, Siqi Zhu, Henry Peng Zou, Wanjia Zhao, Song Wang, Wujiang Xu, Zixuan Ke, Zheng Hui, Dawei Li, Yaozu Wu, Langzhou He, Chen Wang, Xiongxiao Xu, Baixiang Huang, Juntao Tan, Shelby Heinecke, Huan Wang, Caiming Xiong, Ahmed Metwally, Jun Yan, Chen-Yu Lee, Hanqing Zeng, Yinglong Xia, Xiaokai Wei, Ali Payani, Haitong Ma, Wenya Wang, Yu Zhang, Xin Eric Wang, Yongfeng Zhang, Jiaxuan You, Hanghang Tong, Xiao Luo, Xue Liu, Yizhou Sun, Wei Wang, Julian J. McAuley, James Zou, Jiawei Han, Philip S. Yu, Kai Shu

TL;DR

This survey marks the transition of AI research from model scaling to real-world utility, identifying memory as the critical substrate for self-evolving, long-horizon agents. By introducing a comprehensive three-dimensional taxonomy—substrate, mechanism, and subject—the authors provide a unified framework for understanding how agents can move beyond fixed context windows to manage vast amounts of information across extended interactions.

What Problem Does This Paper Address?

The "first half" of AI development focused on scaling data and parameters, leading to Large Language Models (LLMs) that can score 90%+ on standard benchmarks. However, these models often fail in the "second half": real-world, long-horizon tasks like agentic coding, deep research, and complex computer use.

The core challenge is context explosion. LLMs are inherently stateless and limited by fixed context windows. In dynamic environments, agents must continuously accumulate, prune, and selectively reuse information. Without a robust memory system, long-horizon interaction degenerates into a series of isolated, one-shot tasks. The authors argue that memory is not just passive storage; it is the fundamental mechanism through which agents learn from experience and self-evolve.

Methods

The authors analyzed 218 key articles published between 2023 and 2025 to build a unified taxonomy of foundation agent memory. This taxonomy is organized along three core dimensions:

  1. Memory Substrate (The "What"): Where is the information stored?
  2. Internal: Stored in model weights (parametric), latent states, or KV caches.
  3. External: Stored in vector indices, text records, or structural/hierarchical databases (non-parametric).
  4. Cognitive Mechanism (The "How"): What functional role does it play? Inspired by human psychology, this includes Sensory (brief perceptual buffers), Working (active manipulation of current context), Episodic (records of specific experiences), Semantic (abstract facts and knowledge), and Procedural (reusable skills and routines) memory.
  5. Memory Subject (The "Who"): Whom does the memory serve?
  6. User-Centric: Focuses on personalization, user facts, and preferences.
  7. Agent-Centric: Focuses on the agent’s own trajectories, outcomes, and learned heuristics.
The taxonomy of Foundation Agent Memory, categorized by Substrate, Mechanism, and Subject.

The paper also details Memory Operation Mechanisms (storage, retrieval, update, compression, and forgetting) and Optimization Policies, showing how agents are increasingly trained via Reinforcement Learning (RL) to manage their own memory systems.

Datasets

The survey reviews a wide array of benchmarks used to evaluate memory utility:

  • User-Centric Benchmarks: MSC, DuLeMon, and MemoryBank evaluate conversational consistency and persona retention.
  • Agent-Centric Benchmarks: WebArena, OSWorld, and GAIA focus on functional problem-solving and tool use.
  • Newer Evaluations: The authors highlight a shift toward metrics like Memory Integrity (MI) (completeness of extraction) and False Memory Rate (FMR) (hallucination rate during updates).

Key Insights and Ideas

  • Memory as the Substrate for Evolution: Memory is the loop through which agents improve. Short-term memory gates what is perceived, while long-term memory consolidates these experiences into reusable knowledge.
  • The Shift to Learned Policies: We are moving away from hand-coded heuristics (like "delete the oldest info") toward trainable memory management. Agents are now using RL to decide what to store and when to forget to maximize long-term rewards.
  • Procedural Memory as Shareable Skills: A major trend is the externalization of procedural memory into "skill libraries" (e.g., Voyager). These are portable, versionable code snippets that agents can load on demand and share with other agents.
  • The Utility Gap: There is a persistent mismatch between high benchmark scores and real-world performance. The authors advocate for "execution-grounded" evaluation that stresses memory under realistic constraints.
A timeline illustrating the rapid acceleration of foundation agent memory frameworks released between 2023 and 2025.

Key Results

The paper documents a massive surge in research, with a significant acceleration in 2025.

  • Substrate Trade-offs: Internal memory offers high-speed access but risks "catastrophic forgetting" during updates. External memory is highly scalable and easy to update but introduces retrieval latency and noise.
  • Subject Trends: While early work was split, there is a growing dominance of agent-centric research (146 papers) compared to user-centric research (72 papers) as the field pushes toward autonomous problem-solving.
Cumulative publication trends showing the rapid acceleration of research in 2025, particularly in agent-centric memory.

Limitations and Future Work

The authors identify several critical gaps:

  • Multimodal Integration: Unifying vision, audio, and action into a grounded memory remains a major challenge. Current systems are often "text-centric" and struggle with high-dimensional perceptual data.
  • Privacy and Trust: Persistent user memory introduces risks of data leakage and "memory poisoning." Future work must focus on verifiable architectures and user-controlled revocation of stored memories.
  • World Models: The authors suggest elevating memory from a passive log to an explicit, predictive world model where agents can simulate the consequences of an action before committing it to memory.
  • Efficiency: Unbounded logs are too costly. There is a pressing need for hybrid parametric/non-parametric designs that provide high-quality retrieval without exploding token costs.

Summary updated

Abstract

Research in artificial intelligence is shifting from model innovations and benchmark scores towards problem definition and rigorous real-world evaluation. As the field enters the "second half," the central challenge becomes real utility in long-horizon, dynamic, and user-dependent settings such as agentic coding, deep research, and computer use, where LLM-based agents face context explosion beyond fixed context windows and must continuously accumulate, manage, and selectively reuse information across extended interactions. Memory, with hundreds of papers released in 2025, therefore emerges as the critical solution to fill this utility gap. Beyond passive storage, memory is increasingly the substrate through which agents self-evolve: short-term memory gates which experiences are perceived and abstracted during execution, while long-term memory consolidates them into reusable knowledge and skills, forming the loop through which agents improve from their own experience. In this survey, we provide a unified view of foundation agent memory along three dimensions: memory substrate (internal parametric state and external retrieval-augmented stores), cognitive mechanism (sensory, working, episodic, semantic, and procedural), and memory subject (user-centric personalization and agent-centric experience). We then analyze how memory is operated under single- and multi-agent topologies and highlight learning policies over memory operations, showing how memory management itself is becoming a trainable capability spanning reinforcement-learned context curation, experience consolidation at decision time, and the emerging ecosystem of portable, shareable agent skills. Finally, we review evaluation benchmarks and metrics for memory utility, and outline open challenges and future directions.
BibTeX
@article{huang2026survey,
  author = {Wei-Chieh Huang and Weizhi Zhang and Yueqing Liang and Yuanchen Bei and Yankai Chen and Tao Feng and Xinyu Pan and Zhen Tan and Yu Wang and Tianxin Wei and Shanglin Wu and Ruiyao Xu and Liangwei Yang and Rui Yang and Wooseong Yang and Chin-Yuan Yeh and Hanrong Zhang and Haozhen Zhang and Siqi Zhu and Henry Peng Zou and Wanjia Zhao and Song Wang and Wujiang Xu and Zixuan Ke and Zheng Hui and Dawei Li and Yaozu Wu and Langzhou He and Chen Wang and Xiongxiao Xu and Baixiang Huang and Juntao Tan and Shelby Heinecke and Huan Wang and Caiming Xiong and Ahmed Metwally and Jun Yan and Chen-Yu Lee and Hanqing Zeng and Yinglong Xia and Xiaokai Wei and Ali Payani and Yu Wang and Haitong Ma and Wenya Wang and Chenguang Wang and Yu Zhang and Xin Eric Wang and Yongfeng Zhang and Jiaxuan You and Hanghang Tong and Xiao Luo and Xue Liu and Yizhou Sun and Wei Wang and Julian J. McAuley and James Zou and Jiawei Han and Philip S. Yu and Kai Shu},
  title = {{A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents}},
  journal = {Trans. Mach. Learn. Res.},
  year = {2026},
}
Figure