Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey

Wei-Chieh Huang, Weizhi Zhang, Yueqing Liang, Yuanchen Bei, Yankai Chen, Tao Feng, Xinyu Pan, Zhen Tan, Yu Wang, Tianxin Wei, Shanglin Wu, Ruiyao Xu, Liangwei Yang, Rui Yang, Wooseong Yang, Chin-Yuan Yeh, Hanrong Zhang, Haozhen Zhang, Siqi Zhu, Henry Peng Zou, Wanjia Zhao, Song Wang, Wujiang Xu, Zixuan Ke, Zheng Hui, Dawei Li, Yaozu Wu, Langzhou He, Chen Wang, Xiongxiao Xu, Baixiang Huang, Juntao Tan, Shelby Heinecke, Huan Wang, Caiming Xiong, Ahmed A. Metwally, Jun Yan, Chen-Yu Lee, Hanqing Zeng, Yinglong Xia, Xiaokai Wei, Ali Payani, Haitong Ma, Wenya Wang, Yu Zhang, Xin Wang, Yongfeng Zhang, Jiaxuan You, Hanghang Tong, Xiao Luo, Yizhou Sun, Wei Wang, McAuley Julian, James Zou, Jiawei Han, Philip S. Yu, Kai Shu, Dohy Faied

Abstract

The research of artificial intelligence is undergoing a paradigm shift from prioritizing model innovations over benchmark scores towards emphasizing problem definition and rigorous real-world evaluation. As the field enters the "second half," the central challenge becomes real utility in long-horizon, dynamic, and user-dependent environments, where agents face context explosion and must continuously accumulate, manage, and selectively reuse large volumes of information across extended interactions. Memory, with hundreds of papers released this year, therefore emerges as the critical solution to fill the utility gap. In this survey, we provide a unified view of foundation agent memory along three dimensions: memory substrate (internal and external), cognitive mechanism (episodic, semantic, sensory, working, and procedural), and memory subject (agent- and user-centric). We then analyze how memory is instantiated and operated under different agent topologies and highlight learning policies over memory operations. Finally, we review evaluation benchmarks and metrics for assessing memory utility, and outline various open challenges and future directions.

Introduction: The Shift to the "Second Half"

Artificial Intelligence development has transitioned from its "first half"—characterized by scaling model parameters and competing on static benchmarks like MMLU—to a "second half" focused on real-world utility. In this new era, agents are expected to navigate long-horizon, dynamic environments where they encounter "context explosion."

Standard Large Language Models (LLMs) are inherently stateless, losing all information once a session concludes. To bridge the gap between benchmark performance and practical utility, agents must possess memory. Whether serving as a personal assistant remembering medical history or a software agent managing complex codebases, the ability to accumulate, manage, and selectively reuse information is paramount. This survey establishes a unified framework for understanding these rapidly evolving memory mechanisms.

A Taxonomy of Foundation Agent Memory

To organize the surge of research in this domain, we propose a three-dimensional taxonomy: Memory Substrate, Cognitive Mechanism, and Memory Subject.

Figure 2: The three-dimensional taxonomy of foundation agent memory, illustrating substrates, cognitive mechanisms, and subjects.

1. Memory Substrate (The "Where")

The substrate refers to the physical or architectural location where information is stored:

  • Internal: Information resides within model weights (via fine-tuning), latent states, or the KV cache. While offering fast access, it is often ephemeral or computationally expensive to update.
  • External: Data is stored in external structures like vector databases, knowledge graphs, or text logs. This approach is highly scalable and easily updated, though it may introduce latency and noise during retrieval.

2. Cognitive Mechanism (The "How")

Drawing inspiration from human psychology, agent memory is categorized by its functional role:

  • Sensory Memory: Brief buffers for raw perceptual inputs (e.g., visual or audio embeddings).
  • Working Memory: An active workspace for immediate reasoning and planning.
  • Long-term Memory: Persistent storage divided into Episodic (records of past events), Semantic (stable facts and concepts), and Procedural (skills and workflows).

3. Memory Subject (The "Who")

  • User-Centric: Focuses on personalization, user preferences, and privacy.
  • Agent-Centric: Focuses on the agent's own experiences, skill acquisition, and internal world modeling.

Evaluation and Benchmarking

Evaluating memory utility requires moving beyond static datasets. Current benchmarks are categorized into:

  • User-Centric Benchmarks: Datasets such as MSC (Multi-Session Chat) and MemoryBank test an agent's ability to maintain a consistent persona and recall user-specific facts over extended periods.
  • Agent-Centric Benchmarks: Environments like WebArena and SWE-bench require agents to leverage memory to solve multi-step tasks in complex software or web environments.

The survey identifies several critical shifts in the landscape of agent memory research.

Figure 5: Cumulative growth of research papers categorized by memory substrate, cognitive mechanism, and subject from 2023 to 2025.
  • Exponential Growth: Research activity has surged, with a notable peak in late 2024 and early 2025. As shown in the cumulative counts, external substrates and agent-centric subjects currently dominate the research landscape.
  • Learning to Remember: There is a transition from hand-crafted prompting to "Memory Learning Policies." Agents are increasingly trained via Reinforcement Learning (RL) to autonomously manage their storage, deciding what to summarize, retain, or prune.
  • The Context Window Fallacy: While context windows are expanding, they are not a substitute for structured memory. Models without memory management often suffer from the "lost in the middle" phenomenon and prohibitive computational costs.
  • Hybrid Substrates: State-of-the-art agents predominantly utilize hybrid substrates—combining internal memory for immediate logic with external stores for long-term retrieval.

Open Challenges and Future Directions

Despite rapid progress, several frontiers remain:

  • Infrastructure Efficiency: Current Retrieval-Augmented Generation (RAG) systems face latency and cost issues, necessitating hardware-software co-design.
  • Trustworthy Memory: Ensuring privacy and implementing "Machine Unlearning" (the ability to reliably forget) are critical for user-centric applications.
  • Multimodal Grounding: Memory must evolve beyond text to include visual and physical procedural memories, essential for robotics.
  • Life-long Learning: We lack benchmarks that evaluate agents over truly continuous, life-long interactions without resetting their state.

Summary updated

Abstract

The research of artificial intelligence is undergoing a paradigm shift from prioritizing model innovations over benchmark scores towards emphasizing problem definition and rigorous real-world evaluation. As the field enters the "second half," the central challenge becomes real utility in long-horizon, dynamic, and user-dependent environments, where agents face context explosion and must continuously accumulate, manage, and selectively reuse large volumes of information across extended interactions. Memory, with hundreds of papers released this year, therefore emerges as the critical solution to fill the utility gap. In this survey, we provide a unified view of foundation agent memory along three dimensions: memory substrate (internal and external), cognitive mechanism (episodic, semantic, sensory, working, and procedural), and memory subject (agent- and user-centric). We then analyze how memory is instantiated and operated under different agent topologies and highlight learning policies over memory operations. Finally, we review evaluation benchmarks and metrics for assessing memory utility, and outline various open challenges and future directions.
BibTeX
@article{huang2026rethinking,
  author = {Wei-Chieh Huang and Weizhi Zhang and Yueqing Liang and Yuanchen Bei and Yankai Chen and Tao Feng and Xinyu Pan and Zhen Tan and Yu Wang and Tianxin Wei and Shanglin Wu and Ruiyao Xu and Liangwei Yang and Rui Yang and Wooseong Yang and Chin-Yuan Yeh and Hanrong Zhang and Haozhen Zhang and Siqi Zhu and Henry Peng Zou and Wanjia Zhao and Song Wang and Wujiang Xu and Zixuan Ke and Zheng Hui and Dawei Li and Yaozu Wu and Langzhou He and Chen Wang and Xiongxiao Xu and Baixiang Huang and Juntao Tan and Shelby Heinecke and Huan Wang and Caiming Xiong and Ahmed A. Metwally and Jun Yan and Chen-Yu Lee and Hanqing Zeng and Yinglong Xia and Xiaokai Wei and Ali Payani and Yu Wang and Haitong Ma and Wenya Wang and Chengguang Wang and Yu Zhang and Xin Wang and Yongfeng Zhang and Jiaxuan You and Hanghang Tong and Xiao Luo and Yizhou Sun and Wei Wang and McAuley Julian and James Zou and Jiawei Han and Philip S. Yu and Kai Shu and Dohy Faied},
  title = {{Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey}},
  journal = {Trans. Mach. Learn. Res.},
  year = {2026},
}
Figure