Abstract
Large language model (LLM) agents increasingly rely on accumulated memory to solve long-horizon decision-making tasks. However, most existing approaches store memory in fixed representations and reuse it at a single or implicit level of abstraction, which limits generalization and often leads to negative transfer when distribution shift. This paper proposes the Meta-Cognitive Memory Abstraction method (MCMA), which treats memory abstraction as a learnable cognitive skill rather than a fixed design choice. MCMA decouples task execution from memory management by combining a frozen task model with a learned memory copilot. The memory copilot is trained using direct preference optimization, it determines how memories should be structured, abstracted, and reused. Memories are further organized into a hierarchy of abstraction levels, enabling selective reuse based on task similarity. When no memory is transferable, MCMA transfers the ability to abstract and manage memory by transferring the memory copilot. Experiments on ALFWorld, ScienceWorld, and BabyAI demonstrate substantial improvements in performance, out-of-distribution generalization, and cross-task transfer over several baselines.
Introduction
While Large Language Model (LLM) agents have become adept at following instructions, they struggle with long-horizon tasks that require "procedural memory"—the ability to effectively reuse past experiences. Current approaches usually rely on fixed memory formats, such as raw trajectory logs or simple text summaries. This creates a "reuse dilemma": fine-grained memories (step-by-step logs) often overfit to specific environments and fail when a room layout or object changes, while overly abstract memories (high-level summaries) often lack the actionable detail needed to actually complete a task.
Furthermore, when an agent moves to a completely new domain (e.g., from a kitchen to a laboratory), its stored memories often become useless. This research argues that memory management should be treated as a meta-cognitive skill—a learnable ability to decide how to represent and organize experience—rather than a hard-coded design choice.
The Meta-Cognitive Memory Abstraction (MCMA) Framework
The core innovation of this work is the Memory Copilot, a separate, trainable model that manages memory for a frozen Task Model. The system operates in a four-stage pipeline: Collect, Generate, Evolve, and Reuse.
- Collect: The task model performs actions, generating raw trajectories of successes and failures.
- Generate: The memory copilot transforms these raw logs into multi-structural representations. Instead of just text, it uses a mix of trees (for hierarchies), chains (for sequences), and key-value pairs (for object locations).
- Evolve: This is the "meta-cognitive" training phase. The copilot is trained using Direct Preference Optimization (DPO). It generates multiple candidate memory abstractions for a task; the one that leads to the most efficient successful execution is labeled "preferred."
- Reuse: Memories are organized into a hierarchy of abstraction levels. When facing a new task, the agent retrieves the most relevant level of detail—using specific episodic details for familiar tasks and high-level "scripts" for novel ones.
Crucially, if the agent encounters a domain where it has zero prior experience, it can transfer the memory copilot itself. The agent doesn't bring the content of its old memories, but it brings the skill of how to observe, abstract, and organize new ones.
Experimental Evaluation
The researchers evaluated MCMA on three distinct embodied reasoning benchmarks:
- ALFWorld: A text-based environment for household tasks (e.g., "put a cool mug in the cabinet"). It includes "seen" and "unseen" room configurations.
- ScienceWorld: A complex environment requiring multi-step scientific reasoning and interaction with lab equipment.
- BabyAI: A 2D gridworld used specifically to test cross-domain transfer, as its 2D grid observations are fundamentally different from the textual descriptions in the other two datasets.
Key Results
- Performance Gains: On ALFWorld (using Qwen3-8B), MCMA improved success rates by nearly 25% over the "No Memory" baseline and significantly outperformed existing methods like ExpeL and TRAD.
- Efficiency: MCMA consistently reduced the number of steps required to complete tasks, indicating cleaner, more decisive planning.
- Zero-Shot Transfer: In a significant test of generalization, a memory copilot trained on ALFWorld was moved to ScienceWorld. It outperformed the baseline significantly, proving that the method of remembering is transferable even when the facts are not.
- Robustness: The system showed strong "out-of-distribution" generalization, maintaining high success rates even in rooms and task types it had never seen during training.
Discussion and Insights
The most significant shift presented in this work is treating memory as a skill rather than a database. The paper finds that "Tree" and "Chain" structures are far more effective for long-horizon planning than raw text. Trees provide high-level directional guidance, while nested chains handle the execution flow. Furthermore, the "Mix Copilot" (trained on both ALFWorld and ScienceWorld) performed better than copilots trained on either alone, suggesting that the ability to abstract is a generalizable intelligence trait that improves with diverse data.
However, limitations remain. Training the copilot is computationally expensive, requiring the generation and evaluation of multiple candidate abstractions to create DPO training pairs. Additionally, while memories are organized hierarchically, the current system relies on a manually designed strategy to decide which level of abstraction to retrieve. Future work will focus on automating this selection process and exploring cross-architecture transferability.
Summary updated