TL;DR
MemCompiler introduces State-Conditioned Memory Compilation (SCMC), a novel paradigm that dynamically selects and compiles relevant cross-episode memories for embodied agents based on their real-time execution state. By delivering memory through both text and latent channels, it eliminates the "attention dilution" caused by dumping full memories upfront, boosting task success rates by up to 129% while slashing per-step latency by 60%.
What Problem Does This Paper Address?
For embodied agents operating in complex, partially observable environments, memory is crucial for transferring skills across episodes and avoiding repetitive failures. However, current systems rely on Ahead-of-time Monolithic Memory Injection (AMMI), where all retrieved memory entries are dumped into the agent's context window at the start of an episode and kept static.
As the agent executes a task, its state continuously evolves, causing this static memory block to become misaligned with current needs. This leads to attention dilution, where the agent's lightweight policy executor gets overwhelmed trying to filter out irrelevant memory tokens from valuable ones. In fact, the authors show that under the AMMI paradigm, an executor's attention to memory tokens drops drastically over time, sometimes causing the agent to perform worse than if it had no memory at all. Simple rule-based progressive disclosure fails here because memory relevance in embodied tasks depends on complex, multimodal execution states rather than explicit text queries.
Methods
To solve this, the authors propose MemCompiler, drawing an analogy to classical software compilation: task memory acts as source code, the agent's current state is the runtime context, and a learned Memory Compiler dynamically translates relevant memory entries into actionable guidance at each decision step.
The architecture consists of two main components:
- Brief State ($b_t$): A structured, two-dimensional runtime context maintained by the compiler. It tracks Task Progress State (completed, current, and pending subgoals) and Environment Belief (known object locations and agent status).
- Dual-Channel Output Mechanism:
- Text Channel ($m^{*,t}_{\text{text}}$): Provides interpretable, step-by-step strategic guidance.
- Soft-Mem Channel ($m^{*,t}_{\text{soft}}$): Projects latent soft tokens directly into the Executor's embedding space. This channel captures spatial and perceptual knowledge that is difficult to express purely in text.
At each step, the Memory Compiler chooses between four output actions: EXPERIENCE (provide guidance), BRIEF (update the state), HYBRID (do both), or NOACTION (do nothing if no memory applies).
Training Pipeline: MemCompiler is trained in two stages. First, a Supervised Fine-Tuning (SFT) stage uses a teacher model to generate trajectories, optimizing text and action losses alongside a complementarity constraint (orthogonality loss) to prevent the latent channel from simply duplicating the text channel. Second, a Reinforcement Learning (RL) stage uses Group Relative Policy Optimization (GRPO) with sparse, binary task-success rewards to sharpen the compiler's decision policy.
Datasets
The authors evaluated MemCompiler across three distinct benchmarks:
- AlfWorld: A text-based interactive environment featuring 134 unseen household tasks across six categories.
- EmbodiedBench: A multimodal benchmark consisting of two environments: EB-ALFRED (household tasks in the AI2-THOR simulator) and EB-Habitat (physical manipulation tasks using a Fetch robot).
- ScienceWorld: A text-based interactive simulator featuring 30 types of complex science experiments.
Strict "zero-leakage" mechanisms were enforced to ensure complete separation between training and evaluation sets regarding scene layouts, object categories, and instructions.
Key Insights and Ideas
- Stable Attention via Compilation: Visualizing attention weights confirms that AMMI suffers from severe monotonic decay in memory attention as episodes progress. SCMC successfully maintains stable, high attention throughout the entire task execution.
- Delivery Over Raw Capacity: A mid-sized open-source model (e.g., Qwen-3.5-27B) equipped with MemCompiler can match or exceed the performance of frontier closed-source models (like Gemini-3-flash or GPT-5.2) operating without memory. This suggests that the bottleneck for smaller open-source executors lies less in raw capacity and more in how memory is delivered.
- Orthogonal Latent Semantics: The complementarity constraint effectively forces the text and soft-memory channels to decouple. t-SNE visualizations show that the tokens from each channel occupy completely non-overlapping regions, proving that Soft-Mem captures distinct perceptual knowledge that text cannot convey.
Key Results
- Performance Boosts: MemCompiler achieved substantial relative success rate improvements over no-memory baselines across all open-source backbones, reaching up to +110% on EB-ALFRED and +129% on ScienceWorld.
- Efficiency Gains: On the AlfWorld benchmark, SCMC reduced the executor's input tokens by over 50% (from 2481.1 down to 1003.2 tokens) and cut per-step executor latency from 0.30s to 0.12s, achieving a 60% speedup.
- Ablation Validation: Removing the Soft-Mem channel caused a sharp performance drop in spatial and visual tasks, while removing the structured Brief State (and using raw observation history instead) led to severe degradation, confirming both components are vital.
Limitations and Future Work
- Minimal Memory Curation: To isolate the effects of memory delivery, the authors kept the memory bank deliberately minimal, using raw successful trajectories. Integrating advanced memory curation, distillation, or hierarchical indexing could further enhance the system.
- Teacher Dependency: The dataset construction pipeline relies heavily on high-performance closed-source teacher models (like GPT-5.2) to simulate the initial compilation data for SFT.
- Long-Horizon State Management: While the compiler includes a
FOLDoperation to compress the Brief State when it exceeds 10 items, extremely long-horizon tasks might still challenge the compact representation of the state.
Summary updated