TL;DR
LLM agents often "forget" crucial information when long tasks exceed their context window. This paper introduces CAMP-II, a training method that teaches agents to use standard Bash file operations as a persistent memory, leveraging the models' existing pre-training knowledge rather than forcing them to learn complex, custom memory tools. This approach achieves nearly 3x the performance gain of previous state-of-the-art methods and generalizes across diverse tasks from web shopping to software engineering.
What Problem Does This Paper Address?
As AI agents tackle increasingly complex, "long-horizon" tasks—like building a website or conducting scientific research—their interaction history eventually overflows the model's active context window. To survive this, agents need a "memory" where they can write down important state information and read it back later.
Previous attempts to solve this used specialized memory tools (like store/retrieve commands) or complex "context compaction" protocols. However, these methods suffer from a two-sided interface problem:
- The Student's Struggle: Because these tools weren't in the model's original training data, the agent (the student) rarely uses them correctly during exploration.
- The Teacher's Confusion: During distillation (where a larger model teaches a smaller one), the teacher model often hasn't seen these custom tools either, leading to poor-quality supervision.
This mismatch creates a "negative-advantage" problem: the teacher's feedback often just tells the student what not to do, without providing a clear example of what a "good" memory action looks like.
Methods
The authors propose CAMP-II (Coding Agent Memory Post-Training II), which shifts the memory interface to something every modern LLM already knows: ordinary file operations via Bash.
Key Innovations:
- Shared File-Memory Interface: Agents are given a persistent workspace and standard Bash access. They can create, edit, and search files to store their reasoning and intermediate results.
- Context Replacement Protocol: To force the agent to use its memory, the environment periodically clears the context window. Before this "wipe," the agent must write its state to a
CONTINUATION.mdfile. After the wipe, the agent only sees the path to that file and must proactively read it to resume the task. - Fully Asynchronous On-Policy Distillation (OPD): The student model explores the environment, and a larger, frozen teacher (Qwen3.8-27B) scores the student's actions. Because both models have a strong "prior" for file operations from their pre-training, the teacher can provide highly accurate, token-level feedback on the content of the agent's notes.
Datasets
The researchers introduced CAMG-II (Coding Agent Memory Gym II), a suite of five long-horizon environments:
- Shop: Multi-session shopping requiring retention of user preferences across browser resets.
- Coding: Software repair tasks spanning multiple context windows.
- AutoResearch: Iterative ML model development and validation.
- General: Reconciling evidence across business documents and systems.
- Visual: Building websites based on visual briefs and iterative feedback.
For evaluation of generalization, they used Terminal-Bench 2.1 and SWE-Bench Pro V2.
Key Insights and Ideas
- The "Familiarity" Advantage: Models find Bash commands significantly less "surprising" than custom memory tools. As shown in Figure 3, both the teacher and student have much lower negative log-likelihood (NLL) for file operations, meaning they are operating within their "comfort zone."
- Distillation Needs a Shared Language: On-policy distillation works best when the interface is part of the shared prior of both models. This allows the student to sample meaningful actions more often and the teacher to judge them more effectively.
- Memory Content is Causal: Through "memory intervention" experiments (swapping an agent's notes with those from a different task), the authors proved that task success is directly tied to the quality of the information the agent chooses to save.
Key Results
- Superior Performance: CAMP-II achieved 2.97 times the gain of the strongest previous OPD baseline on the CAMG-II suite.
- State-of-the-Art for Small Models: The CAMP-II-4B model reached a 64.6% average success rate, up from 45.3% for the base model.
- Generalization Power: The 9B version of CAMP-II outperformed a much larger 35B model on the SWE-Bench Pro V2 coding benchmark (30.1% vs 27.4% success).
- Stable Training: By using a familiar interface, the authors avoided the training instabilities common in previous agentic-RL work.
Limitations and Future Work
- Scale: The study focused on models up to 9B parameters. It is currently unknown if these specific gains persist for 70B+ models or if they would benefit from multi-teacher distillation.
- Single Teacher: The method relies on a single frozen teacher; exploring how different teacher architectures affect the quality of learned memory is a promising future direction.
- File Management: While Bash is powerful, future work could explore more structured file-based memory systems (e.g., local databases or version control) that still leverage pre-trained priors.
Summary updated