Coding Agent Memory Post-Training II: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via On-Policy Distillation

Lirui Luo, Kelong Mao, Xinwei Yang, Rongqing Li, Luyu Chen, Kieran Wong, Yudong Guo, Xinrui Wang, Jiayin Zhu, Gu Simiu, Sulong Xu, Cong Fang

TL;DR

LLM agents often "forget" crucial information when long tasks exceed their context window. This paper introduces CAMP-II, a training method that teaches agents to use standard Bash file operations as a persistent memory, leveraging the models' existing pre-training knowledge rather than forcing them to learn complex, custom memory tools. This approach achieves nearly 3x the performance gain of previous state-of-the-art methods and generalizes across diverse tasks from web shopping to software engineering.

What Problem Does This Paper Address?

As AI agents tackle increasingly complex, "long-horizon" tasks—like building a website or conducting scientific research—their interaction history eventually overflows the model's active context window. To survive this, agents need a "memory" where they can write down important state information and read it back later.

Previous attempts to solve this used specialized memory tools (like store/retrieve commands) or complex "context compaction" protocols. However, these methods suffer from a two-sided interface problem:

  1. The Student's Struggle: Because these tools weren't in the model's original training data, the agent (the student) rarely uses them correctly during exploration.
  2. The Teacher's Confusion: During distillation (where a larger model teaches a smaller one), the teacher model often hasn't seen these custom tools either, leading to poor-quality supervision.

This mismatch creates a "negative-advantage" problem: the teacher's feedback often just tells the student what not to do, without providing a clear example of what a "good" memory action looks like.

Comparison of memory interfaces. Standard Bash operations leverage the teacher's existing knowledge (prior), providing much clearer supervision than dedicated tools.

Methods

The authors propose CAMP-II (Coding Agent Memory Post-Training II), which shifts the memory interface to something every modern LLM already knows: ordinary file operations via Bash.

Key Innovations:

  • Shared File-Memory Interface: Agents are given a persistent workspace and standard Bash access. They can create, edit, and search files to store their reasoning and intermediate results.
  • Context Replacement Protocol: To force the agent to use its memory, the environment periodically clears the context window. Before this "wipe," the agent must write its state to a CONTINUATION.md file. After the wipe, the agent only sees the path to that file and must proactively read it to resume the task.
  • Fully Asynchronous On-Policy Distillation (OPD): The student model explores the environment, and a larger, frozen teacher (Qwen3.8-27B) scores the student's actions. Because both models have a strong "prior" for file operations from their pre-training, the teacher can provide highly accurate, token-level feedback on the content of the agent's notes.
The CAMP-II training loop: students rollout in diverse environments, saving state to files. A frozen teacher then scores these rollouts to update the student model.

Datasets

The researchers introduced CAMG-II (Coding Agent Memory Gym II), a suite of five long-horizon environments:

  • Shop: Multi-session shopping requiring retention of user preferences across browser resets.
  • Coding: Software repair tasks spanning multiple context windows.
  • AutoResearch: Iterative ML model development and validation.
  • General: Reconciling evidence across business documents and systems.
  • Visual: Building websites based on visual briefs and iterative feedback.

For evaluation of generalization, they used Terminal-Bench 2.1 and SWE-Bench Pro V2.

Key Insights and Ideas

  • The "Familiarity" Advantage: Models find Bash commands significantly less "surprising" than custom memory tools. As shown in Figure 3, both the teacher and student have much lower negative log-likelihood (NLL) for file operations, meaning they are operating within their "comfort zone."
  • Distillation Needs a Shared Language: On-policy distillation works best when the interface is part of the shared prior of both models. This allows the student to sample meaningful actions more often and the teacher to judge them more effectively.
  • Memory Content is Causal: Through "memory intervention" experiments (swapping an agent's notes with those from a different task), the authors proved that task success is directly tied to the quality of the information the agent chooses to save.
Analysis showing that models are most familiar with the Bash interface (a) and use it for memory actions far more frequently than dedicated tools (b).

Key Results

  • Superior Performance: CAMP-II achieved 2.97 times the gain of the strongest previous OPD baseline on the CAMG-II suite.
  • State-of-the-Art for Small Models: The CAMP-II-4B model reached a 64.6% average success rate, up from 45.3% for the base model.
  • Generalization Power: The 9B version of CAMP-II outperformed a much larger 35B model on the SWE-Bench Pro V2 coding benchmark (30.1% vs 27.4% success).
  • Stable Training: By using a familiar interface, the authors avoided the training instabilities common in previous agentic-RL work.
A case study from the Visual environment showing an agent writing a plan to a file, having its context replaced, and then successfully reading that file to finish the task.

Limitations and Future Work

  • Scale: The study focused on models up to 9B parameters. It is currently unknown if these specific gains persist for 70B+ models or if they would benefit from multi-teacher distillation.
  • Single Teacher: The method relies on a single frozen teacher; exploring how different teacher architectures affect the quality of learned memory is a promising future direction.
  • File Management: While Bash is powerful, future work could explore more structured file-based memory systems (e.g., local databases or version control) that still leverage pre-trained priors.

Summary updated

Abstract

Language-model agents increasingly tackle long-horizon tasks whose interaction histories exceed the model's active context. They must therefore decide what to write down and what to read back, and when to do either. Recent work has begun to use on-policy distillation (OPD) to make memory control part of the policy, relying on memory tools and compaction protocols that lie outside either model's pre-training. However, this weakens the distillation on both sides: the teacher cannot give useful guidance on an interface it has never used, and the student samples too few meaningful memory actions for that guidance to take effect. To study this problem, we introduce Coding Agent Memory Gym II (CAMG-II), a suite of long-horizon agentic-RL environments spanning Shop, Coding, AutoResearch, General, and Visual. Alongside each environment's native task interface, CAMG-II provides ordinary Bash access and an episode-persistent workspace, enabling the policy to create, revise, search, and reuse files as memory across context replacements. We also introduce Coding Agent Memory Post-Training II (CAMP-II), which trains a single policy jointly across all five environments with fully asynchronous OPD to learn this file-based memory behavior directly from a frozen teacher with a prior for file operations. On CAMG-II, CAMP-II obtains 2.97 times the gain of the strongest OPD baseline. The memory behavior CAMP-II learns generalizes to two external benchmarks, Terminal-Bench 2.1 and SWE-Bench Pro V2, with improved success rates.
BibTeX
@misc{luo2026coding,
  author = {Lirui Luo and Kelong Mao and Xinwei Yang and Rongqing Li and Luyu Chen and Kieran Wong and Yudong Guo and Xinrui Wang and Jiayin Zhu and Gu Simiu and Sulong Xu and Cong Fang},
  title = {{Coding Agent Memory Post-Training II: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via On-Policy Distillation}},
  year = {2026},
  eprint = {2611.14009},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.14009},
}
Figure