MEMENTO: Learning Context Management for Long Context Processing Agents

Yunfan Zhang, Taehoon Hwang, Kathleen McKeown, Smaranda MuresanarXiv 2026 · arXiv

TL;DR

MEMENTO is a context-management framework that enables small LLM agents (like Qwen3-8B) to process massive documents by learning when and how to "compact" their history into notes. By combining supervised bootstrapping with reinforcement learning, MEMENTO achieves performance comparable to models 100x its size at a fraction of the cost, effectively managing books up to 2.5 million tokens while staying within a 20K token context window.

What Problem Does This Paper Address?

Large Language Model (LLM) agents are increasingly used for "long-horizon" tasks, such as answering complex questions about entire books. However, they face a fundamental "context window" bottleneck. Even as windows grow to 1M+ tokens, models often suffer from "lost in the middle" phenomena or performance degradation as the context fills up.

The standard solution is compaction: the agent summarizes its previous steps into a "note" and clears its history to make room for new information. The catch? Off-the-shelf models are surprisingly bad at this. They either write useless notes, forget to take notes entirely, or fail to follow the formatting required to make the notes useful for future steps. Reinforcement Learning (RL) usually fails to fix this because the model is so unlikely to generate a "perfect" note-taking trajectory by chance that there is nothing for the RL algorithm to reward.

Methods

The authors developed a two-stage training pipeline to "teach" an 8B-parameter model (Qwen3) how to manage its own memory:

  1. Synthetic SFT (Supervised Fine-Tuning): To solve the "cold start" problem, they used a larger teacher model (GLM-5.3) to generate successful task rollouts. They then retrospectively injected high-quality notes into these trajectories and fine-tuned the 8B model on this data. This "bootstraps" the basic behavior of note-taking.
  2. RL with GRPO: They refined the model using Group Relative Policy Optimization (GRPO). The reward function wasn't just based on the final answer; it included "process signals" like note validity (frequency, placement, and diversity) and confidence calibration.

At inference time, MEMENTO uses a "harness" that supports three distinct strategies (shown in Figure 1):

  • Compaction: Starting a fresh window with only the saved notes.
  • Context Folding: Keeping notes plus the most recent turns to maintain "local" momentum.
  • Confidence-based Backtracking: If the agent gives an answer with low verbalized confidence, it "rewinds" to an earlier note and tries a different search path.
Figure 1: Context management methods incorporated in this work. (a) Compaction starts a new window contain- ing only the saved notes and discards all prior turns. (b) Context folding additionally retains the turns after the latest note. (c) Confidence-based backtracking resumes from an earlier note checkpoint after a low-confidence answer and generates an alternative continuation.

Datasets

The researchers focused on Book-level Multi-hop Question Answering (Book QA), where answers are buried across hundreds of pages. They used:

  • LiteraryQA: Narrative-heavy questions.
  • NovelHopQA: Requires connecting multiple pieces of evidence.
  • SagaScale (English subset): Extremely long-context benchmarks.
  • NovelQA: Used as a held-out evaluation set to ensure the model wasn't just memorizing training books.

Key Insights and Ideas

  • Co-design is King: You cannot train the model in a vacuum. The training must be "co-designed" with the test-time harness. Because MEMENTO was trained to expect its context to be reset and replaced by notes, it naturally generalized to more advanced techniques like "folding" and "backtracking" without further training.
  • SFT as a Catalyst: Figure 4 shows that without the initial SFT bootstrapping, RL quickly "gives up" on note-taking because it's too hard. SFT provides the "right strategy," and RL then makes that strategy "reliable."
  • Confidence as a Search Signal: By rewarding the model for being "calibrated" (knowing when it's likely wrong), the authors enabled a search-like behavior at test time. If the model says "I'm not sure," the system automatically triggers a backtrack.
Figure 4: Mean context-window restarts per RL training rollout. Legend names follow Table 1; training uses compaction into fresh windows. Faint curves show per- step values; bold curves show five-step trailing means. SFT + RL checkpoints retain compaction ratio robustly during the RL training process, while compaction ratio in RL only checkpoints collapses.

Key Results

MEMENTO (8B) punches significantly above its weight class:

  • Accuracy: It raised Qwen3-8B's accuracy from 36.9% to 67.8%.
  • Efficiency: It nearly matched the performance of GLM-5.3 (a ~750B model) while using 1/9th the cost and half the tokens per rollout.
  • Vs. Proprietary Giants: It outperformed a GPT-5.5 agent (13.1% in the same harness) and even beat GPT-5.5 using its full native context window (56.7%), all while being 54x cheaper.
  • Consistency: MEMENTO never held more than 20K tokens in context at once, proving that smart management beats raw window size.
Figure 3: Validation answer accuracy over RL train- ing for the models in Table 1. The gray dot marks a single outlier evaluation of the RL (Single Window) run, caused by reward hacking that led us to tighten the answer length limit. While SFT + RL and RL only variants start at similar validation accuracy, SFT + RL outperforms RL only by 15% at the end of training due to more effective compaction behavior instilled by the SFT.

Limitations and Future Work

  • Overconfidence: Late in RL training, the model tends to become overconfident, even when wrong. This reduces the effectiveness of backtracking, as the model doesn't "know" it needs to retry.
  • Heuristic-based Rewards: The "note validity" reward relies on hand-crafted heuristics (e.g., "don't take two notes in a row"). Future work could involve a more semantic "meta-judge" to reward notes based on their actual utility for solving the task.
  • Generalization: The study was limited to English Book QA. Whether these note-taking strategies transfer to coding or web research remains an open question.

Summary updated

Abstract

LLM agents that operate over long inputs must perform context management: deciding what to retain in a fixed-size context window. A common approach is compaction, where the model replaces its history with a self-generated note. Yet off-the-shelf agentic models, including GPT-5.5 and GLM-5.3, often write ineffective notes, and reinforcement learning (RL) alone rarely discovers useful note-taking behavior. We present MEMENTO, a long-context agent built on Qwen3-8B and trained to manage its context through compaction. We first bootstrap note-taking by fine-tuning on synthetic trajectories annotated with notes generated in hindsight, then reinforce it with RL using a reward that combines answer correctness, note-quality heuristics, and confidence calibration. Although trained only to compact into a fresh window, MEMENTO extends at test time to strategies such as context folding and confidence-based backtracking. On book-level question answering across four benchmarks, with books of up to 2.5M tokens, MEMENTO raises Qwen3-8B's accuracy from 36.9% to 67.8% while never holding more than 20K tokens in context. It outperforms GPT-5.5 agent in the same harness (13.1%) and full book context (56.7%) at less than 1/50 the cost. It also nearly matches a ~100x larger GLM-5.3 agent (69.3%) at 1/9 the cost, and comes within 6.4 points of GLM-5.3 full context at 1/26 the cost.
BibTeX
@misc{zhang2026memento,
  author = {Yunfan Zhang and Taehoon Hwang and Kathleen McKeown and Smaranda Muresan},
  title = {{MEMENTO: Learning Context Management for Long Context Processing Agents}},
  year = {2026},
  eprint = {2611.00244},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.00244},
}
Figure