Folding the Agent's Own Thoughts: Trained Self-Compression for Long-Horizon Search Agents

Search agents answer hard questions by alternating between thinking and calling tools such as web search. Each turn adds a block of the agent's own reasoning plus a bulky tool response to the prompt, so the context window fills up after a few dozen turns. The strongest current fix throws away old tool outputs and keeps every word of the agent's past reasoning. That postpones the problem without removing it, because the reasoning keeps piling up. We propose teaching the agent to fold its own past reasoning. At the end of each turn it writes a short summary that records both what it concluded and which facts from the tool output it will need later. It then deletes the verbose reasoning and the raw tool output, and continues from the summaries alone. The agent is trained with reinforcement learning while it actually runs in this folded mode, so it learns to write summaries it can genuinely rely on.

The central question is whether a result from single-problem math reasoning carries over to agents. In math, a model trained under folding eventually matches its unfolded accuracy. In an agent, the folded material includes outside evidence the model cannot re-derive if it forgets it. If folding still works there, agents could run for hundreds of turns, or under much tighter memory budgets, with a context that stays roughly constant in size. If it fails, we learn exactly where self-compression breaks down.

Core Claim

Two prior results meet here without having been combined. Accordion-Thinking splits a math solution into detailed segments, each followed by a step summary. At inference it evicts each detailed segment from the KV cache (the stored attention keys and values for past tokens) and continues from the summaries alone. It reports a "gap-vanishing" effect: after an SFT warm start, a model running in fold mode is clearly worse than full-context mode, and RL with GRPO closes that gap. Its Accordion-Thinker-7B reaches 32.2% Pass@1 on AIME24 in fold mode against 26.7% for the unfolded SFT model. Beyond Turn Limits (DeepMiner) manages agent context with a different rule, "reasoning > raw data." It evicts old tool responses and keeps all reasoning traces intact, which lets it sustain nearly 100 turns in 32k tokens and reach 33.5% on BrowseComp-en. It explicitly treats the agent's reasoning as untouchable and rejects external summarizers as creating "optimization blind spots."

Our claim is that the agent's own reasoning can also be folded, with three conditions. The summary must be written by the policy itself, not by a helper model. It must carry forward the specific tool-derived facts later turns depend on. And the policy must be trained by RL in fold-only execution. Under those conditions we expect a gap-vanishing effect on multi-hop search, so that context length stops growing with the number of turns. The new elements, relative to Accordion, are external evidence inside the folded content and a horizon of many turns rather than 2–6 steps. Relative to DeepMiner, the new element is compressing reasoning, not just tool outputs. We also ask whether fold timing should be learned or simply tied to turn boundaries. We treat this as a secondary, empirical question, because turn boundaries are a natural and strong default.

Research Program

The agent produces, at each turn $t$, a reasoning block, a tool call, and a fold summary. After the tool returns and the summary is written, the reasoning block and raw tool output for that turn are evicted. The next turn sees only the question and the summaries written so far. The summary format has two fields: a short deduction ("the director is X, so we now need X's birth year") and an evidence ledger of quoted facts with their sources. The ledger is the main design change from Accordion. In math, a lost intermediate value can in principle be recomputed. In search, a lost fact must be searched again or is gone for good.

Training follows a two-stage recipe. The cold start differs from Accordion's: we rewrite the policy's own successful ReAct-style trajectories into the folded format using the same base model, prompted with the format specification. This tests whether the teacher dependence Accordion leaves open can be removed. We then run GRPO in strict fold mode, with a correctness reward and no length penalty. Building on FrugalRAG, a second variant adds its stopping reward, which gives bonuses for stopping at the right number of searches and penalties for stopping too early or too late. This lets us see whether folding and learned stopping interact.

To study timing, a third variant lets the agent emit a special fold token at any point, including mid-turn or every few turns, instead of folding at each turn boundary. It is trained with the same reward. All variants use Qwen-family 7B models, with a 32B replication of the main comparison matching DeepMiner's scale if compute allows.

The analysis has three parts. First, we track the fold-versus-unfold accuracy gap across RL training. Second, we run an evidence-retention probe. At the end of a trajectory, we ask the folded agent factual questions about tool outputs it saw and evicted many turns earlier. This separates summaries that preserve the facts the agent needs from summaries that merely sound fluent. Third, we inspect attention, following Accordion's observation that folded traces yield denser attention patterns.

Baselines

The naive reference point is a standard ReAct agent that keeps its full context and truncates the oldest content when it hits the budget. It is fine-tuned on the same cold-start data as our method, so it differs only in context policy. FrugalRAG reports ReAct at 64.20% MBE with 2.76 searches on HotpotQA. That number anchors whether our reimplementation is sane.

The strongest existing method is DeepMiner's dynamic window, which evicts old tool outputs and keeps all reasoning. It is the most direct competitor because it already solves most of the context-bloat problem on the same kind of task. Its reported result is 33.5% accuracy on BrowseComp-en with Qwen3-32B. We reimplement its window rule on our base model, and train it with the same RL algorithm, data, number of rollouts, and tuning budget as our method. The only difference is that our method also folds reasoning.

Beating it means two things. At the standard 32k budget, our method should match its accuracy with substantially lower peak context. Under tight budgets of 8k and 4k tokens, or on tasks deliberately constructed to need more than 100 turns, where its retained reasoning itself overflows, our method should achieve clearly higher accuracy.

For short multi-hop QA, the strongest efficiency baseline is FrugalRAG. It reports 68.47% MBE on HotpotQA with 2.05 searches, and over 20% on BrowseCompPlus zero-shot after training on about 1,000 HotpotQA examples. There, beating it means matching accuracy and search count at lower peak tokens. We do not expect large accuracy gains on HotpotQA, since short trajectories rarely overflow.

A training-free baseline applies ThinKV-style thought-adaptive eviction to the agent's KV cache. That method compresses the cache of a model that is unaware of the pruning, classifying tokens by thought type with offline-calibrated thresholds. It reports near-lossless accuracy at 1.3%–3.7% of full KV memory on AIME and LiveCodeBench. It is matched to our method on peak memory.

The key ablation is the same folded agent evaluated after SFT only, with no RL. It isolates whether training under folding is what makes the method work. A second ablation removes the evidence ledger, keeping deduction-only summaries as in Accordion. A third swaps fixed per-turn folding for learned fold timing.

Why not just…?

Why not just keep DeepMiner's rule and use a longer context window? At 32k, DeepMiner already reaches about 100 turns, so on today's benchmarks this may be enough. That is a real threat to the practical value of the work. Our answer is to measure the regime where it is not enough: tight memory budgets, as on edge or high-concurrency serving, and synthetic tasks requiring longer horizons. If folding yields no advantage even there, we will report that the reasoning stack is not the binding constraint. A longer window also leaves the attention cost growing with every turn, while folded context stays roughly flat.

Why not just summarize periodically with a small helper model? DeepMiner reports that external summarizers create optimization blind spots and drop crucial details. Accordion shows that summaries must be trained under the conditions in which they will be used. We include a helper-summarizer baseline matched on the size of the folded context. The evidence-retention probe will show directly whether its summaries lose the facts that later turns need.

Why not just fold at every turn boundary and skip learning when to fold? This is plausible, and it is why fixed per-turn folding is our default method rather than a strawman. Learned timing is only claimed if it wins. We expect it might help where one turn contains several distinct deductions, or where several turns refine a single deduction. If it doesn't, the paper's contribution rests on the fold-mode RL and the evidence ledger alone.

Why not just prune the KV cache without retraining, as ThinKV does? ThinKV works within a single reasoning trace, where unimportant transition tokens can be identified. Accordion reports that heuristic deletion without model adaptation disrupts logical flow. Agent contexts also contain heterogeneous tool text that ThinKV's calibration never saw, and its offline thresholds may not transfer. The matched-memory comparison will settle this.

Proposed Experiments

The main evaluation uses BrowseComp-en for long-horizon search, comparing directly to DeepMiner's 33.5%, and BrowseCompPlus zero-shot, comparing to FrugalRAG's over-20%. HotpotQA in the full-wiki setting, MuSiQue, and 2WikiMultiHopQA cover shorter multi-hop chains. No reported result was found on Lacuna for MuSiQue or 2WikiMultiHopQA, so we will establish baselines ourselves. Metrics are answer accuracy (MBE and exact match where applicable), peak and mean context tokens, number of searches, number of turns sustained before failure, and evidence-retention probe accuracy.

The first decisive check is the gap curve. The fold-versus-unfold gap should be large after SFT and shrink under GRPO on BrowseComp-style tasks. If it plateaus well above zero, gap-vanishing does not transfer to external-evidence settings. The ledger-free ablation then tells us whether carrying facts forward is what limits it.

The second check is the budget sweep. We plot accuracy against context budget from 4k to 32k for full-context ReAct, DeepMiner's rule, ThinKV-style eviction, the helper summarizer, and our folded agent. The claim holds only if our curve is flatter, meaning it keeps accuracy as the budget shrinks while the others degrade.

The third check is the retention probe. Folded agents that score well on the final answer but poorly on recalling evicted evidence would indicate the summaries work by shortcut, for example by searching again for lost facts. We would detect that through increased search counts and report it as such.