When a language-model agent reaches a hard step in a long task, it can spend extra compute in two ways. It can think longer in place, writing more reasoning before it acts. Or it can split the step off into a fresh, clean context, a sub-task that sees only what it needs and hands back a short summary. Current work treats these as separate questions. Methods that adapt reasoning length are trained on clean, single-turn math problems. Methods that split tasks into clean sub-contexts reason uniformly once inside a sub-task. Both ignore a fact that a third literature has established: models are badly distracted by irrelevant context. Thinking longer inside a cluttered context also adds the model's own words to that clutter.
This proposal claims that the right amount of reasoning for a step depends on how noisy that step's context is, and not only on how hard the step is. When the context is noisy, branching into a clean sub-context with short reasoning should beat thinking longer in place at equal cost. We propose first to measure this interaction directly, then to train one agent policy that picks, at each step, among three actions: answer now, think with a chosen budget, or branch into a clean sub-context. The policy is rewarded for success minus the total tokens it spends. The measurement comes first because it decides whether the joint policy is needed at all.
Why the pieces do not already add up
Three findings motivate the claim. Under adversarially chosen but coherent distractor sentences, frontier models lose over 45% accuracy on average. GPT-4o-mini falls from 85.7% to 22.0% on CommonsenseQA, and prompts telling the model to ignore irrelevant text give almost no protection (Adaptive Distraction, Breaking Focus). Mechanistic work traces part of this to a small set of attention heads that reflexively boost any token present in the context, relevant or not (Llama See, Llama Do). That makes it plausible that the model's own long reasoning, which is full of candidate tokens and half-formed claims, entrains later steps in the same way. Separately, longer reasoning sometimes hurts. On TurtleBench, o1-preview (84.40%) trails Claude 3.5 Sonnet (89.49%), and wrong answers use as many tokens as right ones (TurtleBench). Forcing a slow analysis mode on easy GUI steps also degrades accuracy (Think Twice, Click Once).
Two lines of work supply the method we reuse. Context Folding trains an agent with reinforcement learning to call branch (open an isolated sub-context) and return (discard the sub-context and pass back a summary). It reaches 62.0% Pass@1 on BrowseComp-Plus and 58.0% on SWE-Bench Verified with a 32K active budget, but reasoning depth inside branches is not adapted. Hierarchical Budget Policy Optimization (HBPO) trains a model to choose among token-budget tiers. It cuts tokens by 60.6% while raising accuracy, reaching 31.1% on AIME25, but only on clean, single-turn math. COMPASS separates planning from execution and uses a context curator, but its roles are fixed and its per-step compute is static. Micro-Act is the closest in spirit: it decomposes only when a knowledge conflict is detected and stops adaptively. It is limited to question answering, though, and still makes 37% of its errors by trusting bad evidence.
None of these asks the question this proposal targets. When a step is both hard and noisy, which is the better use of the next thousand tokens: more thought here, or a clean room elsewhere?
Core Claim
The marginal value of extra reasoning tokens falls as context noise rises, and can turn negative. Branching recovers that value by removing the noise before reasoning begins. The first part is testable by measurement. The second is testable by comparing actions at matched token cost.
If both hold, then a policy that sees only difficulty, which is what HBPO learns, will over-think in noisy contexts. A policy that sees only context length, which is roughly what Context Folding's penalty on unfolded tokens encourages, will branch for long-but-clean steps where in-place thought would be cheaper. A joint policy should beat simply stacking the two. If the measurement shows no interaction, meaning the benefit of depth is the same at every noise level, then stacking is sufficient and we will report that rather than build the joint policy.
Research Program
The first stage builds a controlled testbed with known difficulty and known noise at every node. We chain problems from GSM8K and MATH-500 into multi-part tasks, where each part's output feeds the next. Into each part's context we insert distractors at graded strength: none, static template distractors (which the Adaptive Distraction authors report cause under 5% degradation), and tree-searched, error-guided distractors produced with the Adaptive Distraction procedure against the model under study. The result is a factorial grid of difficulty against noise strength.
For each cell we sweep three variables: the reasoning budget, using HBPO's tiers of 512, 1024, 2048 and 2560 tokens; whether the part is solved in place or in a branch; and whether a child's full reasoning or only its summary is returned to the parent. That last condition directly measures how much a child's thinking contaminates the parent. The output of this stage is a set of accuracy-versus-tokens curves per cell. The interaction we predict appears as curves that slope upward for clean contexts and flatten or bend down for noisy ones, while the matching branch curves stay upward.
The second stage trains the joint policy. At each step the agent emits one of three actions: answer, think with a budget tier, or branch with a sub-task description. Branches may themselves choose a depth. Following HBPO, we exceed Context Folding's single level of branching only if the testbed shows nested branches are stable. Training uses group-relative policy optimization, an RL method that compares several sampled attempts at the same task against each other. The reward is
$$R = s - \lambda\,(T_{\text{think}} + T_{\text{main}}),$$
where $s$ is 1 for task success and 0 otherwise, $T_{\text{think}}$ is the total number of reasoning tokens generated across all nodes, $T_{\text{main}}$ is the number of tokens that remain in the parent's active context, and $\lambda$ trades accuracy against cost. We keep HBPO's piecewise penalty for exceeding a tier, and we keep Context Folding's failure penalty for failed tool calls. Charging for both kinds of tokens is what forces the policy to compare branching against in-place thought. The policy is never told the noise level and must infer it from the context.
The third stage moves to real agent tasks: BrowseComp-Plus, SWE-Bench Verified, and GAIA stratified by its three difficulty levels. There, noise arises naturally from tool outputs and web pages. We check whether the policy's learned choice of action tracks noise in the observations that triggered it, measured by the share of observation tokens judged irrelevant by an offline annotator.
Baselines
The naive reference point is a flat ReAct-style agent, one that alternates reasoning and tool calls in a single growing context, using uniform chain-of-thought and no branching. We run a second flat version with the full long context available, to represent the "just use a bigger window." Context Folding reports that its 32K folded agent beats flat 327K baselines, so we expect both flat agents to lose. They anchor the scale of the gains.
The strongest existing method is Context Folding trained with its FoldGRPO objective. On BrowseComp-Plus it reports 62.0% Pass@1 and on SWE-Bench Verified 58.0%, both with Seed-OSS-36B-Instruct under a 32K active budget. Training at that scale may exceed our budget. If so, we retrain Context Folding ourselves on the same smaller base model, data, number of RL steps and rollouts per group as our method, and report both our reproduction and the published figures. On the controlled testbed there is no reported result found on Lacuna for any method, because the testbed is new.
The decisive ablation is the stacked system: Context Folding's branch policy, with each branch running an HBPO-trained depth policy, trained with the same total compute and the same sweep over $\lambda$. It contains every ingredient except joint selection, so it isolates the claim. Beating it means higher success at equal or lower total tokens on the noisy cells of the testbed and on BrowseComp-Plus, with no loss on the clean cells. A second ablation removes $T_{\text{main}}$ from the reward. A third returns children's full reasoning instead of a summary, to test whether self-generated text is a meaningful source of noise.
We also run a hand-built rule baseline: branch when an observation exceeds a length threshold, and think longer when answer confidence falls below a threshold, in the style of the confidence switch in Think Twice, Click Once. Both thresholds are tuned on a validation split with the same number of tuning trials we give $\lambda$.
All systems share the base model, the 32K active-context limit, the evaluation seeds, and a fixed tuning budget.
Why not just…?
Why not just stack Context Folding and HBPO? This is the real threat, and the proposal is designed so it can win. If the first stage shows that the value of depth does not depend on noise, then a difficulty-only depth policy inside branches is already optimal, and the joint policy adds nothing. The stacked ablation, trained at equal compute, settles the question. We consider the idea refuted if stacking matches the joint policy within seed variance on the noisy cells.
Why not just filter the context first, then reason adaptively? Extract-then-generate pipelines such as Ext2Gen, and summarization of the instruction (Less is More), remove noise before any reasoning happens. But Less is More reports only about a 2% gain on competition-level code, suggesting that denoising does not substitute for reasoning on hard steps. A one-time filter also cannot remove noise the agent generates later, from tool outputs or its own reasoning. We include a filter-then-HBPO baseline on the testbed. If it matches the joint policy on the single-step noisy cells but loses on multi-part chains, that locates the benefit in noise that arises during execution.
Why not just use simple thresholds on length and confidence? Rules are cheap and transparent, and if they match the learned policy we will recommend them. Two pieces of evidence suggest they may fall short. Context Folding reports that hand-designed multi-agent structures generalize poorly. Adaptive Distraction shows that the most damaging distractors are short and coherent, so length alone is a weak proxy for noise. The tuned rule baseline tests this directly, and the per-cell analysis shows where the rule mis-assigns actions.
Why not just make the model robust to distractors with preference training? Adaptive Distraction reports that DPO, which trains the model to prefer its answers on clean inputs over its answers on distracted ones, improves accuracy on perturbed inputs by nearly 50% for some smaller models. This approach needs paired distracted and clean data, and it does not address noise the model writes itself. Robustness and routing can also be combined. We therefore run the joint policy on top of a DPO-robustified base model as well. If the advantage of branching disappears once the base model is robust, that is evidence that branching mainly substitutes for missing robustness, which is a useful and reportable result.
Proposed Experiments and Decisive Checks
On the testbed, the primary readout is accuracy as a function of total tokens in each difficulty-by-noise cell, for every system. The first decisive check is whether the slope of accuracy against reasoning tokens decreases with noise strength for in-place reasoning but not for branched reasoning. The second is whether the joint policy's chosen action tracks noise when difficulty is held fixed: it should choose branching more often in tree-searched-distractor cells than in clean cells of the same difficulty, whereas the stacked system will not.
On the agent benchmarks we report Pass@1 and active-context tokens on BrowseComp-Plus and SWE-Bench Verified, and accuracy by level on GAIA. Following the GAIA authors' report of 0% for all models on Level 3, we treat Level 3 mainly as a probe for where nested branching matters. TurtleBench serves as an overthinking check. A depth policy that is aware of noise should not fall into the pattern of wrong answers consuming as many tokens as right ones that TurtleBench reports for o1 models, and we compare token counts on correct versus incorrect items.
The proposal succeeds if the joint policy beats the stacked ablation at matched tokens on noisy conditions while tying on clean ones. It fails, and the simpler stacked design should be preferred, if the first-stage interaction is absent.
References
- Adaptive Distraction: Probing LLM Contextual Robustness with Automated Tree Search
- Breaking Focus: Contextual Distraction Curse in Large Language Models
- Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs
- TurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles
- Think Twice, Click Once: Enhancing GUI Grounding via Fast and Slow Systems
- Scaling Long-Horizon Agent via Context Folding
- Hierarchical Budget Policy Optimization for Adaptive Reasoning
- COMPASS: Enhancing Agent Long-Horizon Reasoning with Evolving Context
- Micro-Act: Mitigate Knowledge Conflict in Question Answering via Actionable Self-Reasoning
- Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
- Less is More: Summary of Long Instructions is Better for Program Synthesis