Language models answer worse when their prompt is padded with irrelevant text. Feeding a model more retrieved passages first helps, because the answer is more likely to be present, and then hurts, because distractors pull the model off course. Where that turning point sits depends on which model is reading. Current fixes act at one fixed level of detail. They choose how many whole passages to include, or compress everything with one learned compressor, or pick a single chunk size per query. Continuous-control research solved a similar problem years ago with adaptive discretization. The idea there is to start with one coarse region covering the whole space and split a region into finer pieces only where data shows that the outcome changes sharply inside it.
We propose to treat the retrieved context as that space. The quantity whose sharp variation triggers a split is the reader model's own answer. A block of retrieved text is first shown to the reader whole. It is split into halves, then sentences, only when the reader gives inconsistent answers depending on which part of the block it sees. That pattern indicates the block mixes evidence with distraction. Blocks whose removal leaves the answer unchanged are dropped while still coarse, and never read at fine resolution. The final prompt mixes resolutions: sentence-level spans where the evidence is, coarse framing where it is needed, and nothing elsewhere. Because every split decision is driven by probes of the actual reader, the resulting context adapts to that reader's tolerance for noise rather than to retriever similarity scores.
Core Claim
The claim is that the resolution at which context is presented should be decided per region, per query, and per reader, using the reader's sensitivity to that region as the refinement signal. A short confidence-bounded zooming procedure, borrowed from metric-space reinforcement learning, can find such a mixed-resolution context with a small, bounded number of cheap reader probes. On distractor-heavy and multi-hop retrieval it should beat both passage-level volume selection and uniform fine-grained selection at equal reader compute.
A secondary, falsifiable prediction concerns how the context should change with the reader. Weaker readers such as Llama-3.1-8B should end up with smaller, finer contexts than stronger readers given the same retrieval. This would give a mechanistic, per-instance account of the inverted-U curve that the file reports only in aggregate.
Where This Sits
The method comes from adaptive discretization. Adaptive Discretization for Episodic Reinforcement Learning in Metric Spaces starts Q-learning from a single ball over the state-action space and splits a ball once its visit count justifies finer resolution. On Oil Discovery it reaches better reward with about 150 cells than a uniform grid does with about 850. State Space Construction by Attention Control supplies the specific trigger we reuse: split a state when the same action from within it produces inconsistent outcomes. Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adapting supplies a constraint. Adapting resolution online to unknown smoothness has a provable price, and running ensembles of fixed resolutions in parallel is expensive. We therefore use a single zooming process with confidence bounds rather than a meta-learner over many granularities. We do not import the metric-space assumption. Text has no reliable metric, so we use only the tree structure of the split process. Refinement is decided by reader behaviour, not by distance.
The motivation comes from the noise-sensitivity literature. Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering shows zero-shot reader accuracy falling as retrieval grows from 5 to 100 documents. Its Distraction-aware Answer Selection (DAS) reweights answers by whether a document could regenerate the query, and we reuse that score to arbitrate among surviving spans. Ada-LEval shows the extreme case: GPT-4 Turbo at roughly 44% on 16k BestAnswer, falling to 0% at 128k. UDA shows that isolating the target figures restores arithmetic accuracy. This is direct evidence that the useful resolution of context is often below the passage level.
Five systems sit closest to the proposal. LDAR learns, with REINFORCE and reader accuracy as reward, which contiguous band of ranked passages to include. It is reader-adaptive, but it treats passages as atomic and never zooms inside one. Mix-of-Granularity routes each query to one of five pre-built chunk sizes. That fixes one uniform resolution per query, decided before the reader is consulted. LongRefiner prunes a document tree built from layout tags under a fixed token budget, with selection driven by query-type scoring rather than reader feedback. RAPTOR builds a summary tree offline. Its best mode flattens all levels into one pool, which adds near-duplicate multi-level text to the prompt. CompAct reads passages in fixed batches of five and stops when a fine-tuned compressor declares the evidence complete. It cannot go back and look more finely at an earlier passage.
None of these use the reader's own inconsistency to decide where to refine. None produce a context whose resolution varies across regions within a single query. The idea that comes most readily to mind at this intersection is an LLM walking a RAPTOR-style summary tree top-down. It differs from ours in that it navigates by relevance and keeps whole nodes. Our procedure navigates by measured influence and disagreement, and builds no summaries.
Research Program
The retrieved pool for a query is the top 50 to 100 passages from a standard dense retriever. These are arranged in a cheap hierarchy with no LLM summarization: rank-ordered groups of passages at the top, then single passages, then paragraph halves, then sentences. Splits are binary along text order, like a one-dimensional k-d tree. The active context $A$ is the set of cells currently retained, each at its current resolution.
For a cell $C$ in $A$, influence is measured as
$$s(C) = \mathrm{TV}\big(p(\cdot \mid q, A),\; p(\cdot \mid q, A \setminus C)\big),$$
where $q$ is the question, $p(\cdot \mid q, A)$ is the reader's distribution over a small candidate-answer set given context $A$, and TV is total-variation distance. Heterogeneity $h(C)$ is the TV distance between the answer distributions the reader produces when $C$'s two children are each substituted for $C$.
The split rule follows the confidence-bounded logic of adaptive Q-learning. A cell is dropped when $s(C) + \beta(n_C) < \tau$. It is split when $h(C) - \beta(n_C) > \tau$. Here $n_C$ is the number of probes spent on $C$, $\beta(n)$ is a bootstrap confidence width over sampled reader outputs that shrinks as roughly $1/\sqrt{n}$ by standard concentration, and $\tau$ is a single threshold tuned on development data. Cells that are influential but homogeneous are kept at their current resolution. Cells that are still ambiguous receive further probes until a total budget $B$ of reader probes is spent.
Probes must be cheap enough to make this worthwhile. Each probe scores candidate answers by log-likelihood rather than generating text. Probes share a cached question prefix. Following Superposition Prompting, which processes documents along isolated parallel attention paths, sibling cells are scored in parallel paths so that a split costs one batched forward pass. The candidate-answer set is drawn from a few sampled generations on the coarse context.
Two weaknesses are built into the design and must be measured. First, a confidently misleading distractor is influential and homogeneous, so it survives. For this reason the final answer reweights surviving fine spans by the DAS query-regeneration score. Second, multi-hop evidence spans are individually incomplete. Their heterogeneity may therefore appear only jointly, so we also test a variant that probes pairs of sibling cells.
A third component is amortization. Zooming trajectories are distilled into a small policy that predicts split and drop decisions from cell text and question, removing probes at test time. This is secondary. The main claims are evaluated on the probe-based version.
Baselines
The naive reference is static top-$k$ retrieval over standard passages, with $k$ tuned on development data per reader. Alongside it we run full-context prompting of the whole retrieved pool, which is the "do nothing to the context" option. Both use the same retriever, pool, and reader as our method. To beat them, the method must show higher accuracy at equal or fewer reader tokens.
The strongest competitor on reader-adaptive volume is LDAR, because it alone learns the reader-specific noise tolerance our method also targets. The file reports that it beats top-$k$ and full context across six benchmarks at 32K and 128K while using 25% to 63% of full-context tokens, and that it runs nearly 4x faster than full context with open models. No reported absolute accuracy for LDAR on any specific dataset was found on Lacuna. We will retrain LDAR on our training split with the same reader and retriever, and give it a REINFORCE budget matched in reader forward passes to our tuning budget for $\tau$ plus the distillation stage. To beat it, our method must achieve higher exact match or F1 on HotpotQA, MuSiQue, and NQ-Open with 100 documents, at equal total reader compute counting our probes.
The strongest competitor on multi-hop compression is CompAct. The file reports a 7.0 F1 improvement over RECOMP on HotpotQA and does not report an absolute F1. We use its released compressor and the same reader, and report reader-plus-compressor compute. LongRefiner is run on the long-document settings. The file reports more than 9% accuracy over LongLLMLingua and a 10x token reduction, again without an absolute number. RAPTOR collapsed-tree retrieval is run on QuALITY. The file reports a 20% absolute gain over DPR and BM25 with GPT-4 and gives no absolute figure. DAS reweighting applied to plain top-$k$ is also run, so that our reuse of its score is not credited to zooming.
Two ablations isolate the key ingredient. The first is a flat one-level variant: passage-level leave-one-out influence with no splitting, at matched probe budget. If this variant matches the full method, the hierarchy adds nothing. The second is a uniform-fine variant: sentence-level cross-encoder selection of the top sentences under the same token budget. This is the text analogue of the uniform grid. We also swap the trigger, replacing reader heterogeneity with retriever-score variance, to test whether reader feedback specifically matters.
Why not just…?
Why not just rerank at the sentence level and keep the top sentences? This is the serious threat. On single-hop factoid questions it may well match us at a fraction of the cost. The evidence against it as a general answer is indirect. Mix-of-Granularity found that fine chunks fail on queries needing surrounding context. RAPTOR's NarrativeQA ablation drew over 57% of used nodes from non-leaf layers, which suggests that coarse framing carries real signal. The uniform-fine ablation settles the question directly. If it ties our method on MuSiQue and QuALITY as well as NQ, the mixed-resolution claim is wrong, and we would report that.
Why not just use LDAR with more passages? LDAR already adapts volume to the reader. It cannot remove noise inside a selected passage, and UDA's arithmetic result shows that within-passage noise matters. Still, if internal passage noise is minor on these benchmarks, LDAR at equal compute will win. The matched-compute comparison, broken down by the fraction of gold sentences per gold passage, shows where each method should lead.
Why not just leave out one passage at a time and keep the influential ones? This test asks whether recursive zooming is needed at all. The one-level ablation answers it directly. We predict the gap appears mainly on long passages and on 100-document pools, where passage-level influence is diluted.
Why not just use a stronger long-context model? Ada-LEval's 0% at 128k for GPT-4 Turbo on BestAnswer, and LDAR's token savings with no accuracy loss, argue that scale alone does not remove distraction. We nonetheless run one strong proprietary reader to check whether our gains shrink as reader strength grows, which is itself part of the second prediction.
Proposed Experiments and Decisive Checks
The main study uses HotpotQA, MuSiQue, and 2WikiMultiHopQA for scattered multi-hop evidence, and NQ-Open and SQuAD with 100 DPR passages for heavy distraction. QuALITY, including its HARD subset, and HELMET at 32K and 128K cover long single documents. Readers are Llama-3.1-8B and one stronger open model, plus a proprietary reader on a subset. Metrics are exact match or F1 (accuracy for QuALITY). Every result carries total reader tokens processed including probes, and wall-clock time. We plot accuracy against compute curves rather than single points, sweeping $B$ for our method and $k$ or budget for the baselines.
The first decisive check is the matched-compute comparison against LDAR, CompAct, and the uniform-fine ablation on MuSiQue and NQ-100. The method must lead on at least one of them, without losing on the other, for the core claim to stand.
The second check tests the reader-adaptation prediction. On identical retrieval pools, the final context of the weaker reader should be measurably smaller and contain a higher share of sentence-level cells. Re-plotting accuracy against retrieved-pool size should flatten the inverted-U for both readers.
The third check is a controlled distractor test that inserts high-overlap confuser passages next to gold ones. We report how often confusers survive zooming, with and without DAS reweighting, which quantifies the method's known blind spot.
A final diagnostic tests whether the method's choices are meaningful. We compare the split locations against gold supporting-sentence annotations in HotpotQA. Refinement should concentrate on gold passages the way adaptive Q-learning's partitions concentrate along the optimal-action diagonal. If splits are spread uniformly, the zooming is not finding structure, and gains would have to be attributed to something else.
References
- Adaptive Discretization for Episodic Reinforcement Learning in Metric Spaces
- State Space Construction by Attention Control
- Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adapting
- Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering
- Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks
- UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-World Document Analysis
- Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding
- Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation
- Hierarchical Document Refinement for Long-context Retrieval-augmented Generation
- RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
- CompAct: Compressing Retrieved Documents Actively for Question Answering
- Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation