When a language model answers a question over a long document, part of its output often goes to grounding: it restates or cites the passages it is relying on, and only then reasons to an answer. Methods that force this step improve long-context accuracy. They also cost tokens and latency, and nobody knows how much grounding is enough. Wireless engineers face the same problem in a different form. A transmitter spends part of each short, fixed-length window, called a coherence block, sending known "pilot" symbols so the receiver can estimate the channel. It spends the rest sending data. Too few pilots give a noisy estimate and garbled decoding. Too many leave too little room for the payload. Communication theory shows that this tradeoff has an interior optimum and that the optimum moves in predictable ways as channel conditions change.
We propose to treat grounding tokens as pilots and reasoning tokens as payload under a fixed generation budget. We will measure whether long-context accuracy has an interior optimum in the grounding fraction. We will then test whether that optimum shifts with two properties of the context that can be measured. The first is how scattered the evidence is. The second is how redundant it is, meaning how many independent surface copies of each needed fact are present. The redundancy variable comes from microbial ecology. There, different species carrying the same enzymes substitute for one another, so a community's function stays stable while its membership changes, until the last carrier of a function is removed. If the two properties predict the optimum, a model can set its grounding length per query. It could then match explicit-grounding methods at a fraction of their token cost, or beat them at equal cost.
Core Claim
Fix a total output budget $B$ tokens per query. The model splits it into $\tau$ grounding tokens and $B-\tau$ reasoning tokens. Let $A(\tau; d, r)$ be answer accuracy, where $d$ is evidence dispersion and $r$ is evidence redundancy (the number of distinct paraphrased copies of each required fact in the context). We claim three things. First, for fixed $B$, the best grounding length
$$\tau^*(d, r) = \arg\max_{0 \le \tau \le B} A(\tau; d, r)$$
is interior, neither zero nor "ground everything." Second, $\tau^*$ rises with dispersion and falls with redundancy. This mirrors the finding that optimal pilot length falls as SNR or backhaul capacity grows in Optimal Channel Training in Uplink Network MIMO Systems, which proves the net rate is concave in training length under regularity conditions. Third, a model that chooses $\tau$ per query beats the best fixed $\tau$ tuned on a development set at equal average budget.
The wireless result is a source of hypotheses, not a theorem that transfers. Concavity in $\tau$ is an empirical question here, and the first experiment is built to reject it. A second analogy supplies a sharper prediction. In Short Packets over Block-Memoryless Fading Channels, short blocks favor either a single high-power pilot or no pilot at all (noncoherent transmission, about 1.2 dB more efficient than pilot-assisted). The analogue is that for short, dense contexts, terse grounding should beat full paraphrase at equal budget. Terse grounding means emitting only the indices of the chunks used. The question of whether no grounding at all can win is left open.
Why This Is Open
The retrieval-reasoning gap motivates the work. Is It Really Long Context if All You Need Is Retrieval? sorts long-context tasks by how scattered the evidence is and how much of it is needed. It shows that most benchmarks sit in the easy corner. NeedleChain builds chains in which every sentence matters. It finds that GPT-4o drops to 78.5% on backward chains of only 20 facts and approaches zero near 200, while single-fact retrieval stays near 100%. FRAMES shows that even with the gold documents supplied, Gemini Pro 1.5 reaches only about 73%. So retrieval success does not fix reasoning.
Several methods add explicit grounding, and each does so without accounting for its cost. Original Text Paraphrasing (OTP) trains models to index and paraphrase evidence before answering. It gains 8.48% on LongBench for Qwen1.5-4B, but paraphrase length is unconstrained and adds one to three seconds per query. PAM QA uses a fixed three-stage decode (repeat the question, predict the chunk index, summarize the answer) and reaches 44.6 Rouge-L on LongBench. LongRLVR proves that rewarding only the final answer gives almost no learning signal for evidence selection in 64k contexts. It adds a grounding head rewarded by F-score against gold chunks. None of these three asks how long grounding should be. LongAttnComp comes closest to budget reasoning, but on the input side: it fixes a 16k token budget and a chunk size by hand, and it degrades on LongBench v2. Our proposal moves the budget question to the output side and makes it adaptive.
The ecology work supplies the redundancy variable and one specific prediction. HUMAnN and Carnelian show that metabolic function is conserved while the set of species varies. Metabolite essentiality in E. coli shows the flip side: metabolism tolerates single knockouts of redundant routes but crashes when the last bypass is removed. We borrow two things. The first is the measurement idea: count functional carriers (copies of a fact), not surface identities. The second is the crash prediction: accuracy should be flat as copies are removed and then collapse. We make no claim that attention literally behaves like metabolic flux.
Research Program
The work has three parts.
Part 1: controllable testbed. We extend NeedleChain's salary chains with two knobs. Dispersion is controlled by padding and interleaving distractors between needles, following the taxonomy of Goldman et al. Redundancy $r \in \{1,2,4\}$ is controlled by inserting each needed fact as $r$ paraphrases at different positions. We keep Forward, Backward and Mixed orderings, because the direction effects in NeedleChain are among the strongest known failure modes. Because every hop has a known set of gold chunks, grounding precision is measurable.
Part 2: budget sweeps. We fine-tune Qwen2.5-7B-Instruct, and Qwen2.5-14B-1M where compute allows. Training data are synthetic chains whose target outputs contain a grounding segment of controlled length. The segment takes one of three forms at matched length: chunk indices only, short quotations, or full paraphrase. A special token closes the grounding segment. At test time we cap total output at $B \in \{256, 512, 1024\}$ and force $\tau$ to each point on a grid. This yields empirical curves of $A(\tau; d, r)$, which test the interior-optimum claim and the shift of $\tau^*$ with $d$ and $r$ directly.
Part 3: adaptive allocator. First, the model emits a budget token from a small vocabulary of lengths. This is trained with reinforcement learning under a hard cap $B$. The reward combines answer correctness with the LongRLVR grounding F-score, so the grounding gradient does not vanish. The key difference from LongRLVR is that grounding length is a chosen action under a shared budget. We then check whether the chosen length tracks proxies for dispersion and redundancy that the model can compute. One proxy is the entropy of attention over chunks. Another is the number of near-duplicate chunks retrieved per hop.
Baselines
Do nothing. The first reference point is the base instruction model answering under the same cap $B$ with free-form chain-of-thought and no grounding instruction. It shows how much any grounding helps at all.
Best fixed grounding length. This is the most important internal baseline. It uses the same fine-tuned model with $\tau$ fixed at the single value that maximizes development-set accuracy, pooled across all $d$ and $r$. It gets the same training data and the same tuning budget as the adaptive allocator. The allocator beats it only if it wins at equal average tokens, with a paired-bootstrap confidence interval excluding zero on held-out conditions.
LongRLVR. This is the strongest external method. LongRLVR reports 88.90 on RULER-QA and 46.5 on LongBench v2 with Qwen2.5-14B-1M, against 73.17 and 39.8 for answer-only RL. We will rerun it on the same base model, data and number of RL steps, and cap its output at the same $B$. Beating it means matching or exceeding its RULER-QA and LongBench v2 scores under our rerun while using at least 30% fewer output tokens, or exceeding them at equal tokens.
OTP and PAM QA. OTP is the representative long-paraphrase method and PAM QA the representative index-first method. Both are trained on our data with their published output formats. We report them at their natural output length and truncated to $B$. The published gains, 8.48% for OTP on Qwen1.5-4B and 44.6 Rouge-L for PAM QA, are reference points, not targets on our models.
Grounding-form ablation. We compare index-only, quotation and paraphrase grounding at identical $\tau$. This isolates whether the gain comes from the budget choice or from the form of grounding.
Why not just…?
Why not just tune one grounding length on a dev set? This is the real threat. If $\tau^*$ barely moves across dispersion and redundancy, a single tuned constant captures almost all the benefit, and the adaptive allocator is unnecessary machinery. The Part 2 sweeps answer this directly: we report how much accuracy the best fixed $\tau$ loses relative to the per-condition optimum. If that gap is under about one point, we drop the allocator. The contribution then becomes a measured recipe for grounding length and form.
Why not just add a length penalty to LongRLVR? A length penalty might discover an adaptive budget on its own. Nothing in the file shows it fails. The difference is that a penalty trades tokens against accuracy through one global coefficient, whereas our allocator acts under a hard per-query cap. We will run LongRLVR with a length penalty tuned so its average tokens match the allocator. If it matches the allocator on the held-out $(d, r)$ grid, we will report that the explicit budget action adds nothing.
Why not compress the input instead? Input compression such as LongAttnComp beats full context on Code-Debug (76.40% vs 74.37%). However, it scores chunks independently and underperforms full context on LongBench v2. It also does not address the dense-chain failure, where every chunk is relevant. We treat it as complementary. We will run it as a front end to both the fixed and adaptive models on LongBench v2 to check whether the gains stack.
Why not spend the budget on repeated sampling? Majority voting over several short answers is a natural use of the same tokens. The file reports that repeated sampling yields only roughly linear gains against sharply rising failure with reasoning depth. We will still run best-of-$N$ with $N$ chosen to match total tokens as a baseline.
Proposed Experiments and Decisive Checks
Interior optimum. On the NeedleChain extension, we sweep $\tau$ at each $B$ across chain lengths $k \in \{10, 20, 50, 100\}$ and all three orderings. The claim fails if accuracy is monotone in $\tau$ for most conditions.
Movement of the optimum. We regress $\tau^*$ on dispersion and redundancy. The prediction is a positive slope for dispersion and a negative slope for redundancy. The short-packet analogy adds a sharper test: in short, low-dispersion chains, index-only grounding should beat paraphrase at equal $\tau$.
Redundancy knockout. We start at $r = 4$ and delete copies of one required fact one at a time. The prediction is flat accuracy until the last copy, then a collapse. A gradual decline instead would mean the model tracks surface frequency rather than whether the fact is present at all. We also test whether larger $\tau$ delays the collapse.
Transfer to natural data. We evaluate on FRAMES in the oracle setting and against the published Gemini Pro 1.5 numbers (about 73% with oracle documents; 0.408 to 0.660 with multi-step search planning), then on RULER-QA and LongBench v2. We report accuracy against mean output tokens and latency. The headline result is whether the adaptive model sits above the tradeoff curve formed by OTP, PAM QA and LongRLVR.
References
- Optimal Channel Training in Uplink Network MIMO Systems
- Short Packets Over Block-Memoryless Fading Channels: Pilot-Assisted or Noncoherent Transmission?
- Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP
- NeedleChain: Measuring Intact Long-Context Reasoning Capability of Large Language Models
- Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation
- Training with "Paraphrasing the Original Text" Teaches LLM to Better Retrieve in Long-Context Tasks
- Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training
- LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards
- LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning
- Metabolic Reconstruction for Metagenomic Data and Its Application to the Human Microbiome
- Carnelian uncovers hidden functional patterns across diverse study populations from whole metagenome sequencing reads
- Metabolite essentiality elucidates robustness of Escherichia coli metabolism