Language models that read long inputs are worse at using information placed in the middle of the input than at the start or end. This is the "lost in the middle" effect. The standard evidence for it comes from question answering over twenty Wikipedia passages. In that benchmark the distractor passages were chosen by BM25, a keyword-matching search engine, using the question as the query. So every distractor shares surface words with the question, and nobody has measured how much of the middle-of-context failure comes from the model being pulled toward keyword-matching decoys rather than from position alone.
The same confound sits inside the most promising training fix. Recent work rewards models, through reinforcement learning (RL, training a model by rewarding its own sampled outputs), for citing the correct evidence chunk before answering. If the correct chunk is usually also the one that repeats the question's words, a model can collect that reward by learning keyword matching instead of reading. This proposal builds a test that breaks the link between "shares words with the question" and "contains the answer." We call this lexical inversion. We use it to measure how much of the lost-in-the-middle curve is lexical attraction, and to find out whether grounding-rewarded RL avoids keyword shortcuts that supervised training is known to fall into.
Core Claim
We make two claims. The first is diagnostic: on natural-language multi-document QA, the middle-position drop grows with how strongly the distractors overlap the question in keywords. Under lexical inversion, where the gold passage is reworded to share few question words and the distractors share many, the U-shaped curve will deepen well beyond what position alone predicts.
The second concerns training. The context reward in LongRLVR scores chunk citations with an F-score against gold chunks. That reward is only as good as the data it is computed on. If the gold chunk's keyword-search rank is uniformly spread across the chunks in every training context, keyword matching earns no expected reward, and the policy must find another signal. The open question is whether RL on such decorrelated data learns reading-based grounding, or whether, like supervised models in Stubborn Lexical Bias in Data and Models, it shifts to higher-order surface cues such as bigrams or entity strings. We expect the answer to differ between supervised fine-tuning (SFT) and RL. We treat that difference as the empirical question, not as a foregone conclusion.
How This Relates to Prior Work
Two papers motivate the diagnostic. Lost in the Middle established the U-shaped curve with BM25-retrieved distractors. For example, GPT-3.5-Turbo on 20-document QA falls from about 75% with the answer first to about 53% with it in the middle. The paper never varied distractor overlap. Its only overlap-free control is a key–value task over random UUIDs, which removes natural language entirely. Positional Contrastive Decoding attributes the drop to nearby tokens drowning out distant ones and corrects for it at inference by position alone. Our factorial design asks how much of the failure a purely positional correction can reach.
Two papers supply methods. We reuse LongRLVR's split between a grounding step and an answer step and its context reward, running from its public code (LongRLVR code). From Never Lost in the Middle (PAM QA) we take the SFT recipe. It has the model repeat the question, predict the source passage index, and then answer, with gold passages placed at random positions. We differ from both in one place: how evidence is constructed. PAM QA's hard negatives are chosen for semantic similarity, and LongRLVR's questions are written by a teacher model looking at the gold chunk. Neither controls the gold chunk's lexical rank.
Two papers constrain the design. Stubborn Lexical Bias shows that balancing unigram–label statistics leaves models free to exploit bigrams, with bigram divergence rising from 0.059 to 0.122 on SNLI. This rules out declaring victory after balancing a single overlap statistic. We therefore decorrelate several overlap measures jointly and audit residual cues afterward. Lexical Repetitions Lead to Rote Learning shows that partitioning test sets by n-gram overlap exposes memorization. We borrow that partitioning logic for our evaluation splits.
Research Program
The first component is a controlled benchmark. We start from the multi-document QA setup of Lost in the Middle and cross three factors:
- Gold position: first, quarter, middle, three-quarter, or last, among twenty passages.
- Distractor overlap: low (random passages without the answer), natural (BM25 top hits), or adversarial (passages chosen to maximize question-term and bigram overlap while verified not to contain the answer).
- Gold overlap: original wording, or reworded by a large teacher model to minimize shared content words and bigrams with the question.
Every reworded gold passage must pass an answer-preservation check: a separate model must still extract the gold answer from it in isolation.
Rewording creates a style artifact, because the rewritten passage may be detectable as "the odd one out." To remove it, every distractor is passed through the same rewording model with overlap held at its target level. A reader model given the passage alone, without the question, should then be at chance when asked to identify the gold passage. We report that check explicitly.
We add the same inversion to RULER-style QA (RULER) at 16K and 32K tokens. To separate retrieval from memorized knowledge, we follow 100-LongBench and report closed-book accuracy for every item, and we score only items the model fails closed-book.
The second component is a lexical-reliance measure, in addition to accuracy. For each model, we compute how often its cited chunk is the BM25 top chunk when that chunk is wrong. We also compare accuracy on "aligned" items (gold is the top BM25 hit) against "inverted" items (gold ranks last by BM25), using the permutation test of Stubborn Lexical Bias. A model that reads should show a small aligned–inverted gap. A keyword matcher should show a large one.
The third component is training on decorrelated data. We generate LongRLVR-style training contexts and then resample distractors and reword gold chunks so that the gold chunk's rank under BM25, under bigram overlap, and under named-entity overlap is each roughly uniform. We train three arms on the same base model, Qwen2.5-7B, with identical data and token budgets:
- SFT arm: PAM QA-style SFT only.
- Outcome-only RL arm: SFT followed by RL rewarded only on answer correctness.
- Grounding-reward RL arm: SFT followed by RL with LongRLVR's context reward.
After training, we probe the models for new shortcuts. We build test items in which one residual cue, such as a rare entity string shared with the question, points to a distractor. We then check whether accuracy collapses on those items.
Baselines
The do-nothing reference is the base instruction-tuned model, prompted to answer directly, on every cell of the factorial benchmark. It sets the size of the lexical effect before any intervention.
The training-free positional baseline is Positional Contrastive Decoding applied to the same base model. It boosted Llama-3-8B from 72.0% to 79.0% on InfiniteBench 8K key–value retrieval and variable-tracking F1 from 64.40 to 69.04 on RULER at 16K. We run it with its published contrast strength, tuned once on a held-out slice of natural-overlap items. It is the right test of whether a positional fix alone flattens the inverted curve.
The reranking baseline uses BM25 to reorder passages so the top hit sits first. It exploits primacy directly. By construction it should fail on inverted items, which confirms that the benchmark measures what we intend.
The strongest existing training method is LongRLVR trained on its original synthetic data. For Qwen2.5-14B-1M it reports 88.90 on RULER-QA (against 73.17 for standard RLVR) and 46.5 on LongBench v2 (against 39.8). We retrain it at 7B on data of the same size and compute budget as our arms so that the comparison isolates data construction. The strongest SFT competitor is PAM QA, which reports 44.6 Rouge-L on LongBench under document shuffling. No reported result was found on Lacuna for any method on lexically inverted multi-document QA; those numbers will come from this study.
The key ablation compares LongRLVR on original data with LongRLVR on decorrelated data, holding algorithm, compute, and base model fixed. A second ablation compares SFT with RL on decorrelated data, which isolates the training regime.
The proposal beats the strongest baseline if two things hold. First, the aligned–inverted accuracy gap must shrink by at least half relative to LongRLVR on original data. Second, RULER-QA and LongBench v2 scores must stay within about one point of that model.
Why not just…?
Why not just add lexically adversarial hard negatives to PAM QA's SFT data? This is a serious threat. Decorrelation is a data property, so SFT may capture most of the gain. That is why the decorrelated-SFT arm exists and receives the same data as the RL arms. Stubborn Lexical Bias gives a reason to expect SFT to shift toward bigram or entity cues: cross-entropy training on a fixed set of demonstrations rewards whatever surface statistic remains. Only the residual-cue probes can confirm this. If decorrelated SFT matches decorrelated RL on those probes, we will report that data construction, not RL, is the contribution, and recommend the cheaper recipe.
Why not just use a dense or cross-encoder reranker instead of BM25, then put the top hit first? A semantic reranker avoids exact-term matching, but its own tendency to reward overlap is untested here, and placing one passage first discards multi-passage evidence. Reranker-plus-primacy models also dropped up to 17.3% under shuffling in the PAM QA study. We will include a cross-encoder variant in the reranking baseline. If it holds up on inverted items, the diagnostic finding still stands, but the training result becomes less necessary for single-hop QA. Our RULER multi-hop items test whether that advantage survives beyond single-hop questions.
Why not just apply positional contrastive decoding? It corrects for distance, not for content, so it cannot tell a keyword decoy from the gold passage at the same position. The factorial benchmark measures this directly. If positional contrastive decoding flattens the curve equally at all overlap levels, lexical attraction is a minor factor, and the first claim fails. We treat that as an informative outcome.
Decisive Checks
The diagnostic claim stands or falls on an interaction effect. At matched position, accuracy on middle positions must fall more steeply from low to adversarial distractor overlap than accuracy on end positions does. This must hold after the style-artifact control and after excluding items the model answers closed-book. If there is no interaction, the lost-in-the-middle curve is mostly positional and the positional account in the PCD paper is sufficient.
The training claim stands or falls on three measurements, taken on held-out inverted items and residual-cue probes at 16K and 32K tokens:
- LongRLVR on original data shows a substantial aligned–inverted gap.
- Decorrelated grounding-reward RL shrinks that gap.
- Decorrelated SFT either shrinks it equally, in which case data construction is the story, or leaves a gap that reappears on residual-cue probes, in which case the reward-driven regime adds something SFT lacks.
Each outcome points to a different design choice for building models that can actually read their context.
References
- LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards
- Stubborn Lexical Bias in Data and Models
- Lost in the Middle: How Language Models Use Long Contexts
- Mitigating Posterior Salience Attenuation in Long-Context LLMs with Positional Contrastive Decoding
- Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training
- Lexical Repetitions Lead to Rote Learning: Unveiling the Impact of Lexical Overlap in Train and Test Reference Summaries
- RULER: What’s the Real Context Size of Your Long-Context Language Models?
- 100-LongBench: Are de facto Long-Context Benchmarks Literally Evaluating Long-Context Ability?