When a language model answers a question from retrieved documents, it tends to be pulled off course by text that is irrelevant or subtly misleading. The model's soft attention spreads a fixed budget of weight over every token in the prompt, so a crowd of similar distractor passages can outvote the one passage that holds the answer. Current fixes either throw passages away based on a relevance score, which risks dropping the "bridge" fact a multi-step question needs, or retrain the model to ignore noise. We propose a third option borrowed from formal verification: quotienting, which means grouping states that behave identically and keeping one representative per group. Applied to retrieval, it merges passages that have the same effect on the model's answer, keeps one representative per group, and discards nothing that behaves differently. The size of a group of look-alike distractors stops mattering, so a hundred near-copies of a misleading story weigh no more than one. A passage that changes the answer, alone or together with another passage, always keeps its own seat.
The formal-methods literature also brings a warning. Computing exact behavioral equivalence can cost more than it saves. Goal Probability Analysis in Probabilistic Planning reports that bisimulation reduction in probabilistic planning has "prohibitive overhead, sometimes making it impractical." The proposal therefore treats that tradeoff as the object of study. We build an approximate, budgeted equivalence whose cost is paid inside forward passes the system already runs. We then measure directly whether the robustness gain and the compute saved exceed the cost of computing the groups.
Core Claim
Many distraction failures in retrieval-augmented question answering come from redundancy in how passages affect the answer. Removing that redundancy with a query-specific behavioral equivalence recovers much of the robustness that relevance pruning and noise-robust fine-tuning achieve, and it does so without training and with less loss of multi-hop evidence than pruning. The key ingredient is the definition of "same". Two passages are equivalent when the model, conditioned on the query, produces nearly the same answer distribution from each. They must also keep doing so when each is paired with a representative of every other group. That second condition is the retrieval analogue of partition refinement for bisimulation, the standard algorithm that splits groups of states until members behave alike and move alike into other groups. It is what lets bridge passages, which look inert on their own, separate from genuine noise.
How This Relates to Prior Work
The closest system is Superposition Prompting. It processes each retrieved document paired with the query as an independent parallel path, scores each path's saliency, prunes low-scoring paths, and joins the survivors. It reports up to a 93x compute reduction and a 43% accuracy gain over naive retrieval on NaturalQuestions-Open with MPT-7B. We reuse its fork-and-join execution as the substrate but replace its decision rule. Superposition keeps or drops each path by thresholding a score. We merge paths that behave identically and never drop a group whose representative is behaviorally distinct. This is a structural difference rather than a scoring tweak. Superposition cannot tell ten interchangeable distractors from ten independent pieces of evidence, and it has no mechanism for protecting a passage whose value only appears in combination with another.
Three lines of noise-robustness work motivate the problem and constrain the design. Breaking Focus shows that semantically coherent distractors cut accuracy by roughly 45% on average, including a fall from 85.7% to 22.0% on CommonsenseQA for GPT-4o-mini. It also shows that the models answer correctly once the distraction is removed, and that telling the model to "ignore irrelevant info" fails. That result motivates an intervention at the level of structure rather than instruction. Focused Learning fine-tunes models to align representations of noisy and masked contexts. It finds that replacing irrelevant text with a mask token beats deleting it, because deletion destroys layout. That finding constrains us: our join step keeps a positional placeholder for each merged group instead of simply deleting members. Ext2Gen trains the generator to extract evidence before answering and beats decoupled compressors such as Recomp, CompAct, and EXIT, because those filters cannot recover evidence they wrongly discard. This is the main warning for any method that reduces context before generation. Our answer to it is that we only merge passages that behave alike, and behavior is measured by the generator itself rather than by an external filter. Distraction-aware Answer Selection reweights answers by how well each document "generates" the query. It is a per-document scoring baseline in the same training-free spirit.
The formal side supplies the method. Simulation Relations and Applications in Formal Methods surveys partition refinement for bisimulation. It also discusses coarser simulation and "may/must" modal relations as cheaper middle grounds, and we borrow that idea to rank how strict the equivalence must be. DNN-to-automata translation quotients intermediate automata by bisimulation and cuts states by over 80%, and it shows that quotienting must happen early, before the explosion compounds. BQ-NCO quotients combinatorial problem states, and grammar quotienting for constrained decoding quotients a formal grammar. Neither applies equivalence to unstructured retrieved text or to distraction. State Aggregations in Markov Chains shows that one-step aggregation misses structure visible only over longer horizons. That is the reason our equivalence has a second, pairwise round rather than stopping at single-passage behavior.
Research Program
The method is training-free and runs in three stages on top of a fork-and-join executor.
The first stage computes a signature for each passage. For each retrieved passage $d_i$, the model runs the path "preamble, $d_i$, query", which Superposition Prompting already runs. We record $p_i$, the model's distribution over the first $L$ answer tokens, and $h_i$, the final hidden state. Passages are placed in the same initial group when
$$\mathrm{JS}(p_i, p_j) < \epsilon ,$$
where JS is the Jensen–Shannon divergence and $\epsilon$ is a tolerance tuned on held-out data. This round adds essentially no forward passes, only clustering.
The second stage refines groups by pairwise behavior. For each group $B$ we pick a representative $r_B$, the member with the smallest average divergence to the others. We then split any group whose members behave differently when paired with the representative of another group. In other words, $d_i$ and $d_j$ stay together only if, for every other group $C$, running "$d_i$, $r_C$, query" and "$d_j$, $r_C$, query" gives answer distributions within $\epsilon$. This mirrors bisimulation refinement, where states stay together only if their transitions into every other group agree. It is the step meant to separate a bridge passage, which does nothing alone but changes the answer next to the right partner, from a truly inert distractor. Refinement costs roughly (number of passages) × (number of groups) short forward passes per round. We cap the number of rounds at $R$ and treat $R$ as the explicit dial on the tradeoff between reduction cost and execution savings.
The third stage is a quotiented join. Generation attends to one key-value path per final group, using the representative. Each group also gets a single placeholder token that marks where the merged members sat. This follows Focused Learning's evidence that masking beats deletion, and it uses Superposition's position-assignment scheme. The "may/must" idea sets how the budget is spent. Groups whose representative shifts the answer distribution away from the query-only distribution are "must" groups and are always refined and kept. Groups that do not are "may" groups; they are collapsed to a single member and are not refined further unless budget remains.
The analysis side has two parts. We measure how often distraction is a redundancy phenomenon at all, that is, what fraction of harmful context consists of large groups of behaviorally equivalent passages. If that fraction is small, the core premise fails, and we would report that. We also plot, for each benchmark, accuracy and end-to-end latency (reduction cost included) against $R$ and $\epsilon$, to locate the break-even point that the planning literature warns about.
Baselines
The naive reference point is standard retrieval-augmented generation. All retrieved passages are concatenated in retriever order and fed to the same model, with no instruction beyond the task prompt. We also run a second naive variant that adds the "ignore irrelevant information" instruction, since Breaking Focus reports it ineffective and we want to confirm that on our setup. Both use the same retriever outputs, the same passage counts, and greedy decoding.
The strongest existing training-free competitor is Superposition Prompting. It shares our execution substrate, so it isolates the effect of our decision rule. Its reported result is a 43% accuracy improvement over naive retrieval and up to a 93x compute reduction on NaturalQuestions-Open with MPT-7B. On MuSiQue the file gives no number, only the qualitative claim that it was often the only method with non-zero scores on the hardest queries. We therefore rerun it ourselves on MPT-7B and Llama3.1-8b-instruct. It gets the same retrieved sets and a tuning budget for its pruning threshold equal to the budget we give $\epsilon$ and $R$, selected on the same validation split. Beating it means higher exact match on NaturalQuestions-Open and higher F1 on MuSiQue at matched or lower total latency, with latency counting our refinement passes.
The strongest per-document scoring baseline is Distraction-aware Answer Selection. It reports an average 64% exact-match increase over zero-shot readers across four open-domain benchmarks and 231% on SQuAD with DPR and 100 documents. We run it on the same SQuAD/DPR 100-document setting, where dilution is most extreme.
As trained reference points we compare against Focused Learning, which reports 48.1 exact match on NQd (52.0 with re-ranking) and 59.62 F1 on Qasper, and Ext2Gen, which reports an average QA score of 0.557 against 0.473 for default Llama3.1-8b-instruct. We do not expect a training-free method to beat both everywhere. Matching or exceeding their numbers on the multi-hop and heavy-redundancy settings, at lower inference cost, would count as a result. We also test whether quotienting stacks on top of Ext2Gen.
Three ablations isolate the key ingredient. The first stops after the single-passage round, which tests whether pairwise refinement matters. The second groups by embedding similarity of the passages instead of behavior. The third keeps every group member instead of one representative, which separates the benefit of removing redundancy from any side effect of the grouping step.
Why not just…?
Why not just cluster retrieved passages by embedding similarity and deduplicate? This is the most serious threat. It is cheap, and for plain near-duplicate distractors it may work just as well. Our bet is that the passages that do the most harm are the ones where embedding similarity and behavioral effect come apart. The distractors in Breaking Focus are built to be semantically coherent with the question, so they sit close to the gold passage in embedding space and risk being merged with it. MuSiQue bridge passages, meanwhile, are topically distant from the question. The embedding-grouping ablation settles the question directly. If it matches behavioral quotienting on the adversarial and multi-hop splits, then our extra forward passes buy nothing, and we would recommend the cheap method.
Why not just lower Superposition Prompting's pruning threshold so that fewer bridge passages are dropped? A lower threshold keeps more passages but also keeps the whole crowd of redundant distractors. The dilution problem returns, and the speedup shrinks. We sweep Superposition's threshold across its full range and report its accuracy-latency curve against ours. If some threshold dominates our curve, the grouping idea adds nothing beyond a tuned scorer.
Why not just fine-tune for robustness, as Focused Learning or Ext2Gen do? Fine-tuning works, per the numbers above, but it needs training data built around the kinds of noise it anticipates. It also leaves the full noisy context in the prefill, so the cost stays quadratic. Quotienting is training-free, reduces prefill, and can be layered on a fine-tuned model. The stacking experiment on Ext2Gen shows whether the two are complementary or redundant.
Why not just retrieve fewer passages? Truncating to a small top-k removes distractors and bridge evidence alike, and MuSiQue is built so that shortcut-friendly single-hop evidence is not enough (MuSiQue). We include top-k truncation, with k tuned on validation data, as a baseline on every benchmark.
Proposed Experiments
The benchmarks are NaturalQuestions-Open and NQd for single-hop retrieval, SQuAD with DPR at 100 documents for extreme dilution, MuSiQue-Ans for multi-hop reasoning with hard distractors, and Qasper as a stress test for questions that need synthesis across a whole document. Qasper is where Focused Learning flags that aggressive reduction risks losing context. We also add the automatically generated distractors of Breaking Focus to CommonsenseQA and MMLU prompts. The models are MPT-7B, for comparability with Superposition, and Llama3.1-8b-instruct, for comparability with Ext2Gen. The metrics are exact match or F1 as each benchmark defines, supporting-evidence F1 where available, end-to-end wall-clock time including refinement cost, and the number of groups kept.
The first decisive check is a controlled redundancy sweep. We take NQ questions where the model is correct with only the gold passage, then add $m$ paraphrased copies of a single misleading passage for $m$ from 1 to 50. Naive retrieval and Superposition should lose accuracy as $m$ grows. Quotienting should stay roughly flat, because the copies collapse into one group. If quotienting also degrades with $m$, the core claim is false.
The second decisive check is bridge retention on MuSiQue. We measure how often gold supporting passages survive as their own group, comparing single-round against pairwise refinement and against Superposition's pruning. The pairwise round must measurably raise retention and answer F1, or the analogy to bisimulation is decorative.
The third decisive check is the break-even analysis. For each benchmark we report the smallest refinement budget $R$ at which quotienting beats Superposition on accuracy while total latency stays below naive retrieval's. If that budget does not exist on the multi-hop sets, we will say plainly that exact-style refinement costs more than it saves there, as the planning literature found. The negative result would still be informative about where distraction comes from.
References
- Goal Probability Analysis in Probabilistic Planning: Exploring and Enhancing the State of the Art
- Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation
- Breaking Focus: Contextual Distraction Curse in Large Language Models
- Reducing Distraction in Long-Context Language Models by Focused Learning
- Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
- Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering
- Simulation Relations and Applications in Formal Methods
- On Challenges and Opportunities in the Translation of Deep Neural Networks into Finite Automata
- BQ-NCO: Bisimulation Quotienting for Efficient Neural Combinatorial Optimization
- Constrained Decoding for Code Language Models via Efficient Left and Right Quotienting of Context-Sensitive Grammars
- State Aggregations in Markov Chains and Block Models of Networks
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition