When a language model is distracted by irrelevant text in its prompt, it can fail in two very different ways. It can become unsure, so that repeated samples scatter across many answers. Or it can become confidently wrong: it builds a coherent but wrong story, for instance by assigning the donor's lab values to the patient or by attaching the distractor entity to the bridge role in a two-step question, and then repeats that story every time it is asked. Most cheap hallucination detectors only measure scatter, through answer variance or self-agreement across samples. They therefore cannot see the second failure, which is the more dangerous one. This note proposes a way to separate the two failures using a tool from computational social choice: distances between elections that are, or deliberately are not, unchanged when the candidates are renamed.
The idea in two sentences: sample the model's ranking of candidate answers many times and treat the samples as voters in an election, then compare the election produced under the full context with elections produced under modified contexts. The part of the distance that survives an optimal renaming of candidates measures how the model's uncertainty changed. The part that renaming removes measures how much the model swapped which entity sits in which role, so a large gap flags confident, coherent misbinding that variance-based detectors miss.
Core Claim
Election-distance research faced the same blind spot in a cleaner setting. Understanding Distance Measures Among Elections compares metrics for elections under relabeling of candidates and voters. It shows that summary metrics based on average rank (Bordawise) or head-to-head counts (pairwise) assign distance zero between "uniformity", where all rankings are equally common, and "antagonism", where voters split evenly between one ranking and its reverse. It also shows that EMD-positionwise is computable in polynomial time and correlates 0.74 to 0.94 (Pearson) with the NP-hard swap isomorphic distance. EMD-positionwise compares, for each candidate, the distribution of positions it occupies, using Earth Mover's Distance (the minimum cost of moving probability mass from one distribution onto another) after matching candidates optimally. Mapped onto LLM sampling, uniformity is the model with no idea and antagonism is the model torn between the evidence's story and a distractor's story. These call for different responses: abstain in the first case, find and adjudicate the offending passage in the second. Average-rank detectors, and majority-vote agreement, conflate them.
Our claim has two parts. The first concerns election shape. Let $E$ be the election formed by $k$ sampled rankings of $m$ candidates under one context and $F$ the election under another. Let $P_E(c)$ be the distribution of positions of candidate $c$ in $E$. We define
$$d_{\text{anch}}(E,F)=\sum_{c} \mathrm{EMD}\big(P_E(c),P_F(c)\big), \qquad d_{\text{iso}}(E,F)=\min_{\sigma}\sum_{c} \mathrm{EMD}\big(P_E(c),P_F(\sigma(c))\big),$$
where $\sigma$ ranges over bijections between candidates. $d_{\text{iso}}$ is EMD-positionwise; $d_{\text{anch}}$ is the same quantity with candidate names held fixed. The relabeling gap $g=d_{\text{anch}}-d_{\text{iso}}\ge 0$ is the second part. We hypothesize that distractors built as coherent alternative narratives (the "semantic misdirection" mode identified in Breaking Focus) produce high $g$ with low $d_{\text{iso}}$. Clutter-style distraction, such as many unrelated numbers, should instead produce high $d_{\text{iso}}$. The detectors then flag different errors: $g$ catches confident misbinding, while $d_{\text{iso}}$ and variance catch collapse into uncertainty.
One subtlety drives the design. A single ranking is isomorphic to every other ranking, so mapping-invariant distance is meaningless for one output against one gold answer. That is why the obvious construction, "use isomorphic distance to detect role swaps between context and answer", is wrong in principle: relabeling invariance erases exactly the swap one wants to catch. The signal only exists at the level of a profile of samples, and only in the difference between the fixed-name and renaming-invariant distances.
Research Program
The program builds a ground-truth-free detector. For each query we sample $k$ rankings of the candidates under the full retrieved context. Candidates are the multiple-choice options, the entities in a comparison question, or the role fillers in a structured record. We then recompute the election under leave-one-passage-out contexts and under passage reorderings. A passage whose removal produces a large gap $g$ with small $d_{\text{iso}}$ is a candidate "steering" passage: it moves entities between roles without changing how confident the model is.
Removing such a passage does not reveal which reading is correct. We therefore adjudicate between the two elections with the query-generation score $P(\text{query}\mid\text{document})$ from Distraction-aware Answer Selection (DAS) in Improving Zero-shot Reader by Reducing Distractions. DAS combines a "return unanswerable" instruction with down-weighting of answers drawn from documents that poorly reconstruct the query. We reuse that score as a tie-breaker, not as the detector. When the shape distance from the uniform election is small (the model is close to having no idea), the system abstains.
For answers that are sets rather than rankings, the same construction applies with approval ballots: each sample "approves" the answers it gives. The approvalwise distance from How to Sample Approval Elections? correlates at 0.989 Pearson with isomorphic Hamming distance on synthetic approval elections, and we use it as the drop-in surrogate.
Computation is easy here, even though How similar are two elections? proves exact isomorphic distance NP-complete and inapproximable. That hardness is about many candidates. The same paper notes fixed-parameter algorithms in the number of candidates. Our elections have $m\le 7$ candidates, so exact swap-isomorphic distance is feasible: enumerate the $m!$ candidate bijections and solve a voter-matching assignment problem for each. This lets us check, for the first time on real model outputs rather than synthetic statistical cultures, whether EMD-positionwise still tracks the exact distance. The cost that matters is sampling: $k$ samples times (number of passages + 1) contexts per query. We will report accuracy as a function of $k$ and of the ablation budget, and include a cheaper variant that ablates only the top passages ranked by DAS score.
Two kinds of prior work motivate this. Consistency probing in Do Language Models Know When They're Hallucinating References? shows that the variance of answers to repeated indirect queries separates fabricated from grounded citations. That is exactly the variance signal we argue is blind to coherent misbinding, because it assumes a fabricated answer will be unstable. Breaking Focus shows that distraction is an ability failure rather than a knowledge gap: models answer correctly once the distractors are removed. That finding is the premise that justifies context ablation.
The closest structural cousin, Generative Social Choice, uses LLMs as oracles that predict voters' ratings and generate consensus statements, and leaves hallucination to human validation. We invert the roles: the LLM's own samples become the electorate under audit. The profile-cleaning literature constrains how far the analogy can go. Deleting "maverick" voters to reach a structured profile is NP-complete in general (Computational Aspects of Nearly Single-Peaked Electorates, Are there any nicely structured preference profiles nearby?). We therefore do not frame passage removal as exact cleaning, and use greedy leave-one-out instead.
Baselines
The naive reference is the full-context majority answer with no detector, and, for detection, the model's own top-token probability. Improving Zero-shot Reader by Reducing Distractions shows this probability can sit near 0.98 on a distractor-induced wrong answer, so it is a weak but honest floor.
The strongest detection competitor is sample-variance self-consistency: the entropy of the $k$ sampled answers, plus the indirect-plus-direct query ensemble from Do Language Models Know When They're Hallucinating References?. That ensemble gave the highest detection AUC on the CCS citation set (no reported AUC value found on Lacuna in the file; the reported context is hallucination rates of 46.8% for GPT-4 and 73.6% for GPT-3). It is the strongest competitor because it is cheap, training-free, and targets the same goal of flagging wrong outputs without ground truth. We run it with the same $k$, the same sampling temperature and the same model. Our method beats it if it achieves higher AUROC on the subset of errors where the model is self-consistent, while not losing on the full set.
The strongest mitigation competitor is DAS. It reports an average 64% Exact Match gain over naive zero-shot readers across NQ, TriviaQA, WebQuestions and SQuAD, and a 231% relative gain with a dense retriever at 100 documents on SQuAD. We rerun DAS on our exact retrieval pools and models, rather than copying its numbers. Our pipeline beats it if adding gap-triggered passage ablation and adjudication raises Exact Match over DAS alone at a matched number of model calls, with DAS given extra calls in the form of more sampled candidates when needed.
On the noisy-chunk setting we also report the trained Ext2Gen model from Aligning Extraction and Generation. It reports an average QA score of 0.557 against 0.473 for default Llama3.1-8b-instruct with up to 25 distractor chunks. It is a strong but differently resourced competitor, since it is trained with preference optimization. We treat matching it without training as success and exceeding it as a bonus.
Three ablations isolate the key ingredient. The first uses $d_{\text{anch}}$ alone. The second uses $d_{\text{iso}}$ alone. The third replaces EMD-positionwise with the Bordawise (average-rank) distance.
Why not just…?
Why not just use answer variance across samples? Because a model that consistently picks the distractor's entity has zero variance. The detector is useless exactly where Breaking Focus reports its largest damage, an average drop of about 45% and GPT-4o-mini falling from 85.7% to 22.0% on CommonsenseQA under coherent distractors. Whether those failures are actually self-consistent is an empirical question this proposal answers first. If most distracted errors turn out to be high-variance, the gap adds little and we will say so.
Why not just check whether the majority answer changes when a passage is removed? This is the real threat, because it captures much of the leave-one-out signal at lower cost. It loses information in two places. Majority flips miss antagonistic splits, where the model sits near 50/50 before and after. They also miss partial rank changes among non-top candidates, which matter for comparison and multiple-candidate questions. The ablation that substitutes majority-flip rate for $g$, at identical sampling cost, settles this directly. If the two tie, the contribution shrinks to the diagnostic decomposition and the validation of exact versus surrogate distance.
Why not just compare average ranks, or use embedding similarity between outputs? Average rank is the Bordawise metric, which provably confuses uniformity with antagonism (Understanding Distance Measures Among Elections). Embedding similarity rewards outputs that contain all the right entity names in swapped roles. The Bordawise ablation measures how much this matters on real model elections.
Why not just tell the model to ignore irrelevant context, or compress the context first? Breaking Focus reports that such instructions are ineffective against coherent distractors. Aligning Extraction and Generation finds that separate compressors such as Recomp, CompAct and EXIT drop multi-step bridging clues. We include the "ignore" prompt as an additional arm of the naive baseline.
Proposed Experiments
Data. We build elections on four sources:
- HotpotQA distractor-setting comparison and bridge questions (HotpotQA). The reported baseline answer F1 is 58.28 against 91.40 for humans. Gold supporting sentences let us label which passages are distractors.
- CDV-perturbed MMLU, CommonsenseQA and TruthfulQA, with the model ranking the answer options.
- The Ext2Gen noisy-chunk set.
- A synthetic role-binding suite modeled on the donor/recipient confusions described for MEDIMP. Here we inject distractors of two controlled kinds: coherent alternative narratives, and numeric clutter.
Metrics. We report detection AUROC overall and on the self-consistent-error subset, and Exact Match or F1 after mitigation at matched call budgets. For the diagnostic, we report agreement between the decomposition and the injected distractor type on the synthetic suite.
Decisive checks. Three results decide the claim. First, on the synthetic suite, narrative distractors must yield high $g$ with low $d_{\text{iso}}$ and clutter must yield the reverse. If both kinds look the same, the decomposition is not real. Second, on HotpotQA and the CDV sets, $g$ must beat variance self-consistency on self-consistent errors. Third, exact swap-isomorphic distance, computed on these small-candidate elections, must correlate with EMD-positionwise at a level comparable to the synthetic 0.74 to 0.94 range. If it does not, the surrogate is unsafe for model outputs and we fall back to exact computation, which remains feasible at $m\le 7$.
References
- Understanding Distance Measures Among Elections
- Breaking Focus: Contextual Distraction Curse in Large Language Models
- Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering
- How to Sample Approval Elections?
- How Similar Are Two Elections?
- Do Language Models Know When They're Hallucinating References?
- Generative Social Choice
- Computational Aspects of Nearly Single-Peaked Electorates
- Are There Any Nicely Structured Preference Profiles Nearby?
- Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
- MEDIMP: 3D Medical Images and clinical Prompts for renal transplant representation learning