Retrieval-augmented generation (RAG) means handing a language model a few retrieved passages and asking it to answer from them. It is supposed to reduce hallucination, meaning fluent statements that are false or unsupported. In practice it often does not. On the CRAG benchmark, a straightforward RAG pipeline answers about 44% of questions correctly, but its truthfulness score (which subtracts hallucinated answers) is only about 13.4%. Recent work blames part of this on redundancy: the top-ranked passages repeat one fact and miss the other facts a multi-step question needs. That work fixes it by making the retrieved set more diverse. A separate literature shows that language models are easily derailed by passages that are on-topic but irrelevant, and the most harmful ones mention the same names and roles as the question. These are exactly the passages a generic "make the set diverse" rule tends to pull in.
This note proposes to measure diversity on the question side instead of the passage side. We break the question into the pieces of information it needs, such as its sub-questions and bridge facts. We then choose passages with a submodular objective, a set score with built-in diminishing returns. The score rewards covering each needed piece up to a saturation cap and gives nothing for anything else. A passage that is on-topic but answers no piece earns zero gain, however different it looks from the passages already chosen. A second passage for an already-covered piece earns little. Because the objective is monotone, greedy selection has a natural stopping rule: stop when no passage adds enough coverage per token. The context then shrinks, possibly to nothing, when the pool has nothing useful. The claim to test is that this selection lowers hallucination, measured by CRAG truthfulness and by how often the model copies a wrong answer from a distractor passage, relative to both plain top-k retrieval and passage-space diversity methods, at equal or smaller context size.
Core Claim
Existing diversity-aware RAG methods spread the selected set out in embedding space. Diversity Enhances an LLM's Performance in RAG and Long-context Task uses Maximal Marginal Relevance (MMR, which scores each passage by relevance minus its similarity to passages already chosen) and farthest-point sampling. Vendi-RAG mixes similarity with a Vendi Score, an eigenvalue-based "effective number of distinct items" measure, and tunes the mix through an LLM-judge feedback loop. These methods motivate the proposal. They show redundancy is real: MMR raises the rate at which the gold answer appears in the context by 2% to 5%, and Vendi-RAG beats Adaptive-RAG by up to 4.2% accuracy on multi-hop QA. Neither reports hallucination measurements.
Two papers constrain the design. Large Language Models Can Be Easily Distracted by Irrelevant Context shows that in-topic distractors with overlapping role names are the most damaging. Under such noise, macro-accuracy with chain-of-thought prompting falls below 30%. Making Retrieval-Augmented Language Models Robust to Irrelevant Context shows models copy entities out of distractor passages. It also finds that on implicit multi-hop questions such as StrategyQA, retrieval can do worse than no retrieval.
Our claim is that passage-space diversity and distraction are two sides of one design choice. If diversity is scored against the question's information needs, with saturation, redundancy and distraction are suppressed by the same mechanism, and the context size adapts to the question.
The Selection Objective
We first make one LLM call to decompose the question $q$ into $m$ information needs. Each need $j$ has an importance weight $w_j$. A bridge need whose entity is not yet known is written with a placeholder, for example "birthplace of the director of X". Next, a cross-encoder reader produces an answerability score $c(p,j) \in [0,1]$ for each candidate passage $p$ and need $j$. A cross-encoder reads the need and the passage jointly and scores whether the passage answers it. We then choose the set $S$ to maximize
$$F(S) = \sum_{j=1}^{m} w_j \,\min\Big(\sum_{p \in S} c(p,j),\ \alpha\Big),$$
where $\alpha$ is the saturation cap: the most coverage credited to any single need. This is the saturated-coverage form from A Class of Submodular Functions for Document Summarization. That paper showed such functions are monotone and submodular, that greedy selection comes within $(1-1/e)$ of optimal, and that saturation beat heuristic redundancy penalties on DUC-04 summarization (ROUGE-1 rose from 32.44% to 38.65%). Setting $\alpha$ above 1 allows a second corroborating passage when the reader's scores are uncertain, but not a third or fourth copy.
We select greedily by gain per token under a token budget $B$, and we stop early once the best gain per token falls below a threshold $\tau$. Greedy selection by cost-benefit ratio under a length budget is the standard route to approximation guarantees for monotone submodular objectives with knapsack-style constraints. Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints studies harder variants.
Bridge needs are resolved sequentially. After a passage covering need 1 is selected, the reader extracts the bridge entity, and need 2 is rewritten and rescored. This is adaptive selection in the sense of Near-optimal Batch Mode Active Learning and Adaptive Submodular Optimization. Its guarantees assume the observations are independent, and linked multi-hop facts are not. We therefore treat this step as a heuristic and report both the placeholder variant and the rewritten variant.
The closest prior work in spirit is MASCOT. It applies model-aware submodular coverage over the composite attributes of a text-to-image query, and so also covers the query rather than the gallery. It targets image retrieval diversity, not evidence sufficiency or hallucination. DIVERGE also conditions retrieval on generated "viewpoints". It does so for open-ended questions with many valid answers, uses an iterative multi-agent MMR loop, and does not measure factual truthfulness. Submodular demonstration selection (Submodular Context Partitioning and Compression for In-Context Learning; COBRA) chooses few-shot examples, not factual evidence.
Research Program
The work has three parts.
Build the selector. We implement it on a fixed candidate pool: the top 50 passages from a combined sparse and dense retriever. It costs one decomposition call plus cross-encoder scoring, with no LLM-judge loop. This is a direct contrast to Vendi-RAG and DIVERGE.
Measure hallucination, not just accuracy. We report the CRAG score, which gives +1 for a correct answer, 0 for a missing one, and −1 for a hallucinated one, along with truthfulness. We also report a distractor-copy rate: the fraction of wrong answers whose answer string appears verbatim in a selected passage that is not a gold support. This operationalizes the copying trap described by Yoran et al.
Diagnose the mechanism. On HotpotQA (Yang et al.), sentence-level supporting-fact labels let us measure the precision and recall of the selected set against gold evidence. We can then check whether gains come from admitting fewer distractors, from covering more gold facts, or from both.
We hold passage order fixed across all methods: evidence for the highest-weight need goes at the prompt's boundaries and the rest follows original order. This keeps the known "lost in the middle" effect from confounding the comparison. We do not study ordering.
Baselines
The naive reference points are closed-book generation and straightforward top-k similarity RAG. For CRAG the file gives reported numbers, though under the benchmark authors' own setup. With GPT-4 Turbo, LLM-only accuracy is about 34%, and straightforward RAG reaches about 44% accuracy with about 13.4% truthfulness in Task 3 (CRAG). Industry systems reach 51% truthfulness (Copilot Pro); we cite this only as a ceiling reference, not a like-for-like comparison. We rerun top-k with our own reader at k equal to 5, 10, and 20, and at a k matched to our method's average passage count.
The strongest passage-space diversity competitor is Vendi-RAG. It is the only prior method evaluated on HotpotQA, MuSiQue, and 2WikiMultiHopQA with multi-hop gains. It reports "up to 4.2%" over Adaptive-RAG; no absolute accuracy or any hallucination number for it was found on Lacuna. We run it with its LLM-judge loop, giving it the same reader and candidate pool, and we report its extra LLM calls separately.
The second diversity baseline is MMR as used in Diversity Enhances, with its trade-off weight tuned on the same development split. For its recall gain, the file reports only a 2% to 5% range, not an absolute figure.
The strongest anti-hallucination competitor is the NLI filter with fallback to closed-book answering from Yoran et al. We use it both on its own and stacked on top-k. SA-RetRobust fine-tuning is reported to improve 2WikiMQA accuracy "by over 30 points compared to the no-retrieval baseline". Where compute allows, we run our selector on top of that fine-tuned reader as well, since the two approaches may be complementary rather than competing.
Every method gets the same candidate pool, reader LLM, maximum token budget, and number of hyperparameter trials on the same development split. Beating the field means a higher CRAG score and truthfulness than every baseline at equal or smaller average context length, with a lower distractor-copy rate, and multi-hop EM/F1 no worse than Vendi-RAG.
Three ablations isolate the key ingredient. First, removing saturation turns the objective into a plain weighted sum and tests whether diminishing returns matter. Second, facility-location coverage over passage embeddings keeps submodularity and greedy selection but moves coverage back to the passage side. This is the central ablation: it tests whether the question side is what helps, or merely submodularity. Third, a fixed k versus the adaptive stopping rule tests whether shrinking the context is where the gains come from.
Why not just…?
Why not just retrieve more passages? Vendi-RAG reports that standard RAG plateaus or degrades as the context fills with near-duplicates. Shi et al. show that added non-essential text can collapse accuracy. We still sweep k, because if top-20 matched our truthfulness, the proposal would fail. We expect the gap to be largest on CRAG multi-hop and false-premise questions and on the distractor-copy rate.
Why not just tune MMR? MMR subtracts a redundancy penalty, which breaks monotonicity and leaves greedy selection without its approximation guarantee (Lin and Bilmes). More importantly, MMR's notion of "different" is embedding distance. An in-topic distractor with overlapping role names can be both similar to the query and different from the chosen passages, which is exactly what MMR rewards. The facility-location ablation and the HotpotQA distractor-precision analysis test this directly.
Why not just rerank with a cross-encoder and keep passages above a threshold? This is a real threat, because it also filters distractors cheaply. What it lacks is any notion that two passages answering the same sub-question are redundant, or that one weaker passage covering a missing bridge fact is worth more than a third strong passage on a covered fact. We run it with its threshold tuned on the same development split. If it matches us on CRAG truthfulness and HotpotQA supporting-fact recall, the coverage structure adds nothing and we will report that.
Why not just decompose the question and take the top passage per sub-question? This is the most serious alternative, because it captures most of the intuition with no optimization at all. Our method differs in three ways: it credits passages that cover several needs at once, it allows corroboration up to the cap $\alpha$ when scores are uncertain, and it selects nothing when no passage is informative. The last of these should matter on StrategyQA, where retrieval is known to hurt. We run this baseline with the identical decomposition and reader. If it ties us, the honest conclusion is that question-side facets, not submodular selection, are the contribution, and the note's claim narrows accordingly.
Decisive Experiments
The main test is on CRAG Task 3. It compares CRAG score, truthfulness, and hallucination rate across all baselines at matched average context length, broken down by question type and by head, torso, and tail entities.
The mechanism test is on HotpotQA in both the distractor and full-wiki settings. It compares the precision and recall of the selected set against gold supporting facts, and the distractor-copy rate, between our selector and the facility-location ablation. If question-side coverage does not reduce the number of admitted distractors relative to passage-side coverage, the core claim is false.
The abstention test is on StrategyQA. If retrieval hurts there, the adaptive stopping rule should select fewer passages and recover at least closed-book accuracy, which fixed-k methods cannot do.
Generality is checked on MuSiQue and 2WikiMultiHopQA, against Vendi-RAG and against RaDIO's reported 2WikiMultihopQA EM of 0.254 with its SBERT retriever (RaDIO). That comparison is loose, because RaDIO optimizes when to retrieve rather than what to keep.
Throughout, we report the number of LLM calls and the latency per question. The single-pass selector must not buy its gains with hidden compute.
References
- Diversity Enhances an LLM's Performance in RAG and Long-context Task
- Vendi-RAG: Adaptively Trading-Off Diversity And Quality Significantly Improves Retrieval Augmented Generation With LLMs
- Large Language Models Can Be Easily Distracted by Irrelevant Context
- Making Retrieval-Augmented Language Models Robust to Irrelevant Context
- A Class of Submodular Functions for Document Summarization
- Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints
- Near-optimal Batch Mode Active Learning and Adaptive Submodular Optimization
- MASCOT: Model-Aware Submodular Coverage for Composite-Attribute Text-to-Image Retrieval
- DIVERGE: Diversity-Enhanced RAG for Open-Ended Information Seeking
- Submodular Context Partitioning and Compression for In-Context Learning
- COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
- CRAG - Comprehensive RAG Benchmark
- RaDIO: Real-Time Hallucination Detection with Contextual Index Optimized Query Formulation for Dynamic Retrieval Augmented Generation