When a question is ambiguous, such as "who won the gold medal in figure skating?" (which year? which event?), a retrieval-augmented system has to do two things. It has to find evidence for every reasonable reading of the question, and it has to know which readings the evidence actually supports. A retrieval-augmented system is a language model that first looks up passages in a document collection and then writes its answer from them. Current systems handle the second step crudely. Diversify-verify-adapt (DIVA) asks a model to propose several interpretations, retrieves for each, and then makes one yes/no decision for the whole question: either the retrieved passages are "useful" and the answer is written from them, or they are "useless" and the model falls back on its own memory. Real retrieval is usually partial. It covers the popular reading well, one minor reading thinly, and another not at all. A single switch cannot express that.
We propose to replace the switch with per-interpretation bookkeeping. The interpretations become a weighted distribution, with weights estimated by repeated sampling. The retrieved passages become a second distribution. An unbalanced optimal transport problem is solved between the two. Optimal transport is the "earth mover's" computation of the cheapest way to move mass from one distribution onto another; "unbalanced" means some mass is allowed to stay unmoved, at a price. The interpretation mass left unmoved is a direct, graded measure of which readings the evidence fails to support. That measure is used three ways. It triggers targeted re-retrieval only for the uncovered readings. It routes whatever stays uncovered to an explicitly flagged answer from the model's memory. It decides which passages fill a fixed context budget.
Core Claim
Ambiguous-question systems lose most when retrieval is partially right, not when it is entirely wrong. A per-interpretation coverage signal, weighted by how plausible each interpretation is, recovers missing readings more cheaply and more reliably than whole-question verification. The claim has two parts, and they can be tested separately. The first part is that per-interpretation, mass-weighted accounting beats a whole-question switch. The second part is that solving a transport problem adds something beyond simply checking, for each interpretation, its best-matching passage. The transport problem forces passages to be shared out according to interpretation weight under a fixed passage budget.
The second part is the one a reviewer should doubt. The experiments are designed so that if it fails, the first part still stands on its own and is reported as the contribution.
Research Program
The pipeline starts from DIVA's diversification step. We then add three components.
The first component is an interpretation distribution. Instead of asking the model once for a list of interpretations, we sample $N=20$ lists, embed the proposed disambiguated questions, merge near-duplicates, and set each interpretation's weight $a_i$ to the fraction of samples that produced it. This choice follows the negative result in SelfReflect. That paper measures how faithfully a model's text describes its own answer distribution using a Wasserstein distance. It found that single-pass verbalizations of uncertainty did worse than a greedy baseline across 20 models, and that only sample-then-summarize was faithful. SelfReflect uses transport only as an offline metric. We use sampled frequencies as the input mass of an active retrieval decision.
The second component is the coverage transport. Retrieved passages receive mass $b_j$ proportional to retriever score. The cost of moving interpretation $i$ onto passage $j$ is $c_{ij} = 1 - s_{ij}$, where $s_{ij}$ is a cross-encoder's probability that passage $j$ answers disambiguated question $i$. We deliberately avoid raw embedding distance here. From Word Embeddings To Document Distances, which introduced document transport distances, showed them degrading when the embedding geometry is poor, as on its clinical OHSUMED data. That warns against relying on raw embedding geometry. We solve
$$\min_{T \ge 0} \sum_{i,j} T_{ij} c_{ij} + \lambda\, \mathrm{KL}(T\mathbf{1} \,\|\, a) + \mu\, \mathrm{KL}(T^\top \mathbf{1} \,\|\, b)$$
using entropic Sinkhorn iterations, as in MBR-OT. In this objective, $T_{ij}$ is the mass of interpretation $i$ assigned to passage $j$. $\lambda$ sets how expensive it is to leave interpretation mass unmoved. $\mu$ sets how freely a passage can absorb more or less mass than its prior. A large $\mu$-slack lets one table-like passage legitimately cover several readings. The uncovered mass is $u_i = a_i - \sum_j T_{ij}$.
The third component is graded routing. Interpretations with $u_i/a_i$ above a threshold get one targeted retrieval round using only their disambiguated question. Anything still uncovered is answered from the model's memory and marked as uncited in the output. Under a fixed budget of $k$ passages, the context is filled with the passages carrying the most transported mass, so the budget is split across readings in proportion to their weight. The generator is prompted with passages grouped by the interpretation they were assigned to.
The added cost is $N$ short sampling calls, one cross-encoder pass over interpretation-passage pairs, and a small Sinkhorn solve. MBR-OT reports that Sinkhorn over candidate sets made decoding about four times slower than standard MBR decoding. Our transport problem is far smaller, since it involves tens of interpretations against tens of passages. We will report wall-clock time and LLM call counts rather than assume the overhead is negligible.
Baselines
The naive reference is vanilla retrieval-augmented generation: top-$k$ passages for the raw question, concatenated. ALCE, a benchmark for citation quality in retrieval-augmented answers, found this surprisingly competitive with interactive retrieval strategies. DIVA reports 39.6 Disambig-F1 for it on ASQA with GPT-4. Disambig-F1 is the fraction of an ambiguous question's per-interpretation short answers that a reading-comprehension model can find in the long answer.
The strongest existing method is DIVA itself, at 42.0 Disambig-F1 on ASQA with GPT-4, ahead of Iterative RAG at 41.8 and reportedly about twice as fast. It is the right competitor because it already diversifies retrieval across interpretations and already falls back to memory. Our proposal changes only the decision layer. We will re-run DIVA, Iterative RAG, and our method with the same open generator, the same retriever and corpus, the same $k$, and the same tuning budget for thresholds, set on a held-out ASQA development split. The published 42.0 serves as a sanity anchor, not the comparison target. Beating DIVA means a Disambig-F1 gain on ASQA outside the bootstrap confidence interval, at no more than DIVA's LLM-call count, with citation recall no worse.
Tree of Clarifications recursively branches and prunes disambiguated questions with up to 20 LLM calls and reports 33.7 Disambig-F1 with a 5-shot GPT-3 setup. We include it as the high-cost reference.
Three ablations isolate the key ingredients. Uniform interpretation weights replace sampled frequencies. A "per-facet max" variant marks interpretation $i$ covered if $\max_j s_{ij}$ exceeds a threshold and fills the context by round-robin top passages per interpretation. A "per-facet LLM verifier" variant applies DIVA's verification prompt separately to each interpretation.
Why not just…?
Why not just check each interpretation's best passage instead of solving a transport problem? This is the most serious threat. Per-facet thresholding on $\max_j s_{ij}$ gives a per-interpretation coverage signal at almost no cost. Transport differs in two ways. It accounts for weight, so a dominant reading needs more than one weak passage. It shares a fixed budget, so the minor readings are not crowded out by ten near-duplicate passages about the popular one. Whether either difference matters is an empirical question. The per-facet-max ablation answers it under an identical $k$, and especially at small $k$, where budget sharing should matter most. If the two tie, we will say so and present the contribution as per-interpretation routing, with transport as unnecessary machinery.
Why not just ask the LLM to verify each interpretation separately? Per-facet LLM verification is likely accurate. It costs one call per interpretation, though, and it returns a binary judgment with no notion of how much evidence is enough. We compare it at its natural cost and also at a matched call budget. If it wins at matched cost, the cross-encoder cost is the weak link, and we would substitute verifier probabilities for $s_{ij}$ inside the same transport.
Why not just retrieve many more passages and let a long-context model sort it out? ASQA's own results argue against this. Even with gold passages supplied, models scored 52.5 on the Disambiguation-ROUGE score against a human 61.8 (ASQA). ALCE reports incomplete citation support about half the time on ELI5. More context does not by itself make a model cover every reading. We still run vanilla retrieval at $k$ equal to the total passages our method touches, so that the gain is not simply more evidence.
Proposed Experiments
The main benchmark is ASQA, evaluated with Disambig-F1, Disambiguation-ROUGE, and ALCE's citation recall and precision, using the ALCE code. The file reports no ASQA citation numbers from ALCE, so all citation figures will be our own reruns. Transfer is tested on AmbigQA multi-answer recall and SituatedQA. No reported results were found on Lacuna for either under these pipelines. Natural Questions and TriviaQA serve as controls where most questions are clear. There, the method should mostly collapse to one interpretation and should not hurt accuracy.
The decisive check is a stratified analysis. Using ASQA's gold disambiguated answers, we label each question by the fraction of its interpretations whose answer appears in the initial retrieval: none, partial, or full. The hypothesis predicts that gains over DIVA concentrate in the partial stratum. It predicts a near-tie in the full stratum and a small or zero gain in the none stratum, where DIVA's memory fallback already does the right thing. A uniform gain across all strata would suggest the improvement comes from something other than coverage accounting.
The second check is whether the coverage signal is honest. Across all interpretation instances, we measure how well $u_i/a_i$ predicts that interpretation's gold short answer being absent from the final answer, using area under the ROC curve. We compare it against $1-\max_j s_{ij}$ and DIVA's whole-question verdict.
The third check studies weights. We compare sampled weights with uniform weights, and $N=1$ with $N=20$, to test whether SelfReflect's finding carries over from describing uncertainty to acting on it.
All results are reported alongside LLM call counts and latency, because a gain that requires Tree-of-Clarifications-level cost would not answer the question this note asks.
References
- Diversify-verify-adapt: Efficient and Robust Retrieval-Augmented Ambiguous Question Answering
- SelfReflect: Can LLMs Communicate Their Internal Answer Distribution?
- From Word Embeddings To Document Distances
- Document-Level Text Generation with Minimum Bayes Risk Decoding using Optimal Transport
- Enabling Large Language Models to Generate Text with Citations
- Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models
- ASQA: Factoid Questions Meet Long-Form Answers