Choosing Which Search Queries to Label by Comparing What the Retriever Believes

A dense retriever is a neural model that maps queries and documents to vectors and returns the documents whose vectors are closest to the query. When such a model trained on web search is moved to a new domain, such as financial questions, retrieval quality drops. Fixing it requires people to judge which documents are relevant to which queries, and those judgments are expensive. Active learning tries to spend that labeling budget wisely: it lets the model pick the queries it would learn the most from. The usual recipe picks queries where the model looks uncertain, then spreads the picks out so they are not all alike. Both halves misfire in retrieval. A query can look uncertain only because many documents really are relevant to it, which labeling cannot fix. And two queries can sit far apart in embedding space while sending the retriever to exactly the same confused set of documents, so labeling both is wasted.

We propose to describe each unlabeled query by the retriever's belief: a probability distribution over regions of the corpus where the retriever thinks the answer lives. We then do two things with these beliefs. First, we measure redundancy between queries with the Hellinger distance between beliefs, a standard distance between probability distributions. Second, we measure uncertainty as disagreement between several noisy copies of the retriever, which counts only the confusion that more labels could remove. Hellinger distance turns into ordinary Euclidean distance once you take square roots of the probabilities. That means the diversity step runs on the same nearest-neighbor indexes already used for retrieval. The core bet is that, at a fixed budget, where the model's attention goes is a better basis for deciding which labels add information than where the query vector sits.

Core Claim

For actively adapting a dense retriever to a new domain under a small query-labeling budget, we expect two changes to recover more retrieval quality per label than the established "uncertainty times embedding distance" selection. The first is to choose a diverse batch in belief space using Hellinger distance, rather than in query-embedding space using Euclidean distance. The second is to score uncertainty as disagreement across dropout passes, rather than as the entropy of retrieval scores. We further expect the advantage to grow on query pools that contain many ambiguous or noisily judged queries. That is the regime where plain entropy is known to chase unlearnable points.

Relation to Prior Work

The method borrows its geometry from simplex retrieval. Efficient Nearest-Neighbor Search in the Probability Simplex shows that Hellinger distance equals Euclidean distance on square-rooted probability vectors. It uses this to index topic distributions with locality-sensitive hashing and k-d trees, reporting roughly 983x to 1425x speedups over exhaustive Jensen–Shannon comparison on NIH grants. That work does passive document-to-document search. We reuse the identity to make diversity-aware batch selection cheap over query beliefs.

Work on topic simplexes, Markov Random Topic Fields and Text Classification based on the Latent Topics of Important Sentences, argues that Euclidean distance misjudges changes near the edges of the simplex, where probabilities approach zero. A belief that moves from 1% to 11% on a corpus region is a large change. These papers motivate the choice of distance but never test it in a selection loop.

The selection rule is a direct modification of Confident Coreset for Active Learning in Medical Image Analysis. That method multiplies the Euclidean distance from a candidate to the labeled set by a predicted uncertainty, so a pick must be both novel and uncertain. We keep the multiplicative "both novel and uncertain" structure and replace both factors.

The uncertainty factor draws on two lines of work. Dropout-based Active Learning for Regression uses Monte Carlo dropout, which runs several forward passes with dropout left on, and finds it beats purely geometric sampling. A separate line warns that maximum total uncertainty attracts noise. Peng et al. filter to median aleatoric uncertainty. Active Data Curation Effectively Distills Large-Scale Multimodal Models keeps only samples a teacher finds easy. Dirichlet-based Uncertainty Calibration for Active Domain Adaptation separates "lack of evidence" from "conflicting evidence" with a Dirichlet over class probabilities. These papers constrain our design: we must not reward the ambiguity that retrieval naturally has. DUC solves the analogous problem for image classification through an evidential head. We instead get the separation from dropout disagreement over corpus beliefs. This needs no change to the retriever's training objective.

Active Learning with Distributional Estimates shows that point-estimate uncertainty sampling gets stuck refining one boundary while missing disconnected regions. Its mean rank is 2.56 against 1.38 for its second-order method across 32 UCI datasets. Its method is limited to kernel density classifiers. Our belief-space diversity term is a cheap substitute aimed at the same failure in a neural retriever.

Two papers on interactive retrieval stay at small scale. Active Information Retrieval uses Dirichlet beliefs and KL projections, but over synthetic cluster trees for one user's search. Information-Theoretic Active Learning for Content-Based Image Retrieval balances uncertainty and diversity through Gaussian-process mutual information, which does not scale to large text pools. Neither selects training queries for a neural text retriever.

Finally, Distribution-Driven Dense Retrieval (DDR) models each document as a Gaussian so that one document can answer many queries. Its variance describes how broad a document's coverage is, not how ignorant the model is. That makes it a natural ablation rather than our uncertainty signal.

Research Program

We first fix a coarse partition of the target corpus into $K$ regions by k-means on document embeddings, with $K$ between 100 and 1000. For a query $q$, the retriever returns its top $n$ documents with scores $s(q,d)$. We define the belief as the softmax mass the retriever places in each region:

$$b_k(q) = \sum_{d \in \text{top}_n(q),\, c(d)=k} \frac{\exp(s(q,d)/\tau)}{\sum_{d' \in \text{top}_n(q)} \exp(s(q,d')/\tau)},$$

where $c(d)$ is the region of document $d$ and $\tau$ is a temperature. Regions give all queries a shared, low-dimensional support. A raw document-level distribution would put different queries on mostly disjoint document sets, so their beliefs would be hard to compare.

We run $T$ dropout passes to get beliefs $b^{(1)},\dots,b^{(T)}$. We take reducible uncertainty to be the gap between the entropy $H$ of the averaged belief $\bar b$ and the average entropy of the individual beliefs:

$$U(q) = H(\bar b(q)) - \frac{1}{T}\sum_{t=1}^{T} H(b^{(t)}(q)).$$

This gap is near zero for a query the model consistently finds broad, which is aleatoric uncertainty. It is large when the passes disagree about where the answer is, which is epistemic uncertainty.

Batch selection is greedy. Each step adds the query that maximizes

$$\min_{s \in L \cup B} \big\| \sqrt{\bar b(q)} - \sqrt{\bar b(s)} \big\|_2 \cdot U(q)^{\alpha},$$

where $L$ is the labeled set, $B$ is the batch so far, the square root is taken elementwise, and $\alpha$ trades novelty against uncertainty as in Confident Coreset. Because the first factor is $\sqrt{2}$ times Hellinger distance, the minimum over $L \cup B$ is a Euclidean nearest-neighbor query in a FAISS index of square-rooted beliefs. Pools of hundreds of thousands of queries therefore stay cheap. Selected queries receive relevance labels, and the retriever is fine-tuned with hard negatives mined from its own rankings. We then repeat.

We will build this loop for two retrievers: a single-vector bi-encoder and DDR. This tests whether the benefit depends on the retriever having distributional documents. We will also study which parameters matter: the region count $K$, the cutoff $n$, the temperature, $T$, and $\alpha$.

Baselines

The naive reference point is random selection of target queries at each budget, using the identical fine-tuning recipe. It shows whether any selection rule is worth its overhead.

The pure uncertainty baseline ranks queries by the entropy of their top-$n$ score softmax, the standard point-estimate approach. A dropout-variance variant in the style of Dropout-based Active Learning for Regression ranks queries by the variance of the top-1 score across passes.

The strongest existing competitor is Confident Coreset, ported to retrieval. It uses Euclidean distance between query embeddings times score entropy raised to $\alpha$. It is the strongest competitor because it already combines uncertainty with diversity in one multiplicative score, which is exactly the structure we keep. Any gain we show must therefore come from the choice of space and the choice of uncertainty, not from combining the two. Its published numbers are on HAM10000, where it reaches 84.3% accuracy with 35% of the data, and on ISIC2017. These are not comparable to retrieval. No reported result found on Lacuna for active query selection on BEIR or MS MARCO, so we will establish this baseline ourselves.

For the source model, DDR's reported MS MARCO Dev MRR@10 of 0.402 against 0.393 for the single-vector MRL (Kang et al.) tells us what our MS MARCO checkpoints should reproduce before adaptation begins.

Two ablations isolate the key ingredients, and each changes one factor of the full method. The first keeps our dropout-disagreement uncertainty but measures diversity with Euclidean distance between query embeddings. The second keeps Hellinger belief diversity but uses total score entropy as the uncertainty. Further ablations swap Hellinger for Euclidean distance on the raw beliefs, without the square root, which isolates the simplex-geometry claim directly. For DDR, we also replace our uncertainty with the mean DDR variance of a query's top documents.

For fairness, every method receives the same query pool, the same per-round budget, and the same fine-tuning schedule and seeds. Every method also gets the same tuning budget of twenty configurations over its own hyperparameters, tuned on a separate development domain. Beating the strongest baseline means a higher area under the nDCG@10-versus-budget curve on held-out target queries. The gain must be consistent across at least five seeds and survive a paired significance test. We report seed variance explicitly, because DDR is known to be seed-sensitive on small test sets.

Why not just…?

Why not just use Confident Coreset on query embeddings? This is the real threat. If query vectors that are close in space also tend to have similar beliefs, the two diversity terms select nearly the same batches and our method adds complexity for nothing. The Euclidean-diversity ablation settles it directly. We will also measure how strongly embedding distance and belief Hellinger distance agree across pool pairs. If the agreement is high and the curves match, we will report that belief space is unnecessary.

Why not just rank by retrieval-score entropy? Score entropy is highest for queries with many truly relevant documents, which is the many-to-one structure that DDR highlights. It is also highest for queries with garbled relevance judgments. Peng et al. report that chasing maximum uncertainty pulls in such noise. Our noise-injection experiment below tests whether entropy over-selects corrupted queries, and whether dropout disagreement avoids them.

Why not just use DDR's document variance as the uncertainty? That variance is trained to express how broad a document's coverage is. Wide coverage is a property of the data, not missing knowledge, so we expect it to behave like the aleatoric term. Running this variant as an ablation lets the data decide. If it matches our method, it would be a cheaper signal and we would say so.

Proposed Experiments

The main experiment adapts MS MARCO-trained retrievers to NQ and FiQA, two BEIR domains where DDR was evaluated zero-shot. We split each domain's training queries into an unlabeled pool, and the oracle reveals the existing relevance judgments when a query is selected. Budgets run over 50, 100, 200, and 500 queries, and we evaluate nDCG@10 and Recall@100 on the standard test queries. Touché-2020 is too small for a pool, so it serves only as a zero-shot check that adaptation does not overfit.

The first decisive check is the Euclidean-versus-Hellinger ablation at matched uncertainty. The second plants a controlled fraction of noisy queries in the pool. Some have shuffled judgments, making them unlearnable. Others are truly ambiguous, with many relevant documents. We record the fraction of each kind that every method selects, and its downstream nDCG. Our method should select fewer planted noisy queries than entropy-based selection at equal budget while matching or improving retrieval quality.

The third check targets the stuck-boundary failure from DEAL. We hold out entire corpus regions from the initial labeled seed. We then track how many rounds each method needs before it labels a query whose answer lies in a held-out region.

Finally, we report wall-clock selection cost against pool size. The square-root index should keep selection time a small fraction of fine-tuning time. If it does not, the efficiency argument inherited from Krstovski et al. fails in this setting, and we will report that.