Most search and question-answering systems run in two stages. A cheap first-stage retriever pulls a small pool of candidates, perhaps 50 documents out of millions, and an expensive re-ranker orders that pool. Whatever the first stage drops is lost for good, because the re-ranker never sees it. When queries come from a domain the retriever was not trained on, such as medical questions, tweets, or typo-ridden user text, the two common first-stage retrievers fail in different ways. A dense retriever compares learned embeddings of the query and the document. It is strong on in-domain paraphrase but brittle under domain shift and typos. BM25, which scores documents by weighted word overlap, is cruder but holds up better. Hybrid systems mix the two with one fixed weight, or interleave their lists half-and-half. That treats the dense retriever as equally reliable on every query, which is exactly what domain mismatch breaks.
We propose to decide, per query, how to split a fixed candidate budget between BM25 and a dense retriever. A small predictor makes this decision from cheap signals about whether this query looks like something the dense model understands. The predictor is trained on a new domain without any relevance labels: offline, the downstream re-ranker scores a large candidate pool for unlabeled target-domain queries, and the predictor learns which split would have captured the documents the re-ranker ends up preferring. The intended result is that a 50-candidate pool chosen this way matches the recall of a naive 100-candidate union. That would halve re-ranker cost under domain shift without retraining either retriever.
Core Claim
Context-aware interpolation has a long record in language modeling and translation. Bulyko et al. and Woodland et al. vary mixture weights by word class and word history. Saunders et al. estimate a domain posterior per sentence with an n-gram model and use it to interpolate general and specialized translation models at decoding time. That approach reaches 57.0 BLEU on IT text, where an IT-only model scores 39.6. All of this work applies the weights to output token distributions. Our claim is that the same idea pays off more in candidate generation than in final scoring, for a structural reason: recall of the final ranking can never exceed recall of the first stage. A bad mixing weight there removes documents permanently, whereas in score fusion it only reorders them.
Two design commitments separate this from simply learning a query-level fusion weight. First, the predictor's target is coverage of the re-ranker's preferred documents at a fixed budget $K$, not a ranking metric. Second, that target can be computed on unlabeled queries from the deployment domain. This matters because the domain shift that breaks the dense retriever would also break a weight predictor trained only on MS MARCO labels.
Concretely, let $B_k(q)$ and $D_k(q)$ be the top-$k$ lists from BM25 and the dense retriever for query $q$. Let $T_m(q)$ be the re-ranker's top $m$ documents from a large offline pool. Let $w \in [0,1]$ be the fraction of the budget $K$ given to BM25. Then the offline target is
$$w^*(q) = \arg\max_{w} \; \frac{|T_m(q) \cap (B_{\lceil wK \rceil}(q) \cup D_{K - \lceil wK \rceil}(q))|}{m},$$
and a lightweight predictor $f(\text{features}(q)) \approx w^*(q)$ is fitted to these targets. A score-fusion variant, $\alpha \cdot \text{BM25} + (1-\alpha)\cdot \text{Dense}$ with $\alpha$ predicted the same way, is studied alongside the allocation form.
Relation to Prior Work
Three papers motivate the gap. BEIR benchmarks MS MARCO-trained retrievers zero-shot across 18 datasets and shows that dense models such as DPR, ANCE, and TAS-B often lose to BM25 out of domain. Reliability therefore varies by domain and, we expect, by query within a domain. Zhuang and Zuccon show the same collapse under typos: dense retrieval falls 52.3% to 0.141 MRR@10, and the cross-encoder downstream loses 34% because the target passages never reach it. The query-by-example interpolation study of contextualized term ranking with BM25 is the most direct evidence. A static mix of BM25 and TILDE-SciBERT raises SciDocs MAP from 85.03% to 86.52%, but a per-query oracle weight does significantly better than any fixed weight. That paper stops at the oracle and provides no way to predict the weight. We build that predictor, and we aim it at the candidate pool rather than the final score.
Three papers supply method or contrast. CoRT trains a dense model on BM25's misses and interleaves the two lists statically. It lifts MS MARCO Recall@1000 from 85.7% to 94.9% and lets a BERT re-ranker saturate at 64 candidates instead of 256. This is the clearest evidence that the composition of the candidate pool controls re-ranker cost, but CoRT's interleaving is fixed and it was never tested out of domain. QuARI uses a hypernetwork to generate a per-query low-rank transform of a dense embedding space. It shows that query-conditioned candidate generation can be cheap (SigLIP mAP 11.1 to 25.6 on ILIAS 100M), but it stays inside one dense space and never mixes in lexical evidence. SymRAG routes queries discretely among symbolic, neural, and hybrid pipelines. Saunders et al. and Kashioka et al. argue that soft interpolation beats hard selection when inputs mix registers. We test that tension directly with a hard-routing ablation.
The predictor's features draw on the domain-mismatch literature. Ren et al. link high model perplexity on unfamiliar text to poor learning, which suggests the dense encoder's own familiarity with a query is a usable reliability signal. Dai et al. note that register shifts as well as topic shifts cause failures, so the features should capture style and surface noise, not only topic.
Research Program
The predictor uses a deliberately small feature set that costs almost nothing at query time. The first features measure the dense encoder's familiarity with the query: masked-token perplexity under the encoder's backbone, and distance from the query embedding to the cloud of MS MARCO training queries. Two features capture surface noise: the rate of subword fragmentation, which is the typo mechanism identified by Zhuang and Zuccon, and the fraction of out-of-vocabulary tokens. Lexical specificity is measured by the mean and maximum inverse document frequency of query terms in the target corpus. Finally, two features measure retriever agreement: overlap between the BM25 and dense top-20 lists, and the score margins of each list. A gradient-boosted regressor or a two-layer network maps these features to $w$. We avoid a hypernetwork in the spirit of Gui et al. on purpose, because the latency analyses in that work and in Saunders et al. show that dynamic parameter generation has real cost, and the decision here is a single scalar.
To train the predictor for a target domain without labels, we take unlabeled queries from that domain and build a 1000-document pool from both retrievers. A cross-encoder re-ranks the pool and its top-$m$ list ($m=10$) defines $T_m(q)$. We then solve for $w^*(q)$ by grid search and fit $f$. Serving requires two top-$k$ lookups with $k \le K$ and one feature pass, the same index footprint as any hybrid system. This avoids the memory cost CoRT's dual representations would add if extended, and the iterative graph exploration of Quam.
The analysis has three goals. First, we measure how much of the per-query oracle gap the predictor recovers. Second, we identify which features carry the signal under topic shift versus surface-noise shift. Third, we check whether the re-ranker teacher's own domain mismatch caps the gains. All evaluation uses gold labels, never the teacher.
Baselines
The naive reference point is the dense retriever alone at budget $K$, followed by the same cross-encoder. BM25 alone at $K$ is the second naive point and, per BEIR, often the stronger one out of domain.
The strongest existing method within this file is static hybrid generation at the same budget $K$, in two forms. One is CoRT-style half-and-half interleaving following Wrzalik and Krechel. The other is linear score fusion with a single $\alpha$ tuned on held-out data, following the QBE interpolation study. The reported numbers for these baselines are 94.9% Recall@1000 on MS MARCO for CoRT+BM25 versus 85.7% for BM25, and 86.52% SciDocs MAP for static BM25+TILDE-SciBERT fusion. Both are in-domain figures. For static hybrids on BEIR's out-of-domain sets, no reported result was found on Lacuna, so we will run these baselines ourselves. For fairness, the static $\alpha$ gets a tuning budget equal to the predictor's: it may be tuned on the same unlabeled teacher targets, or on gold labels from the same queries if those turn out to help. Reciprocal rank fusion is included as a parameter-free static variant.
The brute-force competitor is the union of both retrievers at budget $2K$. This shows what recall costs if one simply pays double re-ranker compute. The proposal beats it if adaptive allocation at $K$ reaches at least 95% of the union's Recall@$K$ and its nDCG@10 after re-ranking, while re-ranking half as many documents.
The domain-adaptation competitor is the dense retriever fine-tuned on the same unlabeled target queries, using the same cross-encoder pseudo-labels. This baseline uses the teacher signal to fix the retriever instead of the mix. It receives the same teacher compute, plus the training compute it needs, which we report. On MS MARCO with typos, we also include the typo-aware dense retriever of Zhuang and Zuccon, which reports 0.219 MRR@10 on typo queries versus 0.141 for the standard model.
Four ablations isolate the key ingredients. A predictor trained on MS MARCO gold labels only, with no target-domain teacher, tests whether label-free target supervision matters. A hard-routing version that sends all $K$ slots to one retriever by thresholding $w$ tests soft versus discrete allocation, the SymRAG contrast. An agreement-features-only predictor tests whether the familiarity and noise features add anything beyond simple list overlap. The per-query oracle $w^*$ under gold labels gives the ceiling.
Why not just…?
Why not just tune one static weight per domain? This is the most serious threat. If most of the oracle gap lies between domains rather than within them, a per-domain $\alpha$ tuned on teacher targets will capture nearly everything. We will measure this directly by splitting the oracle gap into between-domain and within-domain variance on each BEIR dataset. If the within-domain share is small, the honest conclusion is that per-domain static tuning on teacher targets is enough. The project's contribution would then shrink to label-free tuning of that weight, and we would report it as such. The QBE study's per-query oracle gain within a single scientific domain suggests the within-domain share is not small, but it reports no oracle figure, so this remains unproven.
Why not just retrieve more candidates from both retrievers? Union at $2K$ will almost always match or beat adaptive allocation on recall. The question is cost. CoRT's own result, re-ranker saturation at 64 versus 256 candidates, shows that pool size is the main driver of second-stage latency. The matched-budget comparison above settles whether we save half the re-ranking at negligible recall loss. If we do not, the method has no reason to exist.
Why not just adapt the dense retriever to the new domain? Fine-tuning on pseudo-labels may close much of the dense model's out-of-domain gap, and it is a real competitor. Its drawbacks are cost, forgetting, and rigidity. It requires retraining and re-indexing per domain. fBERT took days of continued pre-training for one domain, and Saunders et al. found that domain-only fine-tuning overfits. Still, the head-to-head under equal teacher signal decides the matter. We also test the combination: allocation on top of an adapted retriever. If allocation adds nothing once the retriever is adapted, it is only a cheap substitute for adaptation, and we will say so.
Why not just use a stronger re-ranker? No re-ranker can recover documents that were never retrieved. Zheng et al. report that GPT-4 re-ranking at TREC 2023 does not beat baselines when the candidate pool is poor. In our setup, re-ranking the dense-only pool with a larger cross-encoder serves as a check that the gap persists.
Proposed Experiments
The main testbed is BEIR in the zero-shot setting, with all retrievers trained on MS MARCO, focusing on the datasets where BEIR reports dense failures: TREC-COVID, BioASQ, NFCorpus, Touché-2020, and Signal-1M (tweets). Each dataset's queries are split in half. One half's text is used only to build teacher targets, with its labels discarded, and evaluation runs on the other half with gold labels. We also report a variant where teacher queries are synthesized from target documents, which avoids transductive use of test-distribution queries. MS MARCO dev with Zhuang and Zuccon's five typo operators tests surface-noise shift, and TREC DL19 checks that in-domain quality does not regress. Metrics are Recall@$K$ for $K \in \{20, 50, 100\}$, nDCG@10 after cross-encoder re-ranking, the fraction of the gold-oracle gap recovered, and wall-clock latency per query.
Two checks are decisive. First, on the BEIR subsets, adaptive allocation at $K=50$ must beat the best static fusion at $K=50$ on Recall@50 on most datasets and come within 5% of the $K=100$ union after re-ranking. Second, the target-teacher predictor must clearly beat the MS MARCO-label predictor on BEIR while matching it on DL19. That result would show that label-free in-domain supervision, not merely per-query weighting, is what makes the approach work under domain mismatch. If the first check passes but the second fails, the result is a modest query-adaptive fusion paper. If the oracle decomposition shows that weights vary mainly between domains, we will report that the cheaper per-domain fix is enough.
References
- Getting More Mileage from Web Text Sources for Conversational Speech Language Modeling using Class-Dependent Mixtures
- Language model cross adaptation for LVCSR system combination
- Domain Adaptive Inference for Neural Machine Translation
- BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models
- Dealing with Typos for BERT-based Passage Retrieval and Ranking
- On the Interpolation of Contextualized Term-based Ranking with BM25 for Query-by-Example Retrieval
- CoRT: Complementary Rankings from Transformers
- QuARI: Query Adaptive Retrieval Improvement
- SymRAG: Efficient Neuro-Symbolic Retrieval Through Adaptive Query Routing
- Tied-State Mixture Language Model for WFST-based Speech Recognition
- I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses
- Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media
- Transferring from Formal Newswire Domain with Hypernet for Twitter POS Tagging
- Quam: Adaptive Retrieval through Query Affinity Modelling
- fBERT: A Neural Transformer for Identifying Offensive Content
- University of Tsukuba Team at the TREC 2023 Deep Learning Track