Hybrid search engines pair a keyword index (BM25, which scores documents by exact word overlap) with a dense retriever (a neural network that maps queries and documents to vectors and ranks by vector similarity). The two are supposed to cover each other's failures. Keywords catch exact names, numbers and identifiers. Vectors catch paraphrases, where the query and the relevant document describe the same thing in different words. In practice the neural half spends much of its capacity relearning what the keyword half already does: matching entity names. It still blurs the names it should keep apart, such as "JASPAR 2016" versus "JASPAR 2010". We propose to give each half a fixed job. During training and indexing, every named-entity mention seen by the dense encoder is replaced with a typed placeholder such as [ORG] or [CHEMICAL]. The dense side can then only learn the relational and descriptive context around entities, and entity identity is left entirely to BM25.
The second ingredient decides, per query, how much to trust each half. We constrain dense vectors to the unit sphere and treat the query as the center of a von Mises–Fisher distribution, the spherical analogue of a Gaussian, whose concentration parameter $\kappa$ measures how sharply peaked it is. A small network predicts $\kappa$ from the masked query. Its job is to judge how much discriminating information is left once entities are removed. "Who founded [ORG]" carries almost none, so BM25 should dominate. "Why do pipes crack when water freezes" carries a lot, so the dense score should dominate. In two sentences: mask entities out of the dense encoder so that it and BM25 stop overlapping, and let a learned per-query concentration decide how to weight them.
Core Claim
The redundancy between the two halves of a hybrid retriever comes largely from entities. Removing entities from the dense side by semantic type is a more targeted way to force the two halves apart than masking overlapping tokens at random. Once entities are gone, the dense model's confidence becomes a meaningful per-query fusion weight.
The approach makes three testable predictions. First, entity-type masking should beat random overlap masking at a matched mask rate on standard benchmarks. Second, its gains should concentrate on queries whose gold documents share entities with the query but not context words. Third, a learned $\kappa$ should beat a fixed interpolation weight. Whether it also beats a feature-based fusion predictor is the open question we treat as a real threat below.
Relation to Prior Work
The closest work is On Complementarity Objectives for Hybrid Retrieval. It names the "Intersection Trap", in which dense and sparse branches retrieve the same easy documents. It fixes this by randomly replacing query–document overlapping tokens with [MASK] and adding an orthogonality penalty between dense and sparse vectors. That paper motivates ours, and it is the baseline to beat. Its masking ignores what kind of token is being masked: a stop word and a drug name are treated alike. Its masking also only touches tokens that happen to overlap, so non-overlapping entity names remain available as memorization shortcuts. Its fusion is a single scalar shared by all queries. We change all three.
The trap it diagnoses originated with Complementing Lexical Retrieval with Semantic Residual Embedding (CLEAR). CLEAR trains the dense model on BM25's errors and reports semantic false positives such as "theraderm" for "thermograms". Sparse, Dense, and Attentional Representations for Text Retrieval explains why fixed-length dense vectors lose discriminative margin on long texts. Hybrid First-stage Retrieval Models for Biomedical Literature shows how badly pure dense retrieval does on entity-heavy biomedical search, with MAP 0.1173 against BM25's 0.2718 on BioASQ batch 2. Together these three papers establish the gap: dense retrievers are weak exactly where entity identity matters, yet they are trained to spend capacity on it anyway. None of them removes entities from the dense side.
The masking technique comes from entity blinding and semantic-type masking in relation extraction and reading comprehension. Those methods replace mentions with type labels to block co-occurrence memorization while keeping predicate frames. They have not been used as a division-of-labor device in retrieval. A constraint comes from Context Tuning for In-Context Optimization, which found that aggressive masking destabilizes optimization when training signal is scarce. For the small BioASQ fine-tuning regime, we therefore anneal the masking rate rather than masking everything from step one.
The spherical side draws on several sources. On the Sentence Embeddings from Pre-trained Language Models shows that unnormalized BERT embeddings collapse into a narrow cone whose similarity tracks edit distance. It fixes this with a post-hoc flow but never builds a retriever. Spherical Text Embedding trains on the sphere but fixes $\kappa=1$. Discriminative Topic Mining via Category-Name Guided Text Embedding shows that learned $\kappa$ separates broad from specific concepts, which is the reading we rely on: low concentration means "this query alone does not pin down a document." A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations is the only work in the file that pairs a von Mises–Fisher semantic code with a separate non-spherical code, but it is a sentence autoencoder and never retrieves. We borrow its factorization idea, applied to entity identity versus context rather than syntax versus semantics.
We are not claiming that cosine similarity is new to retrieval. Normalized dense vectors are routine. The claim is narrower: a per-query concentration, trained jointly with the hybrid ranking loss, can serve as the fusion weight once entities have been stripped.
Research Program
The model has three parts. The first is a standard BM25 index over unmodified text. The second is a dense dual encoder that sees text in which every mention detected by an off-the-shelf named-entity tagger is replaced by its type. Within a single passage, repeated mentions keep a shared index ([PERSON_1]), so coreference survives. Indices are not aligned across query and document, because that would require knowing the query at indexing time. The third is a small head on the masked query embedding that outputs $\kappa(q) > 0$. For query $q$ and document $d$ we score
$$ s(q,d) = \kappa(q)\, \mu_q^{\top} x_d + \mathrm{bm25}(q,d), $$
where $\mu_q$ and $x_d$ are the unit-norm masked embeddings of query and document and $\mathrm{bm25}$ is the raw keyword score. For a fixed query, the first term is exactly the von Mises–Fisher log-density up to a constant. We train with a softmax contrastive loss over $s$, using gold passages as positives and BM25 top-ranked non-gold passages as negatives, following CLEAR's error-based sampling.
The gradient on $\kappa(q)$ rises when the dense term separates the positive and BM25 does not, and falls otherwise. The model therefore learns its fusion weight from ranking signal rather than from a held-out grid search. The BM25 weight is fixed at one, so $\kappa$ carries all the relative scaling, which addresses the calibration problem that interpolation weights do not transfer across corpora.
Indexing is a single maximum-inner-product search over the dense vectors plus a standard inverted-index lookup. One merge is possible because $\kappa(q)$ is query-side only, so it can multiply the query vector before search. The only extra cost is running the tagger once over the corpus and once per query.
The analysis has two parts. We measure redundancy between the branches with the overlap and complementarity statistics of On Complementarity Objectives for Hybrid Retrieval. We also partition queries into four cells according to whether the gold document shares the query's entity strings and whether it shares the query's non-entity content words, using WikiQA's observation that 20.3% of correct answers share no content words with the question as the motivating statistic for the last of these cells.
Baselines
The naive reference points are BM25 alone and an unmasked, normalized dense dual encoder on the same backbone used alone. On MS MARCO Dev, CLEAR reports BM25 at MRR@10 of 0.191 and a pure BERT-Siamese model at 0.308. Our reproductions should land near those numbers before any other comparison is trusted.
Above these sits the "do nothing new" hybrid: the same unmasked dense encoder plus BM25 with one interpolation weight tuned on the dev set. Every proposed component has to justify itself against this system.
The residual baseline is CLEAR, which reports MRR@10 of 0.338 on MS MARCO Dev. ME-HYBRID reports 0.343 and serves as a multi-vector reference point.
The strongest competitor is the complementarity-trained hybrid of Lee et al.. It reports MRR@10 of 38.63 on MS MARCO Passage, above CLEAR + coCondenser at 37.99. It also reports Recall@100 of 90.13% on Natural Questions and MAP of 36.74 on TREC Robust04. It is the strongest because it attacks the same redundancy problem with masking and orthogonality, which is the design we claim to improve on. We will reimplement it on the same backbone as ours, which we take to be coCondenser if the reported setup allows. It will get the same training data, the same number of training steps, the same BM25 negatives and the same hyperparameter search budget, set as an equal number of dev-set trials per method. We will report both their published numbers and our reproduction.
Beating it means two things. On MS MARCO, MRR@10 must exceed our reproduction of it by more than the seed-to-seed spread across three seeds. On NQ and Robust04, results must be no worse. A gain on MS MARCO that disappears on the entity-heavy sets would count as failure.
For BioASQ 8, the reference is Ma et al.'s HybridRerank average MAP of 0.4056. That system includes reranking and 103M synthetic queries, so we will compare against it only after adding the same reranker and will flag the data gap.
Two ablations isolate the key ingredients. The first swaps entity-type masking for Lee-style random overlap masking at the same expected fraction of masked tokens, and adds a variant that masks entities only where they overlap. Together these separate "what kind of token is masked" from "how many tokens are masked." The second freezes $\kappa$ to a single learned scalar, isolating per-query concentration from mere normalization.
Why not just…?
Why not just use random overlap masking as Lee et al. do? This is the most direct challenge. If entity typing adds nothing beyond masking rate, the matched-rate ablation will show no difference, and the masking claim fails. Our reason to expect a difference is that random overlap masking never touches non-overlapping entity names, so the dense model can still memorize them. It also spends part of its masking budget on function words. The prediction is specific: the gap should appear in the "entities match, context does not" cell and on BioASQ. A uniform gain everywhere would point to a regularization effect rather than division of labor.
Why not just L2-normalize the dense vectors and tune one interpolation weight? That is the fixed-$\kappa$ ablation together with the tuned hybrid baseline, and it may well capture most of the spherical benefit, since normalization by itself is not new. The test for the learned-concentration component is zero-shot transfer: the same trained model on BEIR, where BEIR found BM25 often beats dense retrievers. A fixed weight tuned on MS MARCO should mis-weight those corpora, whereas a per-query $\kappa$ should shift toward BM25 on entity-dominated queries without any retuning. If the fixed weight transfers as well, we drop the concentration claim.
Why not just predict a per-query fusion weight from cheap features such as query length, entity count, and maximum BM25 score? This is a real threat, because $\kappa(q)$ is functionally a learned fusion weight. We will train a small regressor on those features to predict, on the training set, the interpolation weight that maximizes the gold document's rank, and then use it at test time. If it matches $\kappa$, the honest conclusion is that masking matters and the spherical framing is merely a convenient parameterization. The concentration reading earns its keep only if $\kappa$ beats the regressor out of domain, or if it correlates with dense-branch reliability beyond what entity count explains.
Why not just rewrite queries with a language model or expand documents with docT5query? BEIR reports docT5query outperforming BM25 on 11 of 18 datasets, so expansion is strong. However, expansion and rewriting add generation cost. CQE reports T5-based reformulation pipelines exceeding 500ms against about 11ms for single-stage embedding retrieval. Rewriting can also drift from the user's constraints. Expansion is also complementary rather than competing, so we will report docT5query alone and our method applied on a docT5query-expanded index.
Proposed Experiments
Training uses MS MARCO Passage. In-domain evaluation uses MS MARCO Dev (MRR@10), Natural Questions (Recall@100) and TREC Robust04 (MAP). Entity-heavy evaluation uses BioASQ 8 (MAP) after domain fine-tuning with the annealed masking schedule. Zero-shot evaluation uses BEIR (nDCG@10). For lexically asymmetric evidence, MISCIPLUS is a stress test: refuting passages share little phrasing with the claim, and the benchmark paper reports up to 0.664 MRR for the overlapping-evidence subtask. No reported result was found on Lacuna for hybrid retrievers on the non-overlapping refutation setting.
There are four decisive checks.
The first is the four-cell slice analysis. The method should gain most where entities match but context words do not. It should help modestly where neither matches.
The second is designed to show where the method hurts: queries where the gold document uses an alias of the query's entity, such as an acronym versus a full name. Here entity blinding removes the dense side's ability to bridge the names. We will build this slice with string-mismatched but tagger-coreferent pairs. A loss here is expected. If the loss is large, it motivates keeping a small unmasked channel.
The third tests whether $\kappa$ means what we claim. It should rise with the fraction of query tokens that survive masking and fall on entity-dominated queries. It should also predict, on held-out queries, which branch ranks the gold document higher.
The fourth tests whether the two branches have actually separated. Masking should lower the overlap between the dense and BM25 top-100 lists below that of Lee et al.'s model at equal or better recall. That result would show that entity removal, not the orthogonality penalty, is what pulls the two retrievers apart.
References
- On Complementarity Objectives for Hybrid Retrieval
- Complementing Lexical Retrieval with Semantic Residual Embedding
- Sparse, Dense, and Attentional Representations for Text Retrieval
- Hybrid First-stage Retrieval Models for Biomedical Literature
- Context Tuning for In-Context Optimization
- On the Sentence Embeddings from Pre-trained Language Models
- Spherical Text Embedding
- Discriminative Topic Mining via Category-Name Guided Text Embedding
- A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations
- WikiQA: A Challenge Dataset for Open-Domain Question Answering
- BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models
- Contextualized Query Embeddings for Conversational Search
- Grounding Fallacies Misrepresenting Scientific Publications in Evidence