Where Should Domain Knowledge Enter a Late-Interaction Retriever?

Most neural search systems squeeze each document into one vector and compare it with one query vector. In technical fields this loses exactly what users search for: a rare drug name or gene variant gets averaged away by the rest of the abstract. Late-interaction retrievers avoid the squeeze. They keep one vector per token and let each query token find its best-matching document token. They still inherit a vocabulary written for web text, though. A general tokenizer splits a term like "pembrolizumab" into four or five meaningless pieces, and nothing stops each piece from matching some unrelated fragment elsewhere in a document. We propose to measure this failure directly and to test, under matched compute, which training stage is the cheapest place to fix it. The candidates are the language-model backbone, a retrieval-specific pre-training stage on domain text, or plain fine-tuning with distillation.

The question matters because the two literatures disagree. In the general domain, ColBERT-Zero found that skipping unsupervised multi-vector pre-training and doing only supervised training plus distillation reaches 55.12 average nDCG@10 on BEIR, 99.4% of the full pipeline's 55.43, at under a tenth of the compute. The domain-adaptation literature says shallow adaptation cannot replace domain pre-training when vocabulary diverges. A Comparative Study of PEFT Methods for Python Code Generation found that domain pre-training predicted downstream quality better than the choice of adaptation method, and MonoCoder beat much larger general models by training only on in-domain code. Biomedical literature search is the natural place to settle which finding transfers to token-level retrieval.

Core Claim

We claim that in jargon-dense domains, the advantage of late interaction over single-vector retrieval depends on a measurable property: whether the token vectors of a multi-piece domain term match their counterpart in a relevant document, or scatter onto unrelated pieces and punctuation. We call the second behavior fragment scattering.

We hypothesize that fragment scattering is set mainly during contrastive multi-vector pre-training, not during masked-language-model pre-training of the backbone. If that holds, a short multi-vector pre-training stage on PubMed should beat both the general ColBERT-Zero recipe and a recipe built on a biomedical masked-language-model backbone, at equal total compute. The new part is a falsifiable mechanism plus a compute-matched factorial test of where the fix belongs. It is not another domain retriever. If the backbone alone removes scattering, the proposal says so and the cheap recipe wins.

Research Program

The work has three parts.

The diagnostic. We first build a scattering measure. For each test query, we mark spans that are domain terms split into two or more subword pieces, using the MeSH headings and author keywords already attached to PubMed abstracts as a term list. Late-interaction scoring follows the MaxSim rule of ColBERT: each query token contributes its maximum dot product over document tokens. For each such query token we record where that maximum lands. A "coherent" match lands on a piece of the same term (or an annotated synonym) in a relevant document. A "scattered" match lands on any other token, including markers and punctuation.

The fragment-scattering rate is the share of a query's MaxSim score contributed by scattered matches on its term tokens. This extends the observation of Rethinking the Role of Token Retrieval in Multi-Vector Retrieval, which found that document-level training lets non-informative tokens receive high similarity. That paper studied general English. We condition specifically on domain-term fragments. Because ColBERT-Zero showed that query marker tokens carry hidden sequence information, we also report the share of term-token score landing on markers.

The factorial study. Next we train a grid of retrievers that crosses three factors.

The first factor is the backbone. It is either general ModernBERT-base, as in ColBERT-Zero, or the same model given continued masked-language-model training on PubMed abstracts. The second is the unsupervised multi-vector contrastive stage, which is either absent, run on the general Nomic mixture, or run on PubMed. The third is the PubMed contrastive pair type. It is either generic title/abstract pairs, or keyword-anchored pairs adapted from the Expanded Title Mapping and Reduced Sentence Mapping objectives of Improving Biomedical Information Retrieval with Neural Retrievers. That paper built those pairs for a $K$-facet model, Poly-DPR. We reuse the pair construction, which is the method contribution we borrow, and train full token-level MaxSim on it instead of a fixed number of facet vectors.

Every arm then receives the same supervised fine-tuning and distillation stage. Arms are matched on total GPU hours. The domain multi-vector stage is capped near the 40 GPU-hour budget that ColBERT-Zero reports for its cheap path, so any arm adding it gives up equal budget elsewhere. All training uses PyLate, with PLAID indexing as in ColBERTv2, so engineering differences cannot explain results.

The causal check. Finally we test whether scattering explains the retrieval differences. We regress per-query effectiveness on scattering rate across arms. We also intervene at inference by masking scattered matches above a threshold and checking whether the gap between arms shrinks.

Baselines

BM25. The naive reference point is BM25 run on the identical PubMed index. On BioASQ it reaches 65.10 MAP and beats standard single-vector DPR at 53.31 (Improving Biomedical Information Retrieval with Neural Retrievers). Any neural arm that does not clear BM25 on BioASQ has failed, regardless of its scattering numbers.

Poly-DPR and its hybrid. The strongest published domain competitor is Poly-DPR. It reports 66.66 MAP on BioASQ, and its hybrid with BM25 reports 36.26 MAP on the 19-million-abstract PubMed corpus (same paper). It is the closest existing method because it already uses domain-specific pre-training to break the single vector, at coarse granularity. We compare under its published BioASQ protocol. We also retrain a $K$-facet version on our PubMed pairs with the same compute, so that the facet-versus-token question is separated from data differences. Beating it means exceeding 66.66 MAP on BioASQ, and exceeding 36.26 MAP on the 19M corpus when we too combine with BM25.

ColBERT-Zero, both paths. The strongest general late-interaction competitor is ColBERT-Zero, run two ways. One is the full pre-trained model, which reports 55.43 BEIR nDCG@10. The other is the cheap supervised-plus-distillation path, which reports 55.12. We evaluate the released recipes on our domain tests after the same domain fine-tuning given to our arms. The cheap path is the specific claim we are testing. Beating it means a statistically reliable gain on BioASQ and the biomedical BEIR subsets at equal total GPU hours.

Domain single-vector arm. To isolate the token-level ingredient, we run a single-vector retriever with the identical PubMed pre-training and fine-tuning. If it matches the multi-vector arms, the gains come from domain data, not from escaping the bottleneck.

Generic-pair ablation. The keyword-anchored versus generic PubMed pair comparison isolates whether term-anchored pairs, rather than domain text as such, reduce scattering.

Why not just…?

Why not just use BM25 or a BM25 hybrid? Exact matching sidesteps fragmentation entirely, and BEIR found BM25 hard to beat zero-shot. BEIR also warns that biomedical relevance labels were often built with keyword search, which favors lexical systems. On the other side, Poly-DPR's gains over BM25 and its hybrid gains on the 19M corpus show that synonym and paraphrase matching matter. We report every neural arm alone and fused with BM25. If fusion with a cheap arm closes the gap, the expensive arms are not worth it.

Why not just start from a biomedical masked-language-model backbone and fine-tune ColBERT on it? This is the real threat. ProtoCol did essentially this for proteins: it fine-tuned late interaction on top of ESM-2, a backbone trained only with masking, and a 35M model beat pooled 650M ESM-2 by nearly 7 points of Capped Recall@1 on SCOPe. The factorial design contains this arm explicitly, as domain backbone with no multi-vector pre-training. If it matches the PubMed multi-vector arm on both retrieval and scattering, we will conclude that the cheap path transfers to technical domains provided the backbone is domain-adapted, and the retrieval-specific domain stage is unnecessary.

Why not just make a single-vector model bigger? ProtoCol's 35M-versus-650M result and the collapse of single-vector models to below 5% Recall@5 on LIMIT, against 78.6% for sparse multi-vector coding (No More K-means), argue against it. Our domain single-vector arm, plus a larger single-vector backbone given the same compute, tests it directly on biomedical text.

Why not just expand documents with an LLM? Improving Scientific Document Retrieval with Academic Concept Index found that unguided LLM expansion fixates on prominent concepts, and its concept-guided alternative requires offline generation for every document. Expansion does not touch the token-matching failure we measure. We treat it as out of scope for the main grid. We do add one docT5query-style expanded-BM25 reference point, because BEIR found that expansion beats BM25 on 11 of 18 datasets.

Proposed Experiments and Decisive Checks

Test sets. The main in-domain tests are BioASQ (MAP) and the 19M PubMed corpus (MAP), plus the biomedical datasets inside BEIR (nDCG@10). CSFCube and DORIS-MAE test transfer to scientific text outside biomedicine; we found no reported absolute result on Lacuna for these. LIMIT tests capacity under dense semantics. Full BEIR averages measure forgetting of general ability, against the 55.43 and 55.12 references.

Prediction. If our hypothesis holds, three things happen together. The general cheap path shows the highest scattering rate on multi-piece biomedical terms. The domain-backbone-only arm reduces scattering only partly. The PubMed multi-vector arm with keyword-anchored pairs shows the lowest scattering and the best BioASQ MAP at equal compute.

Decisive checks. Three outcomes count. First, scattering rate must predict per-query effectiveness across arms. Second, masking scattered matches at inference must close most of the gap between the cheap path and the domain arm. If it does, that also suggests a training-free fix worth reporting. Third, the domain gain must be concentrated on queries containing multi-piece terms, with little gain on queries without them.

Falsification. If gains are uniform across query types, or if the domain-backbone-only arm matches the full domain arm, the mechanism is wrong. The proposal would then reduce to a clean negative result: in biomedicine, the ten-fold cheaper path holds as long as the backbone has seen domain text.