Spending the Index Budget on Phrases: Compiled Term Indexes as a Substitute for Dense Bits in Domain Retrieval

Search systems for technical fields such as biomedicine increasingly pair two kinds of index. A keyword index matches exact words. A dense index stores, for every passage, a learned vector of a few hundred numbers that captures meaning, so a query about "heart attack" can find "myocardial infarction." The dense part is expensive: it can need tens of gigabytes of memory where the keyword index needs about one. The standard fix is to compress the vectors hard. Compression works well on general web questions, but published results show it breaks down more sharply once the queries come from a different domain than the one the encoder was trained on.

This note proposes that, in specialized domains, most of what compression destroys is the ability to tell apart long technical terms that differ in one word, for example "small cell lung cancer" versus "non-small cell lung cancer." That fine distinction is cheap to recover a different way. At indexing time we run a domain term grammar over the corpus once and compile every multi-word technical term into a phrase-level lookup index. Doing this offline means no parsing happens when a query arrives. With that index carrying exact-term precision, the dense vectors can be compressed far more aggressively. The question we test is how a fixed memory budget should be split between dense bits and precompiled phrase structure, and whether moving bytes from the first to the second beats every simpler way of spending them.

Core Claim

Our claim is that for domain-pretrained retrievers there is a memory budget at which a three-part index beats the standard compressed hybrid at equal total bytes on biomedical first-stage retrieval. The three parts are a very heavily compressed dense vector (about 96x), an ordinary word-level keyword index, and an offline-compiled index of grammar-identified multi-word terms. The standard hybrid it must beat is a moderately compressed dense vector (about 16x) plus a word-level keyword index.

We also make a diagnostic claim that the gain should be explained by. Queries that compression newly fails on should disproportionately contain multi-word domain terms, and the phrase index should recover those specific queries. If the diagnostic fails, the cause of compression loss lies elsewhere and the design is wrong.

The score we use is

$$ s(q,d) = \langle \tilde{e}_q, \tilde{e}_d \rangle + \lambda \, \mathrm{BM25}(q,d) + \mu \, \mathrm{BM25}_{\text{phr}}(q,d), $$

where $\tilde{e}$ is the compressed dense embedding, $\mathrm{BM25}$ is word-level keyword scoring, and $\mathrm{BM25}_{\text{phr}}$ is the same keyword formula applied to compiled term identifiers instead of words. The weights $\lambda, \mu$ are tuned on held-out queries. All systems are compared at equal bytes per passage, counting dense codes, posting lists and the term automaton.

How This Builds on Prior Work

Two papers motivate the claim. Simple and Effective Unsupervised Redundancy Elimination to Compress Dense Vectors for Passage Retrieval compresses general-domain dual-encoder vectors with principal component analysis followed by product quantization (PQ, which replaces each vector with short codes pointing into learned codebooks). It shows that 16x compression plus BM25 slightly beats the uncompressed model on Natural Questions. It also reports a steeper drop on out-of-domain SQuAD. That paper only fuses word-level BM25 and never tests domain encoders, so the shape of the domain cliff is unknown.

BEIR shows that dense retrievers often lose to BM25 zero-shot on biomedical sets. This indicates that exact lexical evidence carries unusual weight in these domains.

Restoring Pruned Large Language Models via Lost Component Compensation finds that low-magnitude directions removed by pruning carry discriminative signal. That suggests variance-based compression could be discarding exactly the minor directions that separate near-identical technical terms. The paper concerns model weights rather than embeddings, so we treat it only as a hypothesis source.

Three works supply the method. Hybrid First-stage Retrieval Models for Biomedical Literature gives the domain encoder recipe: a dual encoder trained on 103M synthetic PubMed question–passage pairs. It also shows that BM25 can be written as a sparse inner product concatenated with the dense vector. We reuse both, but that paper keeps uncompressed 768-dimensional vectors and only word-level sparse terms. KnowItNow is the template for moving linguistic analysis (noun-phrase recognition) into the indexing phase, which gave a more than 130x query-time speedup. It was pattern-based and had no dense component. For the term grammar itself, A Unification-based Approach to Morpho-syntactic Parsing of Agglutinative and Other (Highly) Inflectional Languages and Polynomial Time and Space Shift-Reduce Parsing of Arbitrary Context-free Grammars show how a lexicon and grammar can be compiled into a static automaton or tables before any input is seen. That gives us cheap, deterministic phrase recognition at both index and query time.

MonoCoder is a close analogue in a different direction. It also pairs domain pretraining with an offline parse (stripping local identifiers from syntax trees), but it uses the parse to shape what the model learns. We instead use the parse to take load off the model's stored vectors.

One constraint comes from multi-vector work. Sparse, Dense, and Attentional Representations for Text Retrieval argues that a single vector blurs as passages grow and fixes this with several vectors per passage. Efficient Multi-Vector Dense Retrieval Using Bit Vectors makes that approach cheap. Our proposal is a competing way to restore fine discrimination, so multi-vector retrieval is a baseline we must face rather than ignore.

Research Program

Phrase index. We build a biomedical term grammar from noun-phrase patterns over a domain lexicon, covering modifier chains, negated modifiers such as "non-", and hyphen variants. We compile it into a finite-state recognizer and run it once over the PubMed collection. Each recognized maximal term and its sub-terms are assigned identifiers and given posting lists. Queries pass through the same compiled recognizer, which costs microseconds.

Domain encoders. We train dense encoders following the biomedical hybrid recipe and keep a general-domain DPR-style encoder for contrast.

Compression sweep. For each encoder we sweep principal-component dimension (256, 128, 64) and PQ bit-width. This answers the open question of whether domain embeddings show the same roughly 200-dimension plateau as general ones, or resist compression.

Attribution analysis. For every query, we label the relevant passages lost when moving from uncompressed to compressed dense retrieval. We then test whether those losses are enriched for queries containing compiled multi-word terms, compared with the base rate in the query set.

Baselines

Naive reference: BM25. Plain word-level BM25 over the same PubMed collection is both the "do nothing" point and a strong one. On BioASQ 8 Batch 2, Hybrid First-stage Retrieval Models for Biomedical Literature reports MAP 0.2718 for BM25 against 0.1173 for its dual encoder alone. Any proposed system must clear BM25 clearly.

Strongest compressed competitor: the 16x hybrid. This is the configuration from Simple and Effective Unsupervised Redundancy Elimination to Compress Dense Vectors for Passage Retrieval: PCA-256 plus 2-bit PQ plus word-level BM25. On Natural Questions it reached 16x compression while improving top-k accuracy by 0.2% over uncompressed DPR-768. No reported result was found on Lacuna for this configuration on BioASQ or BEIR biomedical sets, so we run it ourselves.

For a fair comparison, the 16x hybrid uses the same domain encoder, the same corpus and the same fusion-weight tuning budget (an identical grid on the same held-out queries) as the proposal. Beating it means higher Recall@1000 and MAP at equal or smaller total bytes per passage, across five seeds of query splits, with a paired significance test.

Strongest uncompressed reference. The uncompressed domain hybrid serves as the accuracy ceiling. The best reported number in the file is HybridRerank's average BioASQ 8 MAP of 0.4056, but that includes a reranking stage. We therefore report both first-stage metrics and MAP after an identical reranker applied to every system's candidate list.

Key ablation. We swap the phrase index for word-bigram postings of matched size. This isolates whether grammar-compiled terms matter or any word-n-gram index would do.

Multi-vector baseline. We run the bit-vector multi-vector retriever from Efficient Multi-Vector Dense Retrieval Using Bit Vectors on the same domain encoder at matched bytes.

Why not just…?

Why not just keep more dense dimensions instead of adding a phrase index? This is the central threat. The bytes spent on phrase postings could instead raise the dense vectors from 128 to 256 dimensions or from 2-bit to 4-bit codes. We settle it by tracing the full accuracy–bytes frontier for both choices on the same axes. If the dense-only frontier dominates, the proposal is refuted. The attribution analysis then tells us why: compression losses were not concentrated on multi-word terms.

Why not just use word-level BM25, or bigrams, as the sparse half? Word-level BM25 already recovers some exact-match signal. The 16x result in Simple and Effective Unsupervised Redundancy Elimination to Compress Dense Vectors for Passage Retrieval shows it can fully offset moderate compression. Bag-of-words scoring, however, cannot tell "non-small cell" from "small cell" when both share every word. The bigram ablation tests whether grammar is needed at all. If bigrams match the compiled terms, we drop the grammar and report the simpler system.

Why not just use document expansion or multiple vectors per passage? BEIR reports that docT5query, which appends generated likely queries to each document, beats BM25 on 11 of 18 datasets. Multi-vector methods restore fine discrimination directly. Both are genuine competitors. Expansion inflates the sparse index and needs heavy generation at index time. Multi-vector storage grows 3x to 8x before compression, according to Sparse, Dense, and Attentional Representations for Text Retrieval. We include both at matched bytes and matched indexing compute, and we report honestly if either dominates.

Proposed Experiments and Decisive Checks

Test sets. The main testbed is BioASQ 8, using the five official batches and MAP plus Recall@100 and Recall@1000. Zero-shot transfer is measured on BEIR's biomedical subsets (TREC-COVID, NFCorpus, SciFact) with nDCG@10 and Recall@100. Natural Questions is the general-domain control, where we expect little gain because the queries rarely hinge on multi-word technical terms.

Cost measures. We report bytes per passage including all index structures, query latency on CPU, and indexing time including the grammar pass.

First decisive check: the frontier. We plot accuracy against bytes. The proposal succeeds only if the three-part index at roughly 96x dense compression sits above both the 16x hybrid and the dense-only frontier at equal bytes on BioASQ, and roughly ties on Natural Questions.

Second decisive check: attribution. Compression-induced misses must be enriched for multi-word-term queries, and the phrase index must recover a majority of exactly those misses.

Third check: domain versus general encoders. Comparing the domain encoder with the general encoder under the same compression sweep tells us whether domain pretraining makes embeddings more or less compressible. This answer is useful whichever way it comes out.