A dense retriever turns every document in a collection into a vector once, stores those vectors in an index, and at search time turns the user's query into a vector and returns the nearest documents. These systems are good at matching meaning rather than exact words, but they transfer poorly to new domains. When they are fine-tuned for a new domain, the document vectors change too, so the whole collection has to be re-encoded. We propose adapting only the query side. The document encoder and its index stay exactly as they are, and the query encoder gets a small low-rank weight update (LoRA, which adds a trainable product of two thin matrices to each frozen weight matrix). This update is trained against the cached document vectors, using synthetic queries and LLM-written query expansions that are folded into the weights. The index never moves. Training needs no document forward passes. And because the update starts at zero, the adapted query encoder starts out perfectly aligned with the documents.
The scientific claim goes beyond engineering convenience. We hypothesize that the rank of the update controls how far query vectors can drift from the geometry of the document space. A low-rank update starting from zero can move a raw question like "waste management" toward the region where documents about the circular economy live, without breaking the metric alignment that full query-tower fine-tuning or adapter swapping destroys. If this holds, query-side LoRA should beat full query-tower fine-tuning when target-domain training signal is small or synthetic. It should also approach the quality of retrievers that re-index, at a small fraction of the cost.
Why this, and what already exists
Three findings motivate the proposal. BEIR built an 18-dataset zero-shot benchmark and showed that dense retrievers often lose to BM25 out of domain, so domain adaptation is needed. ClimRetrieve showed an "expert gap" in climate disclosures: the best embedding model reaches only 0.167 F1 on raw questions and 0.179 when analysts' conceptual explanations are added. The missing knowledge sits on the query side, as a translation from lay question to expert concept. DEPT named the "moving-target problem": fine-tuning shifts document embeddings and invalidates the index. DEPT's fix is full-model training of a Qwen3-4B expander-and-encoder, with a document-preservation loss, a whitening transform and straight-through gradients through generation. It reaches 42.59 average nDCG@10 on five BEIR datasets versus 36.14 for raw queries, at heavy compute cost.
Two findings constrain the design. Parameter-efficient zero-shot transfer for cross-language dense retrieval found that encoding queries and documents with different adapters drops DPR by 47% nDCG@100, because the two sides land in misaligned coordinates. That warns that any asymmetric scheme must keep queries tied to the document geometry. Scattered or Connected? found that LoRA and adapters train slowly and unstably under contrastive retrieval losses. Its fix is a side pathway called IAA-L, which beats LoRA at 0.5% trainable parameters. That result is a direct threat to using vanilla LoRA, and we must answer it rather than assume it away.
Two lines of work supply the method. LoRA supplies the zero-initialized low-rank update, which leaves $W_0$ untouched and can be merged into the weights. KALE showed that the query encoder can differ from a frozen document encoder: it pruned the query tower and re-aligned it by distillation for speed. Our proposal differs from KALE in purpose (domain and meaning adaptation rather than speed) and in mechanism (a weight-space low-rank update rather than layer pruning). It differs from DEPT by replacing the preservation loss with a structural guarantee and by removing generation from the gradient path. It differs from the cross-lingual adapter work by studying the low-rank update as a controlled departure from an aligned starting point, rather than a swapped module.
Core Claim
Let $f_\theta$ be the query encoder with weights $W_0 + BA$ in each adapted layer. Here $B$ and $A$ are the rank-$r$ factors, and $B$ is initialized to zero. Let $d_j = f_0(\text{doc}_j)$ be the cached document vectors produced by the frozen encoder $f_0$. For a target corpus we generate synthetic queries $q$ from documents with an LLM, and an expanded version $\tilde q$ of each query (a short LLM-written explanation of the underlying concept, in the spirit of ClimRetrieve's expert prompts). We train only $A$ and $B$ with
$$\mathcal{L} = -\log \frac{\exp(s(f_\theta(q), d^+)/\tau)}{\sum_{j \in \mathcal{N}} \exp(s(f_\theta(q), d_j)/\tau)} + \lambda \,\lVert f_\theta(q) - f_0(\tilde q) \rVert^2 .$$
In this objective, $s$ is cosine similarity, $\tau$ a temperature, $d^+$ the source document, $\mathcal{N}$ the positive plus hard negatives mined from the frozen index, and $\lambda$ the weight on the distillation term. The second term teaches the adapter to place a raw query where its expanded form would land. Expansion therefore becomes part of the weights and costs nothing at query time.
The claim has three parts. First, document drift is zero by construction, and training cost scales with query length only. Second, the contrastive problem becomes one of matching against fixed targets, because document vectors act like fixed class prototypes. We predict this removes much of the optimization instability that Scattered or Connected? observed when both towers move. Third, retrieval quality is a non-monotonic function of rank, and query-only full fine-tuning sits at the over-drifting end of that curve.
Research Program
We build the method on two open backbones. One is a BERT-size dual encoder trained on MS MARCO, which matches the Scattered or Connected? setting. The other is a Qwen3-4B-based embedder, which allows a backbone-matched comparison with DEPT. For each target corpus we generate synthetic queries and expansions with a frozen instruction LLM, mine hard negatives from the frozen index, and train one query adapter. Many adapters, one per domain, then share one index. Because only the cheap query side runs through an adapter, requests from mixed domains can be served unmerged in the same batch. This sidesteps the batching limitation noted in the LoRA paper for the side of the system where it matters.
The analysis tracks how far adapted queries move. We measure the distance between $f_\theta(q)$ and $f_0(q)$, and the rank of each true positive among frozen documents, as functions of rank $r$ and training-set size. This tests whether low rank acts as an alignment-preserving regularizer, and where full-rank query adaptation starts to overfit synthetic queries. Separately, we log loss curves and gradient norms for query-only LoRA, symmetric LoRA and symmetric full fine-tuning. This tests whether the frozen document side is what tames the optimization discrepancy. We also try LoRA+ (LoRA+), which uses different learning rates for $A$ and $B$, as a cheap remedy if instability remains. Learning rates are swept for every method, because A Unified Study of LoRA Variants shows that LoRA results hinge on this.
Baselines
The naive reference points are BM25 and the frozen base encoder with raw queries. BM25 matters because BEIR shows it often wins out of domain. The frozen encoder is the "do nothing" point, and every adapted method must clear it.
The training-free alternative is query expansion at search time with a frozen LLM, in the style of HyDE and Query2Doc. Each query is expanded and then encoded by the frozen encoder, using the same LLM that generates our training expansions. DEPT reports comparing against these methods, but the file gives no separate number, so we report our own. This baseline is also the teacher in our distillation term. Our method should match it at zero generation cost, and anything beyond that is a bonus.
The strongest existing method is DEPT, which reports 42.59 average nDCG@10 on SciFact, ArguAna, NFCorpus, FiQA and SCIDOCS with Qwen3-4B (DEPT). We will run on those five datasets with a Qwen3-4B backbone and the same zero-shot protocol. Beating it means exceeding 42.59 average nDCG@10 without full-model training or a preservation loss. Coming within one point at a fraction of the training memory would count as a strong efficiency result, and we will state it as such rather than as a win.
The strongest parameter-efficient competitor is IAA-L from Scattered or Connected?. We run it at a matched trainable-parameter count, both symmetrically and on the query side only. This separates "LoRA versus side pathway" from "frozen versus moving document side."
The key ablations isolate the proposal's ingredients. Query-only full fine-tuning tests whether the low-rank constraint matters at all. Symmetric LoRA and symmetric full fine-tuning, both with re-indexing, show what re-encoding the corpus buys. Removing the distillation term ($\lambda = 0$) shows whether folding in the expansions matters beyond synthetic-query contrastive training. All methods use the same synthetic queries, negatives, GPU-hour budget and number of learning-rate trials.
Why not just…?
Why not just fine-tune the whole query encoder and keep the document encoder frozen? This also guarantees zero index drift, and it is the most serious threat. Our argument against it is specific. A full-rank update can move queries anywhere, and the training signal here is synthetic and per-domain small. The 47% collapse under mismatched query and document adapters shows how quickly asymmetric encoders lose alignment (cross-language adapters), and that same paper found small adapters beat full fine-tuning by 7–14%. If query-only full fine-tuning matches query-only LoRA across ranks and data sizes, the low-rank part of our claim fails. We would then report the method as "query-side adaptation" and drop the regularization story. The rank sweep settles this directly.
Why not just expand queries with an LLM at search time? It needs no training and may capture the same concepts. But it adds a generation step per query. ClimRetrieve also shows that generic prompting barely moves retrieval quality (0.167 F1 raw) without expert content. Our distillation term uses exactly these expansions as a teacher, so the comparison measures whether the knowledge transfers into weights. Matching the teacher at zero generation latency is the minimum bar. Exceeding it, because contrastive training against the real index corrects teacher errors, is the hoped-for result.
Why not just use BM25 or a hybrid with BM25? It is training-free and strong out of domain per BEIR. But it cannot bridge the concept gaps that are the target here, such as waste management versus the circular economy. We will still report BM25 and BM25 fused with our adapted retriever, because if fusion with the frozen encoder already closes most of the gap, adaptation is less necessary.
Why not just re-index after symmetric fine-tuning? Re-indexing gives the document side capacity too, and KALE's finding that document capacity dominates accuracy suggests it may win. We include it as an upper reference and measure the gap against its re-encoding cost. A small gap at large cost savings is the intended outcome.
Proposed Experiments
The main experiment is zero-shot domain adaptation on the five DEPT BEIR datasets plus TREC-COVID, Climate-FEVER and FiQA. Metrics are nDCG@10 and Recall@100, and we also record training GPU-hours, peak memory, and document re-encoding cost (zero for our method). A second experiment uses ClimRetrieve as a meaning-gap probe. With an open encoder, we measure F1 for raw questions, for raw questions with analyst explanations, and for raw questions through the query adapter. Success there means the adapter reaches the expert-prompted F1 using raw questions alone. With only 16 questions, we will present this as a diagnostic, not a headline. A third experiment repeats the training-curve comparison on MS MARCO and TREC 2019 DL. It tests whether freezing documents removes the instability reported in Scattered or Connected?.
Two checks are decisive. The first is the rank curve: query-only LoRA must beat query-only full fine-tuning at small or synthetic data sizes, with drift from $f_0$ explaining the difference. Otherwise the low-rank claim is wrong. The second is the DEPT comparison at matched backbone: if query-side LoRA plus distilled expansion lands far below 42.59, then full-model training of the expander is doing real work that a weight update on the query side cannot replace.
References
- BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models
- ClimRetrieve: A Benchmarking Dataset for Information Retrieval from Corporate Climate Disclosures
- DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval
- Parameter-efficient Zero-shot Transfer for Cross-Language Dense Retrieval with Adapters
- Scattered or Connected? An Optimized Parameter-efficient Tuning Approach for Information Retrieval
- LoRA: Low-Rank Adaptation of Large Language Models
- Quick Dense Retrievers Consume KALE: Post Training Kullback Leibler Alignment of Embeddings for Asymmetrical dual encoders
- LoRA+: Efficient Low Rank Adaptation of Large Models
- A Unified Study of LoRA Variants: Taxonomy, Review, Codebase, and Empirical Evaluation