Mutation-Weighted Pseudo-Relevance Feedback for Antibody Repertoire Retrieval

When a text search engine returns poor results for a short, ambiguous query, it can run a second pass: it looks at the top few documents from the first pass, assumes they are roughly relevant, and rewrites the query using their content. This trick, called pseudo-relevance feedback (PRF), is well understood in text retrieval but has never been built for biological sequence search. Yet biology offers something text retrieval never had: a generative model of how the "documents" change. Antibodies mutate through somatic hypermutation (SHM), a process whose rate at each position depends strongly on the surrounding nucleotides — the same base change can be 50 times more likely in one flanking context than another, as shown for SARS-CoV-2 and for antibody genes by The mutation rate of SARS-CoV-2 is highly variable between sites and is influenced by sequence context, genomic region, and RNA structure and by the Thrifty CNN model of Thrifty wide-context models of B cell receptor somatic hypermutation. This means that when a retrieval system returns candidate sequences similar to a query antibody, we can compute — from first principles — the probability that each candidate actually arose from the query's lineage under the neutral mutation process, and use that probability to weight the feedback. The core opportunity is a PRF loop in which query drift, the central failure mode of PRF, is detectable by a physical noise model rather than heuristics.

The application that motivates this is repertoire mining. Public antibody databases such as the Observed Antibody Space hold roughly a billion unannotated sequences Observed Antibody Space: A Resource for Data Mining Next-Generation Sequencing of Antibody Repertoires, and a search for the clonal relatives or less-mutated ancestors of a mature antibody is exactly the "remote, underspecified query" regime where single-pass retrieval struggles: Looking for therapeutic antibodies in next-generation sequencing repositories found that only 16 of 242 clinically studied antibody targets (6.6%) had even a 95%-identity match in that corpus. Meanwhile the evolutionary models that could guide such search are stuck at small scale: Conditionally Site-Independent Neural Evolution of Antibody Sequences shows how to model context-dependent mutation and selection, but only for parent–child pairs inside pre-assembled clonal trees, and it explicitly leaves open how to find those lineages in massive unannotated corpora in the first place.

Core Claim

I propose mutation-weighted PRF (Mu-PRF): a two-pass dense retrieval system over antibody repertoires in which the feedback step is weighted by a context-dependent neutral mutation model. Pass one embeds the query antibody with a dual encoder (in the style of Fast, sensitive detection of protein homologs using deep dense retrieval) and retrieves top-$k$ candidates. Pass two does not average those candidates into a centroid, as classical Rocchio feedback would; instead, each candidate $d_i$ receives a weight

$$w_i \;=\; \sum_{j:\, q_j \neq d_{i,j}} \log \mu\big(d_{i,j} \,\big|\, q_{j-2},\ldots,q_{j+2}\big),$$

where $q$ is the germline-aligned query, position $j$ runs over substituted sites, and $\mu(\cdot \mid \cdot)$ is the neutral context-dependent substitution distribution from the Thrifty CNN, a 2,057-parameter model of Thrifty wide-context models of B cell receptor somatic hypermutation, evaluated on the germline-side flanking context so substitutions are oriented along the true SHM direction. These weights enter a feedback encoder (query text concatenated with candidate sequences, ANCE-PRF style from Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback) as attention biases, and the refined query embedding is re-run against a static index. The claim is that this beats both single-pass dense retrieval and unweighted PRF, with the largest gains on heavily mutated queries far from germline — precisely where a centroid of divergent candidates would produce a biologically meaningless "chimera" vector and where uniform feedback invites query drift.

The conceptual contribution is that Mu-PRF converts the mutation-modeling literature from a prediction tool into a retrieval operator. Nucleotide context models outperform protein language models for predicting antibody affinity maturation and Conditionally Site-Independent Neural Evolution of Antibody Sequences both use the Thrifty model to predict substitutions or subtract neutral drift given known trees; neither uses it to decide which retrieved sequences deserve trust inside a search loop over a billion-sequence corpus. Conversely, GV-Rep: A Large-Scale Dataset for Genetic Variant Representation Learning and the dense homolog retriever of Fast, sensitive detection of protein homologs using deep dense retrieval both do single-pass vector search over sequence databases and stop there — DHR explicitly benchmarks against PSI-BLAST, which is itself a classical lexical PRF loop, but implements no dense feedback of its own.

Research Program

Three components need to be built. First, the retrieval substrate: a dual encoder fine-tuned on antibody sequences (initializing from the antibody-capable protein encoders used in Fast, sensitive detection of protein homologs using deep dense retrieval and the protein language models compared in Nucleotide context models outperform protein language models for predicting antibody affinity maturation), indexing OAS-scale repertoires with inner-product search. Second, the mutation scorer: the Thrifty CNN applied germline-to-query and query-to-candidate along V(D)J germline alignments, producing the per-candidate log-likelihood $w_i$. Ralph's compound-Poisson bounding of dependency propagation in Enabling inference for context-dependent models of mutation by bounding the propagation of dependency is what makes this tractable: adjacent substitutions interact, but their joint effect can be bounded within a local window, so candidate scoring over long sequences does not require the whole-sequence joint state space. Third, the feedback encoder: a cross-attention module over the query and top-$k$ candidate sequences in which the attention logits over candidate positions are initialized (or clamped) proportional to $w_i$, with the document index left static, reusing ANCE-PRF's key architectural insight from Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback that cross-attention over raw feedback beats vector averaging because it can suppress noise.

A second analysis strand treats $w_i$ as a drift detector, not just a reweighting. Candidates whose neutral mutation likelihood is so low that the query–candidate gap cannot be explained by SHM (convergent sequences from unrelated clones, or paralog noise) should be flagged and discarded before they touch the query embedding. This is the structural answer to the query-drift pathology documented in Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback, and it is something no text PRF method can do, because text has no equivalent of a neutral drift model separating signal from selection. Conditionally Site-Independent Neural Evolution of Antibody Sequences supplies the template for this disentanglement: its Selection Score is computed by subtracting Thrifty-predicted neutral rates from observed transitions, and Mu-PRF reuses exactly that subtraction, but as a retrieval-time filter rather than a per-tree fitness estimate.

Baselines

The "do nothing" reference point is first-pass dense retrieval alone: the dual encoder returning top-$k$ neighbors with no feedback pass, which is the operating mode of Fast, sensitive detection of protein homologs using deep dense retrieval and the single-step indexing setup of GV-Rep: A Large-Scale Dataset for Genetic Variant Representation Learning. It is the strongest simple competitor because dense retrieval is already faster and more sensitive than alignment-based search at low sequence identity — DHR reports a greater-than-10% sensitivity increase over previous methods on SCOPe and a greater-than-56% gain at the superfamily level, where identity drops below 20% — so any PRF machinery must justify its added latency against this bar. For a reported number on clonal-family recall over OAS, no reported result found on Lacuna for either paper; the retrieval numbers will be established by this project. It will be run with the same encoder, index, and compute budget as Mu-PRF, differing only in the absence of a second pass.

The strongest classical competitor is PSI-BLAST-style iterative profile search: an initial alignment pass, a position-specific scoring matrix built from the top hits, and a re-query — the biological ancestor of lexical PRF. It is the incumbent for sensitive sequence search, and any claim of a new feedback loop must beat the feedback loop practitioners already use. It will be run with the same repertoire database and the same wall-clock and iteration-count budget as Mu-PRF's two passes, and it wins if its recall@k matches or exceeds Mu-PRF at equal or lower latency; DHR's reported up-to-22-fold speedup over PSI-BLAST and up-to-28,700-fold over HMMER suggests this is unlikely, but those numbers are for protein homology search, not clonal-family retrieval, so the comparison must be made directly on our task.

The strongest dense competitor is single-pass dense retrieval (above) plus two neural PRF variants transplanted from text retrieval: Vector-PRF, which moves the query embedding toward the centroid of the top-$k$ candidate embeddings via Rocchio interpolation as analyzed in To Interpolate or not to Interpolate: PRF, Dense and Sparse Retrievers, and an ANCE-PRF-style concatenation cross-encoder with uniform feedback weights, per Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback. These get the same top-$k$, the same tuning budget over feedback hyperparameters, and the same second-pass index. Beating them means a statistically significant improvement in recall@100 and NDCG@10 across held-out repertoires, with the decisive test being the hard stratum: queries with many somatic mutations, where the nearest repertoire neighbors are convergent noise. Finally, the key-ingredient ablation is the mutation-weighted feedback encoder itself against the identical architecture with uniform attention weights — same parameters, same training data, only the $w_i$ weighting removed. If that gap is small, the idea has failed, and the proposal should be judged accordingly.

Why not just…?

Why not just use PSI-BLAST? Because PSI-BLAST is the classical PRF loop and its weaknesses are exactly what dense retrieval was built to fix. Fast, sensitive detection of protein homologs using deep dense retrieval reports that dense retrieval is up to 22 times faster than PSI-BLAST and achieves greater-than-10% higher sensitivity on SCOPe, rising to greater-than-56% at the superfamily level below 20% sequence identity — the regime that antibody clonal retrieval lives in. More fundamentally, PSI-BLAST's profile scores each column independently, ignoring the flanking-context dependencies that dominate SHM, which the mutation-modeling literature (The mutation rate of SARS-CoV-2 is highly variable between sites and is influenced by sequence context, genomic region, and RNA structure; Thrifty wide-context models of B cell receptor somatic hypermutation) shows carry up to 50-fold rate differences. PSI-BLAST will still be run head-to-head, because no reported result found on Lacuna for PSI-BLAST clonal-family recall on antibody repertoires, so the comparison is genuinely open.

Why not just use protein language model embeddings (ESM, AbLang2) and skip nucleotide mutation modeling? This is a real threat and the evidence against it is direct. Nucleotide context models outperform protein language models for predicting antibody affinity maturation compared protein language models against the nucleotide CNN on the Tang et al. human BCR repertoire and found the nucleotide context model dominant — Thrifty-SHM alone reached an overlap of 0.834, and the selection-integrated Thrifty-prod variant reached 0.923 overlap, 0.139 R-precision, and 0.363 substitution accuracy, while the protein models performed poorly — with the authors concluding that protein language models miss the fine-grained nucleotide processes driving SHM. Conditionally Site-Independent Neural Evolution of Antibody Sequences independently found it had to subtract Thrifty neutral rates just to compute a clean fitness signal. Mu-PRF still uses protein-level encoders for retrieval, but the feedback weighting comes from the nucleotide model, which is exactly the component the protein-only alternative discards; the uniform-weight ablation is the experiment that quantifies what that discarding costs.

Why not just use a 5-mer lookup table like S5F instead of the Thrifty CNN? Because the lookup-table approach collapses exactly when the feedback problem gets hard. Thrifty wide-context models of B cell receptor somatic hypermutation shows context extends to 13 nucleotides, where an explicit table needs over 67 million parameters, while a 13-base window is reachable with about 2,000 parameters via the thrifty CNN, which outperformed both S5F and the 250,000-plus-parameter DeepSHM. In Mu-PRF the mutation model is called on millions of query–candidate pairs per feedback step, so parameter count and calibration matter far beyond a single-sequence prediction task.

Why not just do vector-average PRF (Rocchio centroid)? Honest answer: this might recover much of the benefit on easy, lightly mutated queries, and the proposal treats it as a serious baseline rather than a strawman. The counter-evidence is that in text retrieval the centroid method was already beaten by cross-attention feedback with the gains concentrated on hard queries — Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback reports 4.2% MRR@10 on MS MARCO, 7.6% NDCG@10 on TREC DL 2020, and 9.3% NDCG@10 on the hard-query DL-HARD benchmark — and in sequence space the failure mode is worse: averaging embeddings of divergent antibodies risks a "chimera" vector corresponding to no real clone, which then anchors the second pass in the wrong region of the index. The mutation-weighted versus uniform-weight versus centroid comparison, stratified by mutation load, is the experiment that settles this, and if mutation weighting adds nothing over the centroid on hard queries, the proposal's central claim is wrong.

Decisive Checks

The ground truth comes from repertoires with known clonal structure: the clonal-family corpora behind Nucleotide context models outperform protein language models for predicting antibody affinity maturation (Tang et al. human BCR data, plus the Rodriguez, Ford, and Jaffe repertoires evaluated there; no reported result found on Lacuna for numeric values on the latter two) and the roughly 120,000 clonal trees with about 2 million parent–child transitions across five public databases used to train CoSiNE in Conditionally Site-Independent Neural Evolution of Antibody Sequences. One family member is held out as the query; other members, and the tree parent where the query is mutated, are the relevant set. Metrics are recall@k and NDCG@10 for relative retrieval, plus an ancestor-recovery rate. The corpus is an OAS-scale index (~1 billion sequences per Observed Antibody Space: A Resource for Data Mining Next-Generation Sequencing of Antibody Repertoires) augmented with the therapeutic antibodies catalogued in CoV-AbDab: the Coronavirus Antibody Database as an end-use case, since Looking for therapeutic antibodies in next-generation sequencing repositories shows that current search recovers high-identity relatives for only 6.6% of such antibodies — the practical gap this work targets.

Four checks decide the proposal. First, the stratified comparison: Mu-PRF must beat first-pass dense retrieval, PSI-BLAST, centroid PRF, and uniform-weight cross-encoder PRF on hard (heavily mutated) queries specifically, not just on average. Second, the drift experiment: inject known non-relatives (convergent CDR-H3 lookalikes) into the feedback set and measure whether the $w_i$ filter discards them at rates far above the uniform baseline. Third, the calibration check: verify that Thrifty likelihoods computed on retrieved true relatives are well separated from those on retrieved non-relatives — if the distributions overlap, the mutation model has no signal to weight with, and the whole mechanism reduces to uniform PRF. Fourth, latency accounting: the static-index design means the second pass costs one feedback-encoder forward per query, and the method must stay within a constant factor of single-pass dense retrieval, the constraint that kept ANCE-PRF practical in text retrieval. If checks one and three both pass, the result is a new retrieval operator for sequence biology: feedback weighted by the physics of the mutation process itself.