Retrieval-Conditioned Prototypes for Noise-Aware Biomedical Classification

Retrieval-augmented language models look up related documents and paste them into the model's input before answering. In biomedicine they fail in a specific and dangerous way. When the retrieved evidence is wrong, the model accepts it. On the BioRAB benchmark, corrupting 80% of the retrieval corpus drove adverse-drug-event triple extraction down to about 10% F1, and models almost never noticed bad evidence (0% "negative awareness" on PharmKG and BioNLI, 1.07% on ADE) (BioRAB).

We propose a different way to use retrieved evidence for classification-style biomedical tasks. The model does not paste labeled examples into a prompt and trust the generator to weigh them. It retrieves labeled examples and averages their embeddings into a per-query "prototype" for each class: a class center built only from the neighbors relevant to this query. It then classifies by distance to those centers. Because each retrieved example enters the prediction as a weighted average, the classification error sends a clean gradient straight back to the retriever. Each example's position can also be checked against a fixed, domain-wide center for its stated label. Examples whose label disagrees with their geometry get down-weighted, which gives the system a built-in way to notice corrupted evidence.

Core Claim

A prototype bottleneck turns joint retriever training from an expensive and often useless exercise into a cheap and useful one, specifically when classes are internally diverse and the evidence store contains wrong labels. The bottleneck itself is simply a classification layer that computes class centers from retrieved items and predicts by distance.

Two observations motivate the claim. First, joint training of standard text-generating RAG systems gives no reliable gain over training the two halves separately. On HotPotQA, independent fine-tuning reached 30.5 EM and 41.7 F1 in 23.3 hours, while two-phase and joint variants scored about the same (30.8 EM, 42.4 F1 for two-phase) and took up to about 46 hours (Lawton et al.). Second, when the coupling between retriever and output is tight, joint training does change behavior: RA-VQA jointly trained a BERT retriever and a T5 generator and reached its best accuracy with 5 retrieved documents instead of 50 or more.

Our hypothesis is that the payoff of joint training depends on how directly a retrieved item's contribution is reflected in the loss. Distance-to-prototype is about as direct as it gets.

Concretely, let $f$ be a shared encoder. For a query $q$, the retriever returns labeled exemplars $x_1,\dots,x_K$ with labels $y_i$ and retrieval scores $s_i$. A frozen, domain-wide class center $\bar{c}_{y}$ is computed once from the full training set, as in nearest-centroid evaluation (Meta-Baseline). Each exemplar gets a weight that combines retrieval relevance with label consistency, and the per-query prototype for class $k$ is

$$w_i = \mathrm{softmax}_i\big(s_i - \lambda\, d(f(x_i), \bar{c}_{y_i})\big), \qquad c_k(q) = \frac{\sum_{i: y_i = k} w_i\, f(x_i)}{\sum_{i: y_i = k} w_i}.$$

Here $d$ is squared Euclidean distance and $\lambda \ge 0$ controls how much an exemplar is penalized for sitting far from the global center of its claimed class. Prediction is a softmax over $-d(f(q), c_k(q))$, as in Prototypical Networks.

A query that is far from every retrieved prototype, or whose retrieved support is mostly inconsistent, triggers abstention. This is our operational form of negative awareness.

Research Program

Role of each prior line of work.

Prototype classification supplies the mechanism. Prototypical Networks and Meta-Baseline give the distance-based predictor. PSCA supplies the consistency idea: in image hashing, it uses agreement between a classifier and proximity to fixed prototypes to filter noisy pseudo-labels. PSCA applies this to its own training labels. We apply it to retrieved evidence at inference time and inside the retriever's training loss.

RA-SGG is the closest relative. It retrieves neighbors from a memory bank to form multiple fine-grained prototypes for scene-graph predicates and gains up to 3.6% F@K on Visual Genome. It differs from our proposal in three ways: it is visual, it has no learned text retriever trained through the prototype loss, and it has no label-consistency weighting.

ProtoTEx shows that a prototype layer on BART matches black-box accuracy (0.75 F1 on propaganda detection). It also warns that convincing exemplar explanations make people over-trust the model when it is wrong. That warning constrains how we present explanations.

Joint retriever-generator work supplies the training pattern. REVEAL injects retrieval scores into attention so that the task loss reaches the query encoder. We inject the scores into the prototype average instead.

Domain pre-training sets the regime. BSharedRAG shows that a retriever built on a domain-pretrained backbone shared with the generator improved Hit@3 by 5–17% over a separate retriever. BioBART gives a PubMed-pretrained encoder–decoder backbone.

What we build.

The system uses a BioBART encoder with two small adapters on the shared backbone, one for queries and one for exemplars. This follows BSharedRAG, except that we train both adapters jointly through the prototype loss rather than one stage at a time.

The datastore is the labeled training pool: tens of thousands of examples, not millions of passages. Re-encoding it every few hundred steps is cheap, which sidesteps the stale-index problem that plagues joint RAG.

A small generator adapter on the same backbone writes a short rationale. It is conditioned only on the three highest-weight exemplars and must cite them. The rationale is secondary output. We evaluate it for faithfulness to the cited exemplars, not as the main result.

What we analyze.

The analysis asks three questions:

  • Does joint training through the prototype bottleneck beat separately trained retrievers?
  • Does consistency weighting restore robustness under label corruption?
  • Does any of this depend on domain pre-training? We test this by swapping BioBART for general BART. The hypothesis is that consistency weighting only works if the frozen global centers are themselves meaningful, which requires in-domain representations.

Baselines

Do-nothing reference. The do-nothing reference is the domain backbone with a standard linear classification head, fine-tuned on the same training pool with no retrieval. Everything else must beat it to justify retrieval at all.

Strongest non-retrieval competitor. This is nearest-centroid classification with fixed global class centers on the same fine-tuned encoder, following Meta-Baseline. It is the most dangerous baseline because it shares our geometry and simply removes the per-query retrieval. If it ties us, the retrieval-conditioned part of the idea is wrong.

Strongest existing system on the target benchmark. This is BioRAB's detect-and-correct strategy, which reports 94.00% F1 on Ade-corpus-v2 (BioRAB). Alongside it we run BioRAB's plain retrieval setup of MedLLaMA 13B with Contriever, which improved over MedLLaMA 13B alone by 22.37% F1 on ADE. For SemdClass and BioNLI, no reported result was found on Lacuna for the detect-and-correct strategy, so we reproduce it under BioRAB's protocol. Beating it means exceeding 94.00 F1 on clean Ade-corpus-v2 or matching it at a fraction of the parameter count. The more important target is degrading substantially less under label corruption.

Ablations of the key ingredients. All ablations share the same encoder, retriever architecture, datastore, number of retrieved items $K$, and hyperparameter search budget:

  • Prototype classification with a frozen off-the-shelf retriever.
  • Prototype classification with a retriever trained independently by supervised contrastive learning to fetch same-label neighbors. This is the Lawton-style "train separately" control.
  • Our joint model with $\lambda = 0$, which isolates consistency weighting.
  • Our model with a general-domain backbone, which isolates domain pre-training.
  • Plain $k$-nearest-neighbor majority voting over the same retrieved items.

Every variant gets the same tuning budget over learning rate. Lawton et al. show that learning-rate tuning alone can erase apparent gaps between training strategies.

Why not just…?

Why not just use fixed global prototypes without retrieval? If every class formed a single tight cluster, global centers would be enough, and the Meta-Baseline control would tie us. Our bet is that biomedical classes are multimodal: one adverse-event class covers many drug and organ combinations. In that case averaging the whole class blurs it. We test this directly by comparing the two methods on classes stratified by how many clusters their training embeddings form. If the gap does not grow with within-class diversity, we drop the retrieval claim.

Why not just put retrieved labeled examples into an LLM prompt? That is the BioRAB setup, and it is exactly what collapses to about 10% F1 under 80% counterfactual corruption, with near-zero detection of bad evidence. The prompt gives the model no geometric check on whether an exemplar's label is plausible. Our corruption sweep runs both systems on identical corrupted datastores.

Why not just train the retriever separately to fetch same-label neighbors? This is the real threat. Supervised contrastive retrieval may recover most of the benefit, just as independent fine-tuning matched joint training on HotPotQA. The joint signal differs because it rewards exemplars that actually move the query toward its correct prototype, not merely exemplars that share its label. We expect this difference to matter only under label noise, where same-label is no longer a trustworthy signal. If the independent retriever matches the joint one on both clean and corrupted data, we will report that the gain comes from prototypes plus consistency weighting, not from joint training.

Why not just use test-time search such as SeRTS? SeRTS reaches 0.57 accuracy on MedMCQA, 7.55% above no retrieval, but needs 12 Monte Carlo tree-search rollouts per query with a 7B model. We report accuracy against inference compute. SeRTS is not a classifier over labeled evidence, so it serves as a cost reference rather than a head-to-head competitor.

Proposed Experiments

Benchmarks. Primary experiments use BioRAB's classification and inference tasks, Ade-corpus-v2, SemdClass, and BioNLI, following BioRAB's corruption protocol. We inject counterfactual labels into 0%, 20%, 50%, and 80% of the datastore. A general-domain check uses the 18-way propaganda dataset from ProtoTEx, whose 0.75 F1 serves as the reference point for diverse, multimodal classes.

Metrics. We report macro-F1, the area under the curve of F1 versus corruption rate, and negative awareness measured as BioRAB's true-negative rate. For the rationale generator, we measure what fraction of rationale claims are supported by the cited exemplars.

Decisive checks. The decisive checks are a set of predictions; failing any one of them falsifies part of the claim.

  1. At 80% corruption, our model should keep well above BioRAB's roughly 10% figure, and the $\lambda = 0$ ablation should fall much closer to it. Failing this falsifies the consistency mechanism.
  2. The advantage of joint over independent retriever training should be near zero on clean data and grow with corruption. A flat gap means joint training is not earning its cost here either.
  3. Under corruption, swapping BioBART for general BART should hurt our method more than it hurts the linear head. Failing this means domain pre-training contributes nothing specific to the prototype geometry.
  4. Negative awareness, measured by abstention on queries paired with irrelevant support, should rise well above the 0–1.07% reported by BioRAB.

Finally, given ProtoTEx's over-trust finding, we will report rationale faithfulness separately for correct and incorrect predictions. A system that explains its errors as persuasively as its successes is not one we would recommend for clinical use.