Overlap-Aware Few-Shot Adaptation for Retrospectively Selected Clinical Cohorts

Clinical models are often trained on a few dozen labeled patients, and those patients are rarely a random draw. They are the ones whose diagnosis was confirmed by surgery, whose records were complete, or whose scan passed a quality check. The other patients usually sit unlabeled in the same hospital archive. This proposal rests on one observation: the outcome label is scarce, but the label "was this patient included in the study?" is free and abundant. Every archived patient carries it. We propose to learn a model of who gets selected from thousands of archived patients, using frozen foundation-model features. That selection model then tells a few-shot classifier, patient by patient, how far to trust its handful of labeled examples and when to fall back on prior knowledge written in text.

The concrete method extends a training-free few-shot adapter. Today it decides once per class how much to trust image prototypes versus text descriptions. We make that decision per patient instead, based on how much the patient resembles the selected cohort. We will test the method on public imaging and intensive-care archives with realistic selection filters imposed on top. Because the full archive is labeled, we can measure what retrospective studies never can: accuracy on the patients the study left out.

Core Claim

Two research threads currently miss each other. Few-shot clinical adapters treat bias as class imbalance inside the support set, meaning the small labeled set the classifier learns from. Selection-bias methods treat it as a population mismatch, but they assume large training sets or simply decline to predict. We claim that once the inclusion indicator is modeled at archive scale, the two problems separate cleanly. The selection model is a large-sample problem. Only the outcome model is a small-sample one. That separation makes it possible to correct for selection bias without paying the variance cost that usually rules such correction out below 100 labels.

Prior Work and Where This Sits

Three papers motivate the gap. Sample Selection Bias in Machine Learning for Healthcare shows that ignoring selection costs up to 22% AUC when a model moves from the selected study population to the broader population it will serve. It also shows that forcing the two distributions to align can do worse than doing nothing when they overlap little. Its remedy, Target Population Identification, predicts only for target patients who resemble the study population and refers the rest to clinicians. It was built and tested on tens of thousands of tabular records. We keep its idea of a representativeness model but work below 100 labels, and we predict for everyone, with graded trust instead of a hard abstention.

Unveiling COVID-19 from Chest X-ray with deep learning: a hurdles race with small data shows that small, retrospectively assembled chest X-ray sets teach networks which archive an image came from rather than what disease it shows. Cropping to the lungs removes external markers but not the difference in disease severity between archives. The PET study Diagnostic accuracy of 18F-FP-CIT PET for clinically uncertain Parkinsonian syndrome illustrates the data layout we exploit: 111 analyzed patients drawn from 272 scanned, with the excluded 161 sitting in the same archive.

Two papers supply the method. Few-Shot, Now for Real: Medical VLMs Adaptation without Balanced Sets or Validation introduces SS-Text+. It fits a ridge classifier in closed form on frozen vision-language features, with no gradient descent and no validation set. A per-class weight ($\lambda_c$) shifts each class toward its text embedding when that class has few or no support images. We reuse its solver and its no-validation discipline. We replace its class-level weight with a patient-level one driven by the selection model. That addresses something SS-Text+ cannot see: a class may have plenty of support images, all of them from the severe end of the disease spectrum.

Can we generate portable representations for clinical time series data using LLMs? supplies frozen embeddings for intensive-care stays. A language model writes a text summary of each stay, and a frozen text encoder embeds it. Light classifiers on these embeddings reach strong performance from 16 target patients. That makes the paper a natural second arm, using time series instead of images.

Three further papers constrain the design. Boosting Transfer Learning with Survival Data from Heterogeneous Domains reweights source patients toward a small target cohort, which locks in whatever selection already shaped that target. Our weights point the other way, from the selected cohort toward the full archive. Contextual Representation Anchor Network to Alleviate Selection Bias in Few-Shot Drug Discovery anchors few-shot tasks to an unlabeled reference pool that is assumed to be unbiased. We make no such assumption: the archive is itself filtered by who sought care, and we only claim to correct the step from archive to study cohort. Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroid removes shared directions from the embedding space. We deliberately do not project out selection directions, because under spectrum bias selection is correlated with severity and therefore with the label. Removing those directions would delete signal the classifier needs.

Research Program

The setting is an archive of $N$ patients, typically thousands, with frozen embeddings $z$. Each patient has an inclusion indicator $s \in \{0,1\}$, and outcome labels $y$ exist only for the $n \le 100$ included patients. On the full archive we fit a regularized logistic selection model $e(z) = P(s=1 \mid z)$. This is an ordinary large-sample problem.

From the labeled patients we fit the SS-Text+ ridge classifier $W_v$, adding stabilized inverse-propensity weights. These weights up-weight included patients who resemble the excluded ones, and they are clipped so that a few patients cannot dominate. Alongside it we keep the zero-shot text classifier $W_t$. A test patient's class scores are

$$ f(z) = \alpha(z)\, W_v z + \bigl(1-\alpha(z)\bigr)\, W_t z, \qquad \alpha(z) = \min\!\left(1, \frac{e(z)}{\tau}\right). $$

Here $\alpha(z)$ is the trust placed in the labeled data for patient $z$. The threshold $\tau$ is set without a validation split, as the 10th percentile of $e$ over the support set. Patients who look like typical included patients therefore get the data-driven classifier, and patients far outside the selected region drift toward the text prior.

We also report a deferral rule. A patient is referred to a clinician when $\alpha(z)$ is small and the text classifier's prediction has high entropy. This lets us compare directly against the abstain-everything-outside-the-cohort behavior of Target Population Identification.

The benchmark is a second contribution in its own right. We impose realistic retrospective filters on fully labeled public archives, using the CheXpert and NIH-LT chest X-ray benchmarks from the SS-Text+ evaluation with CONVIRT features, and MIMIC-IV intensive-care stays with Record2Vec embeddings. The three filters are these. A protocol filter keeps only upright, posteroanterior chest films; portable anteroposterior films come disproportionately from sicker, bedbound patients. A complete-case filter keeps only stays with a full lab panel. A verification filter preferentially keeps confirmed, severe positives. In each case the labeled support set is drawn from the selected subset, and evaluation runs on the whole archive.

Baselines

The naive reference is a ridge linear probe on the same frozen features, trained on the selected support set with no correction. It shows what an uncorrected retrospective study would deploy. We will also report its apparent accuracy within the selected cohort next to its true accuracy on the full archive, so that the size of the inflation is visible.

The strongest existing competitor is SS-Text+ from Few-Shot, Now for Real. It is built for exactly this clinical few-shot regime: imbalanced classes, absent classes, and no validation set. Its text fallback might partially cover missing parts of the disease spectrum on its own. The paper reports that standard linear probes lose up to about 5 accuracy points in realistic imbalanced settings while SS-Text+ holds its average class accuracy. I found no reported result on Lacuna for SS-Text+ on CheXpert or NIH-LT under selection-filtered support sets. We will therefore rerun it ourselves, with identical features, support draws and seeds, and no tuning for any method.

The second strong competitor is MT-Net, the multitask variant of Target Population Identification from Sample Selection Bias in Machine Learning for Healthcare. It shares one network between selection and outcome prediction. We will train it on the same archive and inclusion indicators. Its selection head sees all $N$ patients and its outcome head sees the $n$ labeled ones. The comparison is on accuracy-versus-coverage curves, and on full-coverage accuracy when its abstentions are forced to predict. The paper reports the 22% AUC drop that uncorrected models suffer. I found no reported AUC for MT-Net itself in the file.

For the survival-style transfer comparison on MIMIC outcomes, we include instance reweighting in the style of Transfer Survival Boosting. Its reported Euro-cohort C-index of 0.677, against 0.500 for a Cox model trained only on target data (Boosting Transfer Learning with Survival Data from Heterogeneous Domains), fixes the scale of gains that reweighting has delivered elsewhere.

Two ablations isolate the key ingredient. The first uses inverse-propensity weights alone, with $\alpha \equiv 1$. The second uses the per-patient blending alone, without weights.

Beating the baselines means a higher balanced accuracy or AUROC on the full, unselected archive, averaged over at least 50 support draws. The gain must hold with confidence intervals separating it from SS-Text+, and most of it should come from the lowest selection-propensity deciles.

Why not just…?

Why not just reweight the labeled patients by inverse selection propensity? With fewer than 100 labels, a handful of included patients who resemble excluded ones receive enormous weights. The effective sample size collapses. Sample Selection Bias in Machine Learning for Healthcare found that correction of this kind can underperform the naive model when overlap is limited. Weighting is still a real threat, and our first ablation tests it directly. If weighting alone matches the full method, we will report that the blending adds nothing.

Why not just trust SS-Text+'s text prior? SS-Text+ falls back to text only for classes with few images. Under spectrum bias, the "pneumonia" class may have many images, all of them severe, so the method trusts those images fully and misreads mild cases. This is the most plausible rival. The severity-stratified comparison settles it. If SS-Text+ already performs as well as our method on low-propensity patients, the per-patient weight is unnecessary.

Why not just abstain on unrepresentative patients, as Target Population Identification does? Under verification filters, the excluded patients are often the ambiguous early presentations. These are the cases where clinicians most need support: the PET study found 37.8% of its patients clinically inconclusive at baseline (Diagnostic accuracy of 18F-FP-CIT PET). Abstention is the right choice only if graded prediction is wrong more often than chance in those patients. Our accuracy-coverage curves show directly where each policy wins.

Proposed Experiments and Decisive Checks

The main experiment crosses three filters with two modalities and support sizes of 4, 16 and 64 labeled patients. It reports full-archive AUROC, balanced accuracy, calibration error, and the gap between apparent and true performance, all stratified by propensity decile.

Three checks could falsify the claim. First, the method must do no harm: under random selection it should match SS-Text+, because $e(z)$ is then flat and $\alpha \approx 1$ everywhere. Second, if selection depends only on factors invisible in the embedding, such as an unrecorded clinician judgment, we expect no gain. We will simulate this case to confirm that the method fails where it should, rather than producing spurious improvements. Third, we will shrink the archive size $N$ and show that the gain disappears as the selection model loses its large-sample advantage. If the gain persists at small $N$, the archive-scale selection model is not what drives it.

A positive result would mean that retrospective few-shot clinical studies should routinely release inclusion indicators for their whole archive, not just the curated cohort.