Knowing When to Abstain on Unfamiliar Tissue: Neighborhood-Aware Selective Cell Classification Across Centers

Pathology models that label individual cells in a tissue image are usually built at one hospital and deployed at another. The new site's staining, scanners and organ mix quietly break them. Two families of fixes exist. Domain adaptation reshapes the model's internal features so that images from the new site resemble images from the old one. Selective prediction lets the model abstain on inputs it is unsure about and send them to a pathologist. Neither fix looks at where a cell sits. A cell whose label is ambiguous, surrounded by hundreds of confidently classified tumor cells, is a different case from the same ambiguous cell lying on a ragged tumor–stroma border. Today's abstention rules score both identically, because they score each cell in isolation.

We propose a rule for deciding whether to answer or abstain that reads a cell's physical neighborhood as well as its own features. The rule estimates how much more likely the cell's features and surroundings are under "the model is right here" than under "the model is wrong here." We then use domain-adversarial training to make these neighborhood features look alike across sites, so that the right-versus-wrong statistics learned at the source site still hold at the target. Target regions the rule rejects are excluded from alignment, which means the model is never forced to adapt tissue it has no business adapting. The deliverable is a cell classifier whose error on the cells it chooses to label stays near a promised level when it moves to a new organ or center. We will test that property directly.

Core Claim

Under cross-site shift, where a cell sits carries information about whether its prediction is correct, beyond what the cell's own confidence carries. That information is also more stable across sites than per-cell confidence. Two consequences follow if this is right. First, a neighborhood-aware abstention score should beat the strongest per-cell scores on the risk–coverage tradeoff. Risk–coverage is the error rate among labeled cells as a function of the fraction of cells labeled. The gain should concentrate at tissue boundaries, and it should not be reproducible by simply averaging neighbors' confidence. Second, source-calibrated risk thresholds should be violated less often at new sites once the neighborhood features are adversarially aligned. The novelty is the object being aligned and scored. Earlier work aligns cell features and then abstains per cell. We treat the cell-plus-context representation as the input to the abstention decision, and we let that decision decide which target cells participate in alignment.

How This Sits Relative to Prior Work

Three abstention papers supply the method and set the bar. Selective Classification for Deep Neural Networks formalized answer-or-abstain for deep networks. Its SGR procedure (selection with guaranteed risk) picks a confidence threshold on held-out data so that test error stays below a target with high probability, provided test data resembles the held-out data. That assumption is exactly what fails across hospitals. Selective Classification Under Distribution Shifts showed that maximum softmax probability degrades under shift. It proposed scale-invariant logit margins but performs no adaptation. Know When to Abstain showed that the optimal abstention rule is a likelihood ratio between the feature densities of correct and incorrect predictions. It estimated this ratio with distances to correct versus incorrect training examples (Δ-KNN, Δ-MDS) and reported roughly 50% lower selective risk on shifted ImageNet variants. We reuse this likelihood-ratio view directly. Our change is the space in which the ratio is estimated: all three papers score each image as an isolated point, and we score a cell together with its tissue neighborhood.

The spatial-context literature motivates that change. OCELOT fed a low-magnification tissue prediction into a cell detector and raised mean F1 from 64.44 to 71.23. It also reports that 93% of tumor cells lie inside annotated cancer areas, so disagreement between a cell's label and its tissue context is informative. BANKSY augments each cell with a neighborhood mean and a rotation-invariant gradient filter, and this separates cell states that identical-looking cells occupy in different niches. DeepSpot and BG-TRIPLEX find that neighborhood branches add substantially even on top of pathology foundation-model features; the BG-TRIPLEX ablation shows the largest drop when the neighborhood branch is removed. All of these works use context to predict better. None uses it to decide whether to predict.

Adversarial adaptation supplies the alignment mechanism and constrains its design. Ren et al. adapted Gleason grading across centers with a domain discriminator plus a slide-consistency loss, raising TCGA→UP accuracy from 54.3% to 77.1%. That still leaves about a quarter of predictions wrong, and there is no option to abstain. FAN-Net found that Fourier style normalization alone underperformed its combination with a gradient-reversal domain classifier. This is evidence that the adversarial term earns its place, although the evidence comes from MRI. The known weakness of global alignment is that it forces every target input into the source feature space, including tissue types absent from the source. Selective pseudo-labeling filters low-confidence target samples out of training as a stabilizing heuristic. Our masking is similar in spirit. It differs in two ways: we filter the alignment loss rather than pseudo-labels, and the same score is the deployed abstention policy that we evaluate for risk control.

Research Program

The base system is a cell classifier on a frozen pathology encoder with a light trainable adapter; CellViT-style embeddings are one option. It predicts a label $\hat y_i$ for each detected cell $i$ with adapted embedding $z_i$. For each cell we build a context vector $c_i$ from four parts. The first is $z_i$. The second is the mean of $z_j$ over physical neighbors within radius $r$, following BANKSY's mean kernel. The third is a gradient term following BANKSY's azimuthal filter, which captures whether the cell sits on an expression or morphology edge. The fourth is two agreement features: the fraction of neighbors sharing $\hat y_i$, and agreement between $\hat y_i$ and an OCELOT-style tissue-branch prediction at the cell's location.

The abstention score is a likelihood ratio estimated non-parametrically in context space:

$$s(c_i) = \bar d_k\big(c_i, \mathcal{W}\big) - \bar d_k\big(c_i, \mathcal{C}\big),$$

Here $\mathcal{C}$ and $\mathcal{W}$ are the context vectors of source cells the classifier got right and wrong, and $\bar d_k$ is the mean distance to the $k$ nearest members of a set. A larger $s$ means the cell resembles past correct predictions more than past errors. Training-set errors are too rare and too optimistic, so $\mathcal{C}$ and $\mathcal{W}$ are built by cross-fitting on held-out source folds. The final score adds the confidence margin from Selective Classification Under Distribution Shifts, following the combination that worked best in Know When to Abstain.

Adaptation works as follows. A domain discriminator receives $c_i$, not just $z_i$, and a gradient-reversal layer (which flips the discriminator's gradient so the encoder learns to fool it) pushes the adapter toward site-invariant context vectors. The purpose is to make the right/wrong neighbor sets $\mathcal{C}$ and $\mathcal{W}$ remain meaningful references for target cells. Target cells with $s(c_i)$ below a rejection level $\tau$ are dropped from the discriminator loss for that epoch. The scores are recomputed each epoch, so the set of aligned tissue grows as the model improves. Deployment uses SGR with source-fold calibration to pick the threshold for a requested risk $r^*$. We report how often that promise holds at the unseen site.

Baselines

The naive reference point is the source-trained classifier with no adaptation and no abstention. Every target cell is labeled, and its full-coverage F1 and error anchor all comparisons. A second naive point is the same classifier with maximum-softmax abstention and SGR thresholds, the recipe of Selective Classification for Deep Neural Networks. It shows how badly source-calibrated guarantees fail under shift.

The strongest existing competitor for abstention is Δ-KNN combined with the logit margin from Know When to Abstain, computed per cell on the same frozen encoder and the same cross-fitted right/wrong sets. It is the strongest competitor because it already approximates the optimal likelihood-ratio rule and was designed for covariate shift. On pathology we found no reported result on Lacuna for it. Its only reported figure is the roughly 50% reduction in average selective risk (AURC and NAURC) on CLIP over shifted ImageNet variants, so we will run it ourselves with identical $k$-selection and tuning budget. We will also run it on top of adversarially adapted features. This gives the obvious two-stage pipeline, "adapt, then abstain per cell," which is the closest composite of existing work.

The strongest adaptation competitor is the discriminator-plus-consistency approach of Ren et al., reimplemented on the same encoder, with its slide-consistency term applied within images. The strongest context-for-accuracy competitor is the OCELOT Pred-to-inter-2 tissue-injection model. In-distribution it reports mean F1 71.23 against 64.44 for cell-only (OCELOT). No cross-organ or selective number was found on Lacuna, so we retrain it under our splits.

Two ablations isolate the key ingredient. The first, spatially smoothed confidence, replaces $s(c_i)$ with the per-cell Δ-KNN score averaged over physical neighbors. It adds spatial information without letting the selector learn which contexts predict error. The second, unmasked alignment, trains the discriminator on all target cells. All methods share encoder, training data, number of adaptation epochs, and a fixed 30-configuration hyperparameter search chosen on source-only validation folds, with no peeking at target labels. The proposal beats the field if three things hold together. Its target AURC must be lower than per-cell Δ-KNN-plus-margin and than adapt-then-abstain, with confidence intervals over held-out domains excluding zero. It must beat the smoothed-confidence ablation. And its SGR violation rate at $r^* = 5\%$ must be lower at equal or higher coverage.

Why not just…?

Why not just average neighbors' confidence scores instead of learning a context-aware ratio? This is the most serious threat, and it is included as an ablation. We expect smoothing to help inside homogeneous regions and to hurt at boundaries. At a boundary, averaging borrows confidence from the other side, while errors concentrate there. The OCELOT observation that tumor cells outside cancer areas are visually confusable points the same way. The decisive test stratifies AURC by distance to the OCELOT tissue-boundary annotation. If smoothing matches our score in every stratum, the learned ratio adds nothing, and we will report the simpler method as the answer.

Why not just use a pathology foundation model with per-cell Δ-KNN and skip both context and adaptation? Foundation features are strong, and Know When to Abstain shows Δ-KNN is robust with very little labeled data. However, DeepSpot and BG-TRIPLEX found that context still adds on top of foundation-model backbones, though for prediction rather than abstention. This baseline is run exactly, with the same encoder. If it matches us, the claim fails.

Why not just use stain augmentation or Fourier style normalization instead of adversarial alignment? FAN-Net found style normalization alone weaker than style normalization plus a domain classifier, but that was on MRI, and adversarial training is brittle. We therefore run a stain-augmented, non-adversarial variant of our full selector. If it matches the adversarial version on AURC and violation rate, we drop the adversarial term. The contribution then reduces to the neighborhood-aware selector, which we would regard as a legitimate, smaller result.

Why not just calibrate thresholds on a few labeled target slides? With exchangeable labeled target data, SGR is valid again, and this is a real alternative when labels are affordable. We will measure how many labeled target images the naive and Δ-KNN baselines need before their violation rate matches ours at zero target labels. This converts our advantage into an annotation budget a pathology lab can weigh.

Proposed Experiments

The primary benchmark is OCELOT under leave-one-organ-out: train on five organs, test on the sixth, rotating across kidney, head-neck, prostate, stomach, endometrium and bladder. This setting provides both the cell labels and the tissue-region labels needed for boundary stratification. The secondary benchmark is PanNuke under leave-tissue-out splits over its 19 tissues and 5 cell classes. It tests whether the effect survives without tissue-region annotations, with neighborhood agreement computed only from predicted neighbors. As a transfer check toward the center shift studied by Ren et al., we run TCGA-internal leave-institution-out prostate splits at patch level, where neighbors are adjacent patches. The UP and CINJ cohorts may not be accessible.

The metrics are AURC, coverage at selective risk 5% and 10%, mean F1 among accepted cells, and the fraction of held-out domains in which SGR's promised risk is exceeded, which we call the violation rate.

Three results would decide the question. First, the boundary-stratified comparison against smoothed confidence tests whether context carries error information beyond pooled confidence. Second, the violation rate of source-calibrated SGR is compared with and without aligning $c_i$; this tests whether alignment restores approximate exchangeability for the abstention decision specifically, not just accuracy. Third, masked and unmasked alignment are compared on target cells of classes or tissue types rare in the source. Our hypothesis predicts that masking lowers the number of confident errors on this tissue, since it is not forced into source clusters. If the second check shows no reduction in violations, the adversarial half of the proposal is unsupported regardless of AURC gains.