Pathology slides are gigapixel images, too large for any neural network to take in at once. Standard practice cuts each slide into thousands of small tiles, encodes each tile with a frozen pretrained network, and combines the tile features into one diagnosis. This setup is called multiple instance learning, because the label belongs to the whole bag of tiles rather than to any single tile. These models degrade when moved to a new hospital, because staining and scanners differ. On the Camelyon17 lymph-node benchmark, plain training falls from about 93% accuracy in distribution to about 70% at unseen centers (WILDS). The new hospital has no labels, so a team can neither retrain the model nor check whether its fixes worked.
This proposal rests on a simple observation. Pathologists already know that some tissue can never be evidence for metastatic cancer: fat, blood, empty glass, and capsule and fibrous connective tissue with no tumor cells in it. Suppose the model's slide score is an exact sum of per-tile contributions, as in Additive MIL. Then at a new site we can measure how much "cancer evidence" the model assigns to tiles of these impossible-evidence types, using a tissue-type labeler that is independent of the classifier and needs no target diagnoses. We call this quantity null-tissue evidence. We propose to use it twice: as an adaptation objective that pushes that evidence back to zero, and as a label-free score for choosing hyperparameters and checkpoints at the new site. The interpretable decomposition stops being only a display for clinicians and becomes the supervision signal for handling domain shift when labels are scarce.
Core Claim
Our claim is that, under cross-center shift, a large share of a patch-based classifier's false-positive slide score comes from positive evidence placed on tissue types that are medically incapable of carrying the diagnosis. If that is true, null-tissue evidence should do three things. It should track target-site error without target labels. It should rank adaptation hyperparameters and checkpoints better than entropy or confidence. And when used as a constraint during adaptation, it should recover accuracy without the self-reinforcing collapse that pseudo-labeling shows under shift. The idea differs from domain-invariance methods in a specific way. It does not ask target features to match source features, which Out-of-Distribution Robustness via Targeted Augmentations shows can discard genuinely predictive site-dependent signal. It only imposes a sign constraint on a medically defined subset of tiles.
Research Program
The model is an additive multiple-instance classifier on frozen tile features. For a slide $X$ with tiles $x_1,\dots,x_N$, the slide logit is
$$s(X)=\sum_{i=1}^{N}\phi_i,\qquad \phi_i=p\big(\alpha_i\,h_\theta(f(x_i))\big),$$
where $f$ is a frozen pathology foundation encoder, $h_\theta$ is a small site-specific feature transform (identity on the source site), $\alpha_i$ is an attention weight, $p$ maps a weighted tile feature to a tile logit, and $\phi_i$ is tile $i$'s exact additive contribution. Additive MIL supplies this construction; we reuse it unchanged. That paper evaluated only in-distribution on Camelyon16 and TCGA and never studied what its tile contributions do under scanner shift. That open question is what we study.
The null set $\mathcal{N}(X)$ contains the tiles that a tissue-typing model labels, with high confidence, as fat, blood, background, or capsule/fibrous connective tissue free of tumor cells. We will build that tissue labeler in two ways so the result does not depend on one source. The first uses zero-shot prompts to a pathology vision-language model, in the spirit of the class-agnostic concept anchors in CATE. The second is a small tissue classifier trained on a few hundred source-site tile annotations. On an unlabeled target site, adaptation fits only $h_\theta$, a low-rank or per-channel affine correction in feature space, by minimizing
$$\mathcal{L}=\mathcal{L}_{\text{src}}+\lambda\,\frac{1}{|\mathcal{N}|}\sum_{i\in\mathcal{N}}\max(0,\phi_i)^2+\mu\,\max\!\big(0,\;\bar\phi^{\,0}_{\mathcal{T}}-\bar\phi_{\mathcal{T}}\big).$$
Here $\mathcal{L}_{\text{src}}$ is the ordinary classification loss on retained source slides. The middle term penalizes positive cancer evidence on null tiles. The last term guards against a trivial solution in which all evidence shrinks. It requires the mean contribution $\bar\phi_{\mathcal{T}}$ over tiles the tissue labeler calls "tumor-like" to stay at or above its pre-adaptation value $\bar\phi^{\,0}_{\mathcal{T}}$, and $\lambda,\mu$ are weights. Training never uses the classifier's own pseudo-labels on positive tiles. That is the deliberate difference from the self-training loops that SFDA-DeP shows collapse to the dominant class.
This also bears on an apparent contradiction in the literature. SFDA-DeP reports that self-training collapses under shift. In contrast, Noisy Student succeeds on Camelyon17 (WILDS 2.0) and pseudo-label consistency succeeds in MiDSS. Our hypothesis is that self-training fails exactly when its errors concentrate on tissue the model should ignore. If so, null-tissue evidence should predict which sites a self-training run will collapse on, and we will test that directly.
The second deliverable is model selection. For every hyperparameter setting and checkpoint of every adaptation method, including baselines, we compute the target null-tissue evidence mass. We then measure its rank correlation with true target accuracy and AUC, which are computed only for evaluation. Most adaptation methods are tuned on target labels that would not exist in deployment, an open problem the file itself flags. A selection rule that works across methods would be useful even if our adaptation objective were not.
Annotation scarcity enters in three places. Source slide labels are subsampled to 10%, 25%, and 50% of slides. Only a few hundred tissue-type tile annotations are allowed. No target annotations are used except a small held-out evaluation set. That evaluation set also measures how accurate the tissue labeler itself is at each target center, since the tissue labeler could also be affected by shift.
Baselines
The naive reference is the additive model trained on the source site and applied unchanged to target sites. A strong model of this kind is reported: Additive MIL gives a Camelyon16 slide AUC of 0.846 and a localization AUPRC of 0.42, against 0.750 and 0.36 for attention pooling. We also run plain attention pooling, so we can confirm that the additive structure does not cost accuracy under shift.
The strongest shift-specific competitor is targeted stain augmentation. Out-of-Distribution Robustness via Targeted Augmentations reports that it raises Camelyon17-WILDS out-of-distribution accuracy from 75.3% to 90.5%. That figure is a tile-level result on the WILDS split, not a slide-level one. We will reproduce it at the tile level for comparability and then apply the same augmentation when training the slide models.
The strongest label-free adaptation competitors are Noisy Student self-training, reported at 86.7% versus 70.8% for plain training on Camelyon17 (WILDS 2.0), and SFDA-DeP, reported to hold up to 86.2% on hard centers where self-training baselines fell to 50%. We also include TAKT, which uses optimal-transport feature alignment and reports an AUC of 0.952 for NSCLC→Camelyon16 transfer. We include CATE, which reports roughly 4–5% out-of-distribution AUC gains on TCGA-BRCA, plugged into the same aggregator.
All methods share the same frozen encoder features, the same labeled-slide subsets, the same adaptable parameter budget, and equal hyperparameter-search budgets. Each is reported twice: once with oracle target-label selection and once with label-free selection. Beating the strongest baseline means one of two things. Either the method matches or exceeds stain augmentation plus self-training in target slide AUC under label-free selection, or it adds a significant gain on top of stain augmentation when combined with it, across centers and seeds.
The key ablations isolate tissue knowledge. The first replaces $\mathcal{N}$ with a random tile set of the same size. The second removes the guard term. The third swaps the vision-language tissue labeler for the source-trained one. If random null sets work about as well as tissue-defined ones, the core claim is false.
Why not just…?
Why not just use stain-color augmentation? It is the most serious threat. On Camelyon17 it already reaches 90.5% out of distribution (Targeted Augmentations), and modern foundation encoders may be fairly stain-robust already. We treat it as a component, not a rival. If null-tissue adaptation adds nothing on top of augmentation, the adaptation claim fails. The selection and monitoring claim would still stand, because augmentation offers no way to know, without labels, whether a given new site is being handled correctly.
Why not just run self-training or SFDA-DeP? Self-training's success is site-dependent. It collapses on some Camelyon17 centers (SFDA-DeP) yet works on others (WILDS 2.0). SFDA-DeP's forgetting ratio must be set without target labels. We run both methods, select their hyperparameters with our label-free score, and test whether that score predicts their collapses. If it does, the score is useful even for methods we did not design.
Why not just use CATE or a concept-bottleneck classifier? CATE uses concepts to filter features upstream during labeled training and then feeds them to an opaque aggregator. KnoBo and the chest X-ray concept bottleneck work on whole single images, not tile bags. None of them uses concept knowledge as a constraint on unlabeled target data or as a selection signal. CATE plus our null-tissue term is a planned combination.
Why not just classify slides zero-shot with the vision-language model? If the tissue labeler can find fat, it might also be able to find metastases. We report zero-shot slide classification as a baseline. The proposal relies on the asymmetry that coarse tissue typing is far easier than detecting small metastases. We measure that asymmetry directly on the held-out annotated tiles at each center.
Proposed Experiments
The source site is Camelyon16 and the targets are the individual Camelyon17 centers, evaluated at the slide level, with Camelyon17-WILDS tile splits for tile-level comparisons (WILDS). A second axis is TCGA-BRCA with site-held-out splits, following CATE's out-of-distribution protocol. Every condition is crossed with the 10%, 25%, and 50% label regimes and with two frozen encoders, and reports slide AUC, F1, and tile localization AUPRC.
The first decisive check is diagnostic. Before any adaptation, we test whether the null-tissue evidence mass correlates with per-center error across centers, encoders, and label regimes. We also check whether false-positive slides carry disproportionate evidence on null tiles. If neither holds, the premise fails and we stop.
The second check is the selection rule. Pooling every method's hyperparameter grid and checkpoints, we compare the rank correlation of null-tissue evidence with true target AUC against entropy, mean confidence, and source validation accuracy. We also measure the accuracy gap between label-free and oracle selection.
The third check is adaptation. The constrained transform is compared alone and combined with stain augmentation, against the baselines above under equal budgets. The random-null-set ablation decides whether medical knowledge about tissue, rather than generic regularization, explains any gain.
The fourth check concerns explanations. Using the same annotated target tiles, we measure whether localization faithfulness, which Additive MIL and the mammography curriculum study only measured in distribution, degrades under shift and recovers after adaptation.
References
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Additive MIL: Intrinsically Interpretable Multiple Instance Learning for Pathology
- Out-of-Distribution Robustness via Targeted Augmentations
- Free Lunch in Pathology Foundation Model: Task-specific Model Adaptation with Concept-Guided Feature Enhancement
- Adaptation of Weakly Supervised Localization in Histopathology by Debiasing Predictions
- Extending the WILDS Benchmark for Unsupervised Adaptation
- Constructing and Exploring Intermediate Domains in Mixed Domain Semi-supervised Medical Image Segmentation
- TAKT: Target-Aware Knowledge Transfer for Whole Slide Image Classification
- A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis
- Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models
- Annotation-efficient, patch-based, explainable deep learning using curriculum method for breast cancer detection in screening mammography