Contrastive fine-tuning against stain and scanner bias improves robustness and downstream performance across pathology foundation models

Ryan Kim, Hassan Muhammad, Anish Dulal, Paul S. Scotti, Tanishq Mathew AbrahamarXiv 2026 · arXiv

TL;DR

The authors introduce SPECTRA, a lightweight, model-agnostic framework that uses contrastive fine-tuning to strip away scanner and staining biases from pathology foundation models. By training on spatially matched tissue images, they significantly improve both the robustness and the downstream accuracy of leading models like UNI2-h and Virchow2.

What Problem Does This Paper Address?

In digital pathology, "foundation models" (FMs) are trained on millions of tissue images to learn features useful for diagnosing cancer or predicting biomarkers. However, these models often accidentally learn "acquisition bias"—the specific visual signatures of a particular slide scanner or a lab's unique staining protocol.

This is a major hurdle for real-world deployment. A model might perform perfectly in the lab where it was trained but fail when faced with images from a different scanner or a slightly different shade of H&E stain. Current solutions like stain normalization often discard useful biological information or fail to account for hardware-level differences like optical resolution and compression.

Methods

The researchers propose SPECTRA (Scanner and Pathology Embedding Contrastive Tuning for Robust Alignment). Instead of a massive retraining effort, they use a "post-training" approach:

  1. Contrastive Learning (InfoNCE): They use a "pull and push" strategy. For a given tissue region (the "query"), they pull its representation closer to the same region imaged under a different condition (the "positive") while pushing away representations of different tissue regions (the "negatives").
  2. LoRA Adapters: To keep it efficient, they don't update the entire model. They use Low-Rank Adaptation (LoRA), training less than 5% of the total parameters. This allows the method to be applied to almost any existing backbone.
  3. Dual-View Objective: They align both global features (via the CLS token) and local patch features (via Generalized Mean pooling), ensuring the model is robust at multiple scales.
Figure 1: Left: schematic grid batch on PLISM tiles; columns are registered locations, rows are conditions. Scanner labels are illustrative; S210 is held out during training. Right: for the query, the registered tile of another condition is pulled in and the other T −1 tiles of that condition are pushed away.

Datasets

  • Training: PLISM, a unique dataset of spatially registered tissue microarrays imaged across 7 different scanners and 13 staining protocols.
  • Evaluation:
  • PathoROB: Measures robustness to site and acquisition shifts.
  • HEST: Evaluates spatial transcriptomics (gene expression) prediction.
  • CPTAC: A 38-task benchmark for slide-level cancer classification.
  • THUNDER: A comprehensive suite for tile-level classification and segmentation.

Key Insights and Ideas

  • Robustness is not a trade-off: A common fear in AI is that making a model more robust will hurt its accuracy. SPECTRA shows the opposite: by removing "shortcut" features (like scanner noise), the models are forced to focus on actual biology, which actually improves downstream performance.
  • Early stopping is critical: The best results were achieved very early in the fine-tuning process (often within the first 100–200 steps). This suggests that foundation models already "know" the biology; they just need a quick nudge to ignore the technical noise.
  • Stains are the "Final Boss": The authors found that aligning images across different stains is much harder than aligning across scanners. This is partly because different stains are often applied to adjacent tissue sections, which—while similar—are not identical at a cellular level.

Key Results

The authors tested SPECTRA on seven state-of-the-art backbones, including UNI2-h, Virchow2, and H-optimus-0.

  • Universal Robustness Gains: Every single model showed improved robustness on the PathoROB index. For example, Phikon-v2’s robustness index (RI) surged from 0.470 to 0.836.
  • Better Downstream Performance: Mean gene expression prediction (HEST) and slide-level classification (CPTAC) improved across the board. UNI2-h saw its CPTAC macro-AUC rise from 0.675 to 0.698.
  • Near-Perfect Retrieval: On held-out scanners, the models achieved over 99% top-1 retrieval accuracy for matching identical tissue regions.
SPECTRA consistently improves robustness (PathoROB) and downstream utility (HEST, CPTAC) across all tested foundation models.

Limitations and Future Work

While SPECTRA is a powerful "patch" for existing models, it has some constraints:

  • Single-Source Training: It was trained on the PLISM dataset from a single institution. Testing on more diverse, multi-institutional matched data is a logical next step.
  • Calibration Regressions: Interestingly, while classification accuracy went up, some models became slightly less "calibrated" (more overconfident in their errors).
  • Biological Noise: The use of adjacent serial sections for stain matching introduces slight biological differences that the contrastive loss might inadvertently encourage the model to ignore.

The authors have open-sourced their code and released the fine-tuned adapters, providing a practical path for researchers to make their pathology models more reliable in the clinic.

Summary updated

Abstract

Histopathology foundation models achieve strong performance across diverse downstream tasks, yet their representations have been shown to encode scanner- and site-specific biases, creating generalization gaps and potentially limiting real-world utility. We introduce SPECTRA (Scanner and Pathology Embedding Contrastive Tuning for Robust Alignment), a model-agnostic contrastive learning post-training framework which explicitly encourages scanner- and stain-robust representations, to optimize performance and generalization on downstream applications. Using parameter-efficient LoRA adapters trained on matched multi-scanner and multi-stain whole-slide images sourced from the PLISM dataset, our method aligns spatially corresponding regions across acquisition conditions (scanner, stain), using other locations within each candidate condition as negatives. SPECTRA improved PathoROB robustness index, gene-expression prediction on HEST, slide-level classification AUC on CPTAC, and held-out PLISM retrieval for every model. Overall tile-level classification on THUNDER also improved, with exceptions for segmentation and calibration tasks. We open-source our training and evaluation codebases and release adapters and model weights for selected backbones.
BibTeX
@misc{kim2026contrastive,
  author = {Ryan Kim and Hassan Muhammad and Anish Dulal and Paul S. Scotti and Tanishq Mathew Abraham},
  title = {{Contrastive fine-tuning against stain and scanner bias improves robustness and downstream performance across pathology foundation models}},
  year = {2026},
  eprint = {2611.10093},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10093},
}
Figure