Domain Adaptation Without the Compute Burden for Efficient Whole Slide Image Analysis

Umar Marikkar, Muhammad Awais, Sara Atito

TL;DR

This paper introduces EfficientWSI (eWSI), a novel framework that combines Parameter-Efficient Fine-Tuning (PEFT) with Multiple Instance Learning (MIL) to enable efficient end-to-end training for Whole Slide Image (WSI) analysis. eWSI significantly reduces the computational burden of WSI analysis, achieving performance comparable to or better than existing methods, even when using general-purpose ImageNet pre-trained models.

What Problem Does This Paper Address?

Analyzing Whole Slide Images (WSIs) is crucial for tasks like cancer diagnosis and classification. However, WSIs are massive (gigapixels), making traditional end-to-end deep learning training computationally prohibitive. Current approaches often rely on pre-trained feature extractors (like those trained on ImageNet) to generate fixed "patch" representations, which are then fed into Multiple Instance Learning (MIL) models. While domain-specific pre-training on histopathology data can yield better features, it's computationally expensive and still doesn't fully capture task-specific nuances. This leaves a gap: how to achieve efficient, end-to-end training for WSI analysis that adapts to specific tasks without requiring massive computational resources or extensive in-domain pre-training.

Methods

The core of eWSI is the integration of two key components:

  • Parameter-Efficient Fine-Tuning (PEFT): Specifically, the authors use LoRA (Low-Rank Adaptation). Instead of fine-tuning the entire pre-trained encoder, LoRA selectively updates a small number of parameters (primarily the query and MLP components of the transformer architecture). This allows for efficient adaptation of general-purpose encoders to the histopathology domain without the high computational cost of full fine-tuning.
  • Multiple Instance Learning (MIL) with a novel aggregator (LinMax): eWSI uses a sparse sampling strategy to select patches from each WSI during training. These sampled patches are then processed by the PEFT-adapted encoder. The resulting patch representations are aggregated using LinMax, a custom MIL aggregator designed to be robust to varying patch sampling rates and to leverage the enriched patch-level information from PEFT.

This combination allows eWSI to learn task-specific representations directly from WSI data in an end-to-end fashion, while keeping computational demands manageable.

Datasets

The authors evaluated eWSI on the following datasets:

  • Camelyon16: Used for tumor metastasis detection.
  • TCGA: Used for cancer subtype classification (BRCA, NSCLC) and molecular property prediction (mHRD, tHRD, TNBC).
  • BRACS: Used for 3-way classification (benign, atypical, malignant).

Key Insights and Ideas

  • ImageNet Pre-training + PEFT is surprisingly effective: The paper demonstrates that even general-purpose ImageNet pre-trained models, when fine-tuned with PEFT (eWSI), can achieve performance comparable to or better than MIL methods using in-domain pre-trained encoders. This significantly reduces the need for expensive, specialized pre-training.
  • PEFT enables efficient domain adaptation: By only updating a small subset of parameters, PEFT allows models to adapt to the unique characteristics of histopathology images without the prohibitive computational cost of full fine-tuning.
  • Robustness to sparse sampling: The proposed LinMax aggregator and PEFT integration make eWSI robust to variations in patch sampling rates, a critical factor given the computational constraints of WSI analysis. This is important because sparse sampling is necessary to keep training feasible.
  • Task-specific adaptation is key: When eWSI is used with in-domain pre-trained encoders, it further boosts performance, highlighting its ability to capture task-specific information effectively.

Key Results

  • Camelyon16: eWSI with ImageNet encoders achieved performance matching or exceeding MIL methods using in-domain features. With in-domain encoders, eWSI further improved results. For example, eWSI512 achieved 98.5% AUC with the Hist-SSL encoder.
  • TCGA and BRACS: eWSI consistently outperformed MIL methods across various tasks and encoders, showing significant improvements with the in-domain Hist-SSL encoder.
  • Computational Efficiency: eWSI offers a practical alternative to compute-heavy pre-training. Training for 100 epochs on Camelyon16 took approximately 2.5 hours with M=512 patches on an RTX 3090 GPU, significantly faster than some existing end-to-end methods.

Limitations and Future Work

  • Sparse Sampling Bias: The paper notes that sparse sampling can introduce a bias in predicted probabilities, leading to higher false positives. While this can be mitigated by adjusting decision thresholds or performing a secondary frozen-encoder training, it's an area for further refinement.
  • Multi-Magnification: The current study focuses on single magnification. Future work could explore extending eWSI to handle multi-magnification WSIs.
  • Further PEFT Exploration: While LoRA was used, the authors suggest eWSI can be applied with other PEFT strategies.
  • Domain Transfer Analysis: The paper provides insights into weight transformations from ImageNet to histopathology, suggesting that MLP components in later layers undergo the most significant changes, offering avenues for more targeted domain adaptation.

Summary updated

Abstract

Computational methods on analyzing Whole Slide Images (WSIs) enable early diagnosis and treatments by supporting pathologists in detection and classification of tumors. However, the extremely high resolution of WSIs makes end-to-end training impractical compared to typical image analysis tasks. To address this, most approaches use pre-trained feature extractors to obtain fixed representations of whole slides, which are then combined with Multiple Instance Learning (MIL) for downstream tasks. These feature extractors are typically pre-trained on natural image datasets such as ImageNet, which fail to capture domain-specific characteristics. Although domain-specific pre-training on histopathology data yields more relevant feature representations, it remains computationally expensive and fail to capture task-specific characteristics within the domain. To address the computational cost and lack of task-specificity in domain-specific pre-training, we propose EfficientWSI (eWSI), a careful integration of Parameter-Efficient-Fine-Tuning (PEFT) and Multiple Instance Learning (MIL) that enables end-to-end training on WSI tasks. We evaluate eWSI on seven WSI-level tasks over Camelyon16, TCGA and BRACS datasets. Our results show that eWSI when applied with ImageNet feature extractors yields strong classification performance, matching or outperforming MILs with in-domain feature extractors, alleviating the need for extensive in-domain pre-training. Furthermore, when eWSI is applied with in-domain feature extractors, it further improves classification performance in most cases, demonstrating its ability to capture task-specific information where beneficial. Our findings suggest that eWSI provides a task-targeted, computationally efficient path for WSI tasks, offering a promising direction for task-specific learning in computational pathology.
BibTeX
@inproceedings{marikkar2026domain,
  author = {Umar Marikkar and Muhammad Awais and Sara Atito},
  title = {{Domain Adaptation Without the Compute Burden for Efficient Whole Slide Image Analysis}},
  booktitle = {MIDL},
  year = {2026},
  url = {https://openreview.net/forum?id=gNyqM50ZY2},
}
Figure