Reducing self-supervised learning complexity improves weakly-supervised classification performance in computational pathology

Jakob Nikolas Kather, Md, MSc, Tim Lenz, Omar S. M. El Nahhas, Marta Ligero, Jakob Nikolas KatherarXiv 2024 · DOI · arXiv

Abstract

Computational pathology is currently dominated by an "arms race" toward increasingly massive foundation models, creating significant barriers to entry due to high computational costs. This study investigates whether reducing the complexity of Self-Supervised Learning (SSL) can democratize the field without sacrificing performance. By strategically reducing data volume, modifying transformer architectures, and implementing semantically relevant sampling, the authors achieved a 90% reduction in training time. Their optimized models matched or exceeded state-of-the-art performance in breast cancer gene mutation prediction and metastasis detection, demonstrating that efficiency and performance can coexist in pathology AI.

Introduction

The development of foundation models in pathology typically requires thousands of GPU hours and massive unlabeled datasets. This reliance on high-end compute clusters limits innovation to a few elite institutions. This paper addresses this challenge by exploring "complexity reduction" in the SSL framework MoCo-v3. The goal is to determine if a more efficient training pipeline can run on consumer-grade hardware while remaining effective for downstream clinical tasks like predicting gene mutations or detecting cancer metastasis.

Methodology

The researchers employed a MoCo-v3 (Momentum Contrast) framework with a tiny Swin Transformer as the encoder. They explored three primary avenues for reducing complexity:

  1. Data Volume Reduction: Testing the impact of training on 10%, 25%, and 50% of the original dataset to identify the point of diminishing returns.
  2. Architectural Modifications: Instead of relying solely on the final output layer, the team extracted features from earlier stages (Stages 1, 2, and 3) and combinations thereof.
  3. Semantically Relevant Sampling: New sampling strategies were introduced to improve representation learning:
  4. Negative Sampling (N-Sam): Selecting highly dissimilar patches to force the model to learn harder distinctions.
  5. Dynamic Sampling (DS): Adjusting sample selection based on intermediate similarity throughout the training process.
Overview of the SSL pipeline and the three complexity reduction experiments: (1) Data Ratio, (2) Swin Modification, and (3) Sampling Experiment.

Datasets

The models were trained on the breast cancer (BRCA) subset of TCGA (The Cancer Genome Atlas), comprising over 4 million image patches from 1,125 slides. Evaluation was performed on two external benchmarks:

  • CPTAC-BRCA: For validating the prediction of TP53, CDH1, and PIK3CA gene mutations.
  • CAMELYON16: For detecting cancer metastasis in lymph nodes.

Results and Discussion

The experimental results revealed that massive data and deep architectures are not always optimal for pathology tasks.

The 50% Rule and Efficiency: Reducing training data by 50% resulted in performance that matched the full-data baseline, suggesting significant redundancy in current large-scale datasets. Overall, the optimized pipeline reduced SSL training duration by 90%.

Architectural Insights: A key finding was the "Last Layer Trap." While the final layer is often considered the most informative in general computer vision, the authors found that the final stage of the Swin Transformer lost useful information for pathology. Utilizing features from the Last 2 stages or Stage 3 alone significantly improved metastasis detection. As shown in the results table, the "Last 2" configuration achieved an average performance (AVG) of 0.840, outperforming the state-of-the-art CTransPath (0.807).

Performance comparison across different models and experimental configurations on CPTAC-BRCA and CAMELYON16 datasets.

Mutation Prediction: The optimized model outperformed CTransPath on TP53 and PIK3CA mutation prediction by approximately 3% (AUPRC), despite using significantly fewer resources. For metastasis detection, the "Last 2 stages" modification boosted the MoCo-v3 baseline from a struggling 0.415 AUPRC to a robust 0.811 AUPRC.

Conclusion

This research demonstrates that strategic reductions in data and architectural complexity can lead to more efficient and higher-performing pathology models. By focusing on semantically relevant features and sampling, researchers can bypass the need for massive compute clusters. While this study focused on breast cancer, future work will explore whether these efficiency-focused techniques can be generalized to pan-cancer datasets to create a universal, lightweight foundation model for computational pathology.

Summary updated

Abstract

Deep Learning models have been successfully utilized to extract clinically actionable insights from routinely available histology data. Generally, these models require annotations performed by clinicians, which are scarce and costly to generate. The emergence of self-supervised learning (SSL) methods remove this barrier, allowing for large-scale analyses on non-annotated data. However, recent SSL approaches apply increasingly expansive model architectures and larger datasets, causing the rapid escalation of data volumes, hardware prerequisites, and overall expenses, limiting access to these resources to few institutions. Therefore, we investigated the complexity of contrastive SSL in computational pathology in relation to classification performance with the utilization of consumer-grade hardware. Specifically, we analyzed the effects of adaptations in data volume, architecture, and algorithms on downstream classification tasks, emphasizing their impact on computational resources. We trained breast cancer foundation models on a large public patient cohort and validated them on various downstream classification tasks in a weakly supervised manner on two external public patient cohorts. Our experiments demonstrate that we can improve downstream classification performance whilst reducing SSL training duration by 90%. In summary, we propose a set of adaptations which enable the utilization of SSL in computational pathology in non-resource abundant environments.
BibTeX
@misc{msc2024reducing,
  author = {Jakob Nikolas Kather, MD, MSc and Tim Lenz and Omar S. M. El Nahhas and Marta Ligero and Jakob Nikolas Kather},
  title = {{Reducing self-supervised learning complexity improves weakly-supervised classification performance in computational pathology}},
  year = {2024},
  eprint = {2403.04558},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.04558},
}
Figure