RankMix: Data Augmentation for Weakly Supervised Learning of Classifying Whole Slide Images with Diverse Sizes and Imbalanced Categories

Yuan-Chih Chen, Chun-Shien Lu

Abstract

Whole Slide Image (WSI) analysis in digital pathology faces significant hurdles due to the gigapixel scale of images, the sparsity of diagnostic regions, and inherent class imbalances in medical datasets. We present RankMix, a novel data augmentation strategy designed for Multiple Instance Learning (MIL) frameworks. RankMix operates in the feature domain, utilizing a two-stage teacher-student workflow to identify and mix the most informative patches from pairs of slides. This approach effectively standardizes input sizes and focuses the model on critical diagnostic signals. Experimental results on public and private datasets show that RankMix consistently outperforms state-of-the-art augmentation techniques, particularly in scenarios with extreme class imbalance.

Introduction

In digital pathology, a single Whole Slide Image (WSI) can contain billions of pixels. Analyzing these images is computationally intensive and often relies on Multiple Instance Learning (MIL), where an image is treated as a "bag" of small patches. However, standard MIL approaches struggle with three primary issues: diverse image sizes that prevent uniform resizing, the "needle in a haystack" problem where diseased regions are sparse, and severe class imbalance in clinical datasets.

Standard Multiple Instance Learning (MIL) pipeline for WSI classification.

Traditional data augmentation methods like Mixup or CutMix, which operate on raw pixels, are often unsuitable for WSIs. They may inadvertently mix irrelevant background tissue, diluting the diagnostic signal and confusing the model. RankMix addresses these challenges by moving augmentation to the feature domain.

Methodology

The RankMix framework follows a two-stage self-training workflow designed to filter and combine the most relevant diagnostic features.

The RankMix framework featuring a two-stage teacher-student self-training process for feature-domain augmentation.

Stage 1: The Teacher

A standard MIL model is first trained on the dataset to learn a "score function." This function identifies which patches within a WSI are most likely to contribute to a positive diagnosis.

Stage 2: The Student and Feature Mixup

In the second stage, a "Student" model is trained using augmented data generated through the following process:

  1. Pseudo-Labeling & Ranking: For any two slides ($X_a$ and $X_b$), the teacher scores all constituent patches.
  2. Selection: The top $k$ highest-scoring patches are selected from each slide. This ensures the model mixes the core diagnostic signals rather than random background noise.
  3. Feature Mixup: These ranked features are blended using a linear combination. By selecting a fixed number of patches ($k$), RankMix can effectively mix slides regardless of their original dimensions.
  4. Order Preservation: To maintain spatial context, the selected patches are rearranged into their original relative order before being processed by the student model.

Experimental Results

The researchers evaluated RankMix across three distinct WSI challenges:

  • Camelyon16: A benchmark for breast cancer metastasis detection characterized by sparse tumor regions. RankMix improved the accuracy of the DSMIL backbone from 86.82% to 89.92%.
  • TCGA-Lung: A multi-class dataset for lung cancer subtyping (LUAD vs. LUSC).
  • WSI-usability: A private, extremely imbalanced dataset for quality control (23 positive vs. 427 negative slides). RankMix provided a dramatic improvement here, raising baseline accuracy from 76.11% to 90.27%.

Across all tested architectures (DSMIL and FRMIL), RankMix consistently improved Accuracy, AUC (Area Under the Curve), and AUPRC (Area Under the Precision-Recall Curve) compared to vanilla MIL and other augmentation methods like ReMix.

Conclusion

RankMix introduces a robust mechanism for WSI data augmentation by leveraging feature ranking and self-training. By focusing on high-importance patches, it overcomes the limitations of image size diversity and signal sparsity. While the two-stage training adds computational overhead, the substantial gains in model performance and stability, especially in imbalanced datasets, make it a valuable tool for medical AI applications. Future work may explore adaptive selection of the patch count $k$ and extending the ranking mechanism to localization tasks.

Summary updated

Abstract

Whole Slide Images (WSIs) are usually gigapixel in size and lack pixel-level annotations. The WSI datasets are also imbalanced in categories. These unique characteristics, significantly different from the ones in natural images, pose the challenge of classifying WSI images as a kind of weakly supervise learning problems. In this study, we propose, RankMix, a data augmentation method of mixing ranked features in a pair of WSIs. RankMix introduces the concepts of pseudo labeling and ranking in order to extract key WSI regions in contributing to the WSI classification task. A two-stage training is further proposed to boost stable training and model performance. To our knowledge, the study of weakly supervised learning from the perspective of data augmentation to deal with the WSI classification problem that suffers from lack of training data and imbalance of categories is relatively unexplored.
BibTeX
@inproceedings{chen2023rankmix,
  author = {Yuan-Chih Chen and Chun-Shien Lu},
  title = {{RankMix: Data Augmentation for Weakly Supervised Learning of Classifying Whole Slide Images with Diverse Sizes and Imbalanced Categories}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.02292},
}
Figure