Exploring Low-Rank Property in Multiple Instance Learning for Whole Slide Image Classification

Jinxi Xiang, Jun ZhangICLR 2023 · OpenReview

Introduction

In digital pathology, Whole Slide Images (WSIs) are massive—often containing billions of pixels. Because these gigapixel-sized images are too large for standard deep learning architectures, researchers typically formulate WSI classification as a Multiple Instance Learning (MIL) problem. In MIL, an image is broken into thousands of small "patches" (instances), which are embedded into vectors and then aggregated to make a single slide-level diagnosis.

However, two major hurdles remain in this domain:

  1. Feature Quality: Standard models often use encoders pre-trained on natural images (like ImageNet), which don't translate perfectly to the nuances of human tissue. Supervised fine-tuning is difficult because patch-level labels are rarely available.
  2. Computational Complexity: To achieve optimal results, models need to understand how different patches relate to one another (cross-instance correlation). While Transformers excel at this, their self-attention mechanism has a quadratic complexity ($O(n^2)$), making them prohibitively slow or memory-intensive when dealing with bags of 10,000+ patches.

Researchers from Tencent AI Lab have developed a new framework to address these challenges by exploiting the inherent "low-rank" structure of WSIs. By introducing a Low-Rank Constraint (LRC) for feature learning and an Iterative Low-Rank Attention (ILRA-MIL) model for data aggregation, they achieved state-of-the-art accuracy while reducing the computational complexity of modeling patch-to-patch interactions from quadratic to linear.

Methods: Exploiting Low-Rank Properties

The authors propose a two-stage solution centered on the observation that WSIs exhibit low-rank properties. Despite their high dimensionality, biological tissues repeat structures, meaning the data actually lies in a much simpler, structured subspace.

Overview of the proposed WSI classification framework featuring LRC feature extraction and ILRA-MIL aggregation.

The framework consists of two main innovations:

  • Pathology-Specific Low-Rank Constraint (LRC): The authors extended contrastive learning (a self-supervised technique) to include a low-rank constraint. By treating different pathological tissues as distinct subspaces, the LRC method effectively performs a "soft" clustering during feature extraction. This allows the model to pull together patches belonging to the same type of tissue in a low-rank subspace and push apart different tissues, leading to much more discriminative embeddings than standard self-supervised methods like MoCo or SimCLR—all without needing manual patch-level annotations.
  • Iterative Low-Rank Attention MIL (ILRA-MIL): To solve the $O(n^2)$ complexity problem of standard self-attention, the authors introduced a small, learnable "latent matrix" that acts as a bottleneck.
Architecture of the Iterative Low-Rank Attention (ILRA) module, demonstrating how input instances interact with a latent matrix bottleneck.

Instead of every patch attending to every other patch, all patches interact with this small latent matrix. This bottleneck changes the computational cost from quadratic ($O(n^2)$) to linear ($O(n)$), allowing the model to capture global correlations across thousands of patches efficiently without massive memory overhead.

Key Results and Datasets

The model was evaluated on three major public benchmarks, outperforming several strong baselines (including CLAM and TransMIL) across all tasks:

  • CAMELYON16: Focused on binary metastasis detection in breast cancer (approx. 1.5 million patches). The model achieved 96.49% AUC, a significant jump over the 87.79% achieved by the popular CLAM-MB model.
  • TCGA-NSCLC: Used for lung cancer subtyping (Lung Squamous Cell Carcinoma vs. Lung Adenocarcinoma; approx. 3.4 million patches). The framework reached 97.63% AUC.
  • PANDA: A large-scale dataset for prostate cancer grading (approx. 1.1 million patches). The model achieved a 0.6562 Cohen’s kappa, demonstrating strong performance on complex multi-class grading.

Furthermore, heatmap visualizations showed that the model's attention highly correlates with regions identified by expert pathologists, even in "needle in the haystack" cases where the tumor region was very small.

Limitations and Future Work

While the results on public benchmarks are impressive, the authors acknowledge a few key areas for improvement:

  • Clinical Validation: The model has not yet been validated on large-scale, multi-center clinical datasets, which often contain more noise and variability.
  • Direct Interpretability: Because the ILRA-MIL uses a latent bottleneck, it does not provide a direct "patch-to-patch" attention score. This might make it slightly harder for a pathologist to trace exactly why the model thinks two specific distant patches are related, though the slide-level heatmaps still provide strong localization.
  • Future Directions: Exploring even more sophisticated low-rank decompositions could further refine the feature embedding process.

Summary updated

Abstract

The classification of gigapixel-sized whole slide images (WSIs) with slide-level labels can be formulated as a multiple-instance-learning (MIL) problem. State-of-the-art models often consist of two decoupled parts: local feature embedding with a pre-trained model followed by a global feature aggregation network for classification. We leverage the properties of the apparent similarity in high-resolution WSIs, which essentially exhibit \textit{low-rank} structures in the data manifold, to develop a novel MIL with a boost in both feature embedding and feature aggregation. We extend the contrastive learning with a pathology-specific Low-Rank Constraint (LRC) for feature embedding to pull together samples (i.e., patches) belonging to the same pathological tissue in the low-rank subspace and simultaneously push apart those from different latent subspaces. At the feature aggregation stage, we introduce an iterative low-rank attention MIL (ILRA-MIL) model to aggregate features with low-rank learnable latent vectors to model global interactions among all instances. We highlight the importance of instance correlation modeling but refrain from directly using the transformer encoder considering the $O(n^2)$ complexity. ILRA-MIL with LRC pre-trained features achieves strong empirical results across various benchmarks, including (i) 96.49\% AUC on the CAMELYON16 for binary metastasis classification, (ii) 97.63\% AUC on the TCGA-NSCLC for lung cancer subtyping, and (iii) 0.6562 kappa on the large-scale PANDA dataset for prostate cancer classification. The code is available at https://github.com/jinxixiang/low_rank_wsi.
BibTeX
@inproceedings{xiang2023exploring,
  author = {Jinxi Xiang and Jun Zhang},
  title = {{Exploring Low-Rank Property in Multiple Instance Learning for Whole Slide Image Classification}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=01KmhBsEPFO},
}
Figure