Lesion-Aware Contrastive Representation Learning for Histopathology Whole Slide Images Analysis

Jun Li, Yushan Zheng, Kun Wu, Jun Shi, Fengying Xie, Zhiguo Jiang

Introduction

In digital pathology, Whole Slide Images (WSIs) are the gold standard for diagnosis. However, their gigapixel scale presents a significant computational challenge, necessitating the division of images into thousands of smaller patches. Learning meaningful representations for these patches without extensive manual labeling is a primary hurdle. While Self-Supervised Learning (SSL) frameworks like MoCo or SimCLR have shown promise, they suffer from the "class collision" problem in histopathology: patches from different patients containing the same lesion type are incorrectly treated as negative pairs, forcing the model to learn divergent features for similar biological structures.

The LACL Framework

To address this, the authors propose Lesion-Aware Contrastive Learning (LACL). LACL leverages slide-level labels—which are far easier to obtain than patch-level annotations—to guide the representation learning process. The framework introduces two primary innovations to make the model "lesion-aware":

1. The Lesion Queue

Standard SSL uses a single memory bank for negative samples. LACL instead maintains $K$ distinct queues, where $K$ corresponds to the number of slide categories. When processing a patch from a specific category, the model only selects negative samples from queues of different categories. This ensures that the model never contrasts a lesion against another instance of the same lesion type, effectively solving the class collision problem.

2. Queue Refinement Strategy (QRS)

Not all patches in a tumor-labeled slide contain malignant tissue; many are healthy tissue or blood vessels. Including these "noisy" patches in the lesion queue can degrade feature quality. The QRS module uses Kullback-Leibler (KL) divergence to evaluate the similarity distribution of a patch against its slide-level label. Only patches that are highly representative of their category are permitted to update the queue.

Overview of the LACL framework, illustrating the dual-branch Siamese network, the category-specific lesion queues, and the QRS module.

Experimental Results

The LACL framework was evaluated on two challenging datasets: the EGFR Dataset (754 WSIs of lung adenocarcinoma) and the Endometrium Dataset (698 WSIs). Performance was measured using downstream classification benchmarks, CLAM (attention-based MIL) and TransMIL (Transformer-based MIL).

Performance Comparison

LACL consistently outperformed existing SSL methods, including MoCo v2, SimSiam, and SimTriplet. On the EGFR dataset, LACL achieved a state-of-the-art AUC of 0.855 using the TransMIL benchmark, significantly higher than the 0.826 achieved by MoCo v2. Similar gains were observed on the Endometrium dataset, where LACL outperformed the next best self-supervised method by over 6% in AUC (0.732 vs 0.670), demonstrating robustness to class imbalance.

Comparison of LACL against state-of-the-art methods on the EGFR dataset across CLAM and TransMIL benchmarks.

Ablation Analysis

Ablation studies confirmed the necessity of both the Lesion Queue and the QRS module. Removing the queue led to a performance drop as class collision returned, while removing QRS allowed non-representative patches to contaminate the memory bank, reducing the discriminative power of the learned features.

Conclusion

LACL demonstrates that weak supervision via slide-level labels can significantly enhance self-supervised representation learning in medical imaging. By intelligently managing negative pairs and filtering non-representative data, LACL produces features that are more robust and biologically relevant for downstream diagnostic tasks. Future work may explore "hard positive" mining—identifying patches from different areas of the same slide that share pathology—to further refine the feature space.

Summary updated

Abstract

Image representation learning has been a key challenge to promote the performance of the histopathological whole slide images analysis. The previous representation learning methods followed the supervised learning paradigm. However, manual annotation for large-scale WSIs is time-consuming and labor-intensive. Hence, the self-supervised contrastive learning has recently attracted intensive attention. In this paper, we proposed a novel contrastive representation learning framework named Lesion-Aware Contrastive Learning (LACL) for histopathology whole slide image analysis. We built a lesion queue based on the memory bank structure to store the representations of different classes of WSIs, which allowed the contrastive model to selectively define the negative pairs during the training. Moreover, We designed a queue refinement strategy to purify the representations stored in the lesion queue. The experimental results demonstrate that LACL achieves the best performance in histopathology image representation learning on different datasets, and outperforms state-of-the-art methods under different WSI classification benchmarks. The code is available at https://github.com/junl21/lacl.
BibTeX
@inproceedings{li2022lesion,
  author = {Jun Li and Yushan Zheng and Kun Wu and Jun Shi and Fengying Xie and Zhiguo Jiang},
  title = {{Lesion-Aware Contrastive Representation Learning for Histopathology Whole Slide Images Analysis}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2022},
  url = {https://openreview.net/forum?id=vdgrGq7jAm},
}
Figure