S5CL: Unifying Fully-Supervised, Self-supervised, and Semi-supervised Learning Through Hierarchical Contrastive Learning

Manuel Tran, Sophia J. Wagner, Melanie Boxberg, Tingying Peng

Introduction

In the field of computational pathology, the acquisition of high-quality, annotated datasets remains a significant bottleneck. While Whole Slide Images (WSIs) are abundant, the process of manual annotation by expert pathologists is labor-intensive and time-consuming. To mitigate this, researchers typically employ Semi-Supervised Learning (SSL) techniques. However, conventional SSL often relies on a disjointed two-stage pipeline: an initial self-supervised pre-training phase followed by a supervised fine-tuning phase. This separation can lead to feature misalignment and sub-optimal performance, particularly on the highly imbalanced datasets common in medical research.

This paper introduces S5CL, a unified framework designed to collapse these stages into a single, cohesive training step. By leveraging a hierarchy of contrastive losses, S5CL achieves state-of-the-art performance in sparse-label scenarios, demonstrating remarkable robustness on both balanced and imbalanced histopathological benchmarks.

The S5CL Framework

The core philosophy of S5CL (Supervised, Self-Supervised, Semi-Supervised Contrastive Learning) is the integration of multiple learning paradigms into a single objective function. Unlike traditional methods that treat labeled and unlabeled data sequentially, S5CL processes them simultaneously.

The framework utilizes three distinct contrastive losses:

  1. Supervised Contrastive Loss ($L_L$): This loss encourages the model to cluster labeled images of the same class together in the embedding space while pushing images of different classes apart.
  2. Self-Supervised Loss ($L_U$): This loss focuses on instance-level discrimination, pulling an unlabeled image and its augmented versions (e.g., rotations, color jitters) toward each other.
  3. Semi-Supervised Loss ($L_P$): After an initial "warm-up" period, the model generates pseudo-labels for unlabeled data. These pseudo-labels allow the model to treat unlabeled samples as labeled, further refining the class-specific clusters.

Hierarchical Contrastive Learning

The "hierarchical" aspect of S5CL is its most significant innovation. The authors define a specific distance hierarchy within the model's embedding space:

  • Primary Proximity: An image and its own augmentations should be the most similar.
  • Secondary Proximity: Different images belonging to the same class should be close, but less so than an image is to its own augmentation.
  • Distal Proximity: Images from different classes should be far apart.

To enforce this hierarchy mathematically, the authors manipulate the "temperature" parameter ($\tau$) within the contrastive loss functions. By setting the self-supervised temperature higher than the supervised temperature ($T_U > T_L$), the model is forced to prioritize instance-level consistency before class-level clustering. This prevents the model from collapsing different classes together prematurely and results in more compact, well-separated feature representations.

Experimental Evaluation

The S5CL framework was evaluated on two challenging histopathological datasets:

  • NCT-CRC-HE-100K: A balanced dataset containing 100,000 H&E-stained patches of colorectal cancer and normal tissue across nine classes.
  • Munich AML Morphology: A highly imbalanced dataset of 18,365 single-cell images from leukemia patients across 15 classes.

Results

S5CL demonstrated superior performance, particularly when labeled data was extremely scarce. On the NCT-CRC-HE-100K dataset, using only 0.5% of available labels (50 images per class), S5CL achieved an accuracy of 95%. This represents a 9% improvement over standard supervised learning and outperforms complex alternatives like Meta Pseudo Labels (MPL).

On the Munich AML Morphology dataset, the framework addressed the challenges of class imbalance effectively. S5CL achieved a 6% increase in F1-score compared to baseline methods. UMAP visualizations further confirmed that S5CL produces cleaner, more distinct clusters than standard methods, which often suffer from "smearing" between similar morphological classes.

Discussion and Conclusion

S5CL represents a shift toward more efficient, unified training pipelines in medical AI. By treating supervised, self-supervised, and semi-supervised learning as a single hierarchical task, the model avoids the pitfalls of multi-stage training and "confirmation bias" often found in pseudo-labeling schemes.

However, the method is not without limitations. The success of the pseudo-labeling component depends on a carefully timed "warm-up" period; initiating pseudo-labels too early can degrade performance. Additionally, the hierarchical structure is sensitive to the selection of temperature hyperparameters. Future research may explore the application of S5CL to other contrastive loss variants and its scalability to even larger, more diverse medical imaging tasks.

Summary updated

BibTeX
@inproceedings{tran2022s,
  author = {Manuel Tran and Sophia J. Wagner and Melanie Boxberg and Tingying Peng},
  title = {{S5CL: Unifying Fully-Supervised, Self-supervised, and Semi-supervised Learning Through Hierarchical Contrastive Learning}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2022},
  doi = {10.1007/978-3-031-16434-7\_10},
}
Figure