Abstract
Deep learning methods for medical image segmentation are often limited by the high cost and domain expertise required for pixel-wise annotations. While self-supervised contrastive learning has shown promise in learning representations from unlabeled data, it often ignores the small amount of labeled data available in semi-supervised settings. This paper proposes a novel supervised local contrastive loss that leverages limited pixel-wise annotations during the pre-training phase. By forcing pixels with the same label to cluster together in the embedding space, the method improves segmentation performance. To manage the computational complexity of pixel-wise comparisons, the authors introduce downsampling and block division strategies. Evaluation on MRI and CT datasets demonstrates that this approach consistently outperforms state-of-the-art contrastive and semi-supervised techniques.
Introduction
Training accurate semantic segmentation models requires dense, pixel-level annotations. In the medical domain, this process is exceptionally expensive, requiring expert physicians to manually outline organs or tumors slice by slice. To mitigate this, researchers have turned to semi-supervised learning and contrastive pre-training.
A common paradigm involves a two-step pre-training phase: first learning global (image-level) features and then local (pixel-level) features using only unlabeled data. The actual labels are typically introduced only during a final fine-tuning stage. This paper identifies a missed opportunity: by ignoring available labels during contrastive pre-training, models fail to explicitly group pixels of the same anatomical structure together in the latent space.
Methodology
The proposed framework utilizes a standard 2D U-Net backbone and introduces a hybrid pre-training approach that integrates labels earlier in the pipeline.
Contrastive Learning Strategies
The framework consists of three main components as illustrated below:
- Global Contrastive Learning (Self-Supervised): The encoder is trained on unlabeled data to recognize that different augmentations of the same image slice share similar high-level features.
- Supervised Local Contrastive Learning: This is the core innovation. Using the labeled subset of data, the decoder is trained with a loss function that forces feature embeddings of pixels with the same class label (positive pairs) to be close together, while pushing pixels with different labels (negative pairs) apart.
Efficiency and Optimization
Computing a contrastive loss between every pixel in high-resolution maps leads to $O(h^4)$ computational complexity. To address this, the authors propose two selection strategies:
- Downsampling: Computing the loss using features separated by a fixed stride, leveraging the high spatial redundancy in medical images.
- Block Division: Splitting the feature map into smaller grids and computing the contrastive loss only within local blocks to save memory.
Additionally, the authors explicitly exclude background pixels from the positive set to focus the model's representational power on complex foreground anatomy.
Key Results
The framework was evaluated on two public datasets: the Hippocampus (MRI) dataset and the MMWHS (CT) cardiac dataset. The proposed method consistently outperformed training from scratch and state-of-the-art semi-supervised methods across various label scarcity settings (5% to 40% labeled data).
- Quantitative Performance: On the MMWHS dataset with 40% labeled data, the
global+local(block)method achieved a Dice score of 0.764, significantly higher than the state-of-the-art self-supervised contrastive method (0.730) and the random initialization baseline (0.715). - Qualitative Analysis: t-SNE visualizations confirmed that the supervised local contrastive loss creates clearly separated, tightly clustered feature embeddings for different anatomical classes, whereas baseline methods resulted in scattered and overlapping clusters.
Conclusion
By incorporating limited label information directly into the contrastive pre-training phase, this method provides a superior initialization for medical image segmentation tasks. While the supervised local contrastive loss is memory-intensive and requires careful tuning of block sizes and strides, its ability to learn class-specific local features significantly reduces the annotation burden for high-performance medical AI models.
Summary updated