Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation for Semi-Supervised Medical Image Segmentation

Tao Tan, Yuanbin Chen, Tao Wang, Hui Tang, Longxuan Zhao, Ruige Zong, Tao Tan 0002, Xinlin Zhang, Tong TongarXiv 2023 · DOI · arXiv

Introduction

Deep learning has revolutionized medical image segmentation, yet it remains tethered to a significant "radiologist-in-the-loop" bottleneck. Achieving high-performing, clinically viable models typically necessitates thousands of pixel-perfect annotations. Such datasets are prohibitively expensive and time-consuming to curate, requiring extensive manual labor from medical experts.

Semi-supervised learning (SSL) offers a promising solution by leveraging a small pool of labeled data alongside a much larger repository of unlabeled images. However, traditional SSL methods often struggle with "noisy" pseudo-labels—incorrect model predictions used as ground truth—and fail to capture the inherent uncertainty in complex anatomical structures like the pancreas or heart. The DCPA (Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation) framework addresses these challenges by combining a Mean-Teacher architecture with a novel dual-decoder strategy to maximize information extraction from unlabeled data while ensuring model stability.

Methodology

The DCPA framework is built upon four primary technical pillars designed to improve robustness in low-label regimes.

1. Dual-Decoder Architecture

Unlike standard U-Net or V-Net models that utilize a single decoder, DCPA employs two distinct decoders branching from a shared encoder. Crucially, these decoders utilize different upsampling strategies: one employs transposed convolutions, while the other uses linear interpolation. This structural disparity forces the decoders to process features differently, providing a natural, architectural measure of model uncertainty without requiring manual noise injection.

Overview of the DCPA framework showing the interaction between the Teacher and Student models, the dual-decoder structure, and the pseudo-label guided data augmentation pipeline.

2. Mean-Teacher Framework

The system maintains a "Student" model and a "Teacher" model. The Teacher’s weights are updated via an exponential moving average (EMA) of the Student’s weights. During training, the Teacher generates pseudo-labels for unlabeled data, which then serve as targets to guide the Student’s learning process, providing a stable temporal consistency mechanism.

3. Pseudo-Label Guided Mixup

The researchers adapted the "Mixup" augmentation technique specifically for the medical imaging domain. By blending unlabeled images with labeled ones, the model is forced to learn more generalized features. Instead of assigning a single uncertain label to the resulting composite image, the framework uses the Teacher’s pseudo-labels as the ground truth for the unlabeled portion of the mix, effectively bridging the distribution gap between labeled and unlabeled sets.

Visualization of the Mixup strategy: an unlabeled image and a labeled image are combined to create a mixed training sample, guided by pseudo-labels.

4. Cross-Consistency and Sharpening

To refine the pseudo-labels, the model applies a "sharpening" function, which pushes the Teacher's soft probability outputs closer to binary one-hot encodings. The two decoders then engage in cross-supervision: Decoder 1’s sharpened prediction acts as the target for Decoder 2, and vice versa. This forces the model to converge on a holistic representation that is independent of the specific upsampling mathematics used in either decoder.

Datasets and Evaluation

The DCPA framework was rigorously evaluated on three diverse and challenging medical imaging benchmarks:

  • Pancreas-CT: Comprising 82 3D abdominal CT scans. The pancreas is notoriously difficult to segment due to its small size, irregular shape, and low contrast against surrounding tissues.
  • Left Atrium (LA): 100 3D gadolinium-enhanced MR scans from the 2018 Atrial Segmentation Challenge.
  • ACDC (Automated Cardiac Diagnosis Challenge): 100 2D cardiac MR scans featuring the left/right ventricles and myocardium.

Key Results

DCPA consistently outperformed existing state-of-the-art (SOTA) methods, such as UA-MT and SASSNet, particularly in extreme data-scarcity scenarios:

  • Pancreas-CT: In the most challenging case with only 5% labeled data, DCPA achieved a 79.16% Dice score. For comparison, a standard supervised V-Net trained on the same 5% subset achieved only 29.32%, demonstrating the framework's ability to prevent model collapse.
  • Left Atrium: With 20% labeled data, DCPA reached a 91.64% Dice score, nearly matching the performance of a model trained on 100% of the labels (91.71%).
  • ACDC: The model demonstrated high stability in 2D multi-class segmentation, outperforming competitors in segmenting complex cardiac structures even with minimal training samples.

Discussion and Future Work

The success of DCPA highlights that structural diversity (via different upsampling methods) is a more principled way to capture model uncertainty than simple input noise. Furthermore, the use of Mixup as a regularizer effectively prevents the model from over-fitting on small labeled datasets.

While the results are state-of-the-art, the authors identify several areas for future exploration. Currently, the architectural disparity is limited to upsampling methods; future iterations could explore mixing CNN and Transformer-based decoders. Additionally, while the dual-decoder approach increases training memory overhead, the inference speed remains unaffected as only the Student model is required for deployment. The authors suggest that the "Pseudo-Label Guided Augmentation" logic could be extended beyond segmentation to medical image classification tasks.

Summary updated

BibTeX
@misc{tan2023dual,
  author = {Tao Tan and Yuanbin Chen and Tao Wang and Hui Tang and Longxuan Zhao and Ruige Zong and Tao Tan and Xinlin Zhang and Tong Tong},
  title = {{Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation for Semi-Supervised Medical Image Segmentation}},
  year = {2023},
  eprint = {2308.16573},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.16573},
}
Figure