Teach Me to Segment with Mixed Supervision: Confident Students Become Masters

Jose Dolz, Christian Desrosiers, Ismail Ben Ayed

Abstract

Training medical image segmentation models typically requires massive amounts of pixel-perfect labels, which are expensive and slow to produce. This paper introduces a dual-branch "Teacher-Student" framework that thrives on mixed supervision—using just a few fully labeled images alongside many weakly labeled ones (like scribbles). Remarkably, the study finds that the "student" branch eventually outperforms its "teacher." Evaluated on the ACDC cardiac MRI dataset, the proposed method demonstrates significant improvements in Dice Similarity Scores and boundary accuracy over existing mixed-supervision baselines.

Introduction

Deep learning has revolutionized medical imaging, but its hunger for data is a major bottleneck. In clinical settings, asking a radiologist to manually outline every pixel of an organ in hundreds of MRI scans is impractical. While researchers have explored "mixed supervision" (combining a few full labels with many weak ones), these models often underuse the weak data or fall into "trivial solutions" where the model becomes confidently wrong. This paper aims to bridge the performance gap between these data-starved models and their fully supervised counterparts by better integrating information from unlabeled or partially labeled pixels.

Methods

The authors propose a dual-branch architecture designed to decouple the learning of strong and weak features:

  1. The Teacher (Top Branch): Trained on the small set of fully labeled images using standard cross-entropy loss.
  2. The Student (Bottom Branch): Trained on the weakly labeled images.
The dual-branch framework architecture showing the interaction between the strongly supervised teacher and the less supervised student branch.

The synergy between these branches is governed by a joint objective function:

  • Knowledge Distillation ($L_{kd}$): The teacher "teaches" the student by forcing the student’s predictions on fully labeled images to match the teacher’s predictions using Kullback-Leibler (KL) divergence.
  • Shannon Entropy Minimization ($L_{ent}$): To handle pixels without labels, the model minimizes Shannon entropy. This encourages the student to be highly confident in its predictions (pushing pixel probabilities toward 0 or 1).
  • The Synergy: Entropy minimization alone is risky as the model might become confidently wrong. However, when guided by the Teacher via KL divergence, the entropy term is "reined in," leading to much more accurate segmentations.

Experimental Results

The study utilized the 2017 ACDC Challenge dataset, containing 100 cine Magnetic Resonance (MR) exams, with the task of segmenting the left-ventricle (LV) endocardium. The authors tested three scenarios (Set-3, Set-5, and Set-10), where only 3, 5, or 10 images were fully labeled.

Quantitative comparison of the proposed method against baselines across different labeling scenarios (Set-3, Set-5, Set-10).

One of the most striking findings is that the student branch consistently outperforms the teacher branch. In the most challenging scenario (Set-3), the "Student" achieved a Dice Similarity Score (DSC) of 75.92, significantly higher than the "Teacher" at 69.25. Compared to other state-of-the-art mixed-supervision strategies like the "Decoupled" approach, this method improved Dice scores by 10-20% and reduced the Hausdorff Distance (HD-95) by nearly 30%.

Discussion and Conclusion

The results suggest that the student's exposure to a wider variety of images—even if weakly labeled—allows it to generalize better than a teacher trained on a limited set of perfect labels. Furthermore, the authors show that Shannon Entropy is superior to standard "pseudo-masking" because it is "softer," preventing the model from locking in early mistakes. While currently focused on cardiac MRI, future work includes validating the method on more complex multi-organ tasks and exploring automated hyperparameter balancing for the various loss terms.

Summary updated

Abstract

Deep neural networks have achieved promising results in a breadth of medical image segmentation tasks. Nevertheless, they require large training datasets with pixel-wise segmentations, which are expensive to obtain in practice. Mixed supervision could mitigate this difficulty, with a small fraction of the data containing complete pixel-wise annotations, while the rest being less supervised, e.g., only a handful of pixels are labeled. In this work, we propose a dual-branch architecture, where the upper branch (teacher) receives strong annotations, while the bottom one (student) is driven by limited supervision and guided by the upper branch. In conjunction with a standard cross-entropy over the labeled pixels, our novel formulation integrates two important terms: (i) a Shannon entropy loss defined over the less-supervised images, which encourages confident student predictions at the bottom branch; and (ii) a Kullback-Leibler (KL) divergence, which transfers the knowledge from the predictions generated by the strongly supervised branch to the less-supervised branch, and guides the entropy (student-confidence) term to avoid trivial solutions. Very interestingly, we show that the synergy between the entropy and KL divergence yields substantial improvements in performances. Furthermore, we discuss an interesting link between Shannon-entropy minimization and standard pseudo-mask generation, and argue that the former should be preferred over the latter for leveraging information from unlabeled pixels. Through a series of quantitative and qualitative experiments, we show the effectiveness of the proposed formulation in segmenting the left-ventricle endocardium in MRI images. We demonstrate that our method significantly outperforms other strategies to tackle semantic segmentation within a mixed-supervision framework. More interestingly, and in line with recent observations in classification, we show that the branch trained with reduced supervision and guided by the top branch largely outperforms the latter.
BibTeX
@inproceedings{dolz2021teach,
  author = {Jose Dolz and Christian Desrosiers and Ismail Ben Ayed},
  title = {{Teach Me to Segment with Mixed Supervision: Confident Students Become Masters}},
  booktitle = {IPMI},
  year = {2021},
  doi = {10.1007/978-3-030-78191-0\_40},
}
Figure