Paced-curriculum distillation with prediction and label uncertainty for image segmentation

Mobarakol Islam, Lalithkumar Seenivasan, S. P. Sharan, V. K. Viekash, Bhavesh Gupta, Ben Glocker, Hongliang Ren

Introduction

In the realm of machine learning, Curriculum Learning (CL) draws inspiration from the human educational process, where students master fundamental concepts before progressing to complex subjects. While this "easy-to-hard" strategy has proven effective for image-level tasks like classification, its application to image segmentation remains challenging. Unlike classification, where an entire image is assigned a single label, segmentation requires pixel-wise precision. A single medical image often contains a spectrum of difficulty: clear, unambiguous regions at the center of an organ and fuzzy, highly contested boundaries at the edges.

Standard training protocols typically treat all pixels equally from the first epoch, forcing models to struggle with ambiguous boundaries before they have mastered the core structures. This often leads to sub-optimal convergence and sensitivity to noise. To address this, researchers have introduced Paced-Curriculum Distillation (P-CD), a novel training strategy that quantifies pixel-level difficulty to guide the model through a structured learning path.

Methodology: Quantifying Pixel Difficulty

The core innovation of P-CD lies in its ability to treat an image not as a single data point, but as a collection of "pixel-wise mini-batches." The difficulty of each pixel is determined by fusing two distinct sources of uncertainty:

  1. Prediction Uncertainty (PU): This is derived from the confidence scores of a pre-trained "teacher" model. To ensure these scores are reliable indicators of difficulty rather than mere overconfidence, the authors employ Temperature Scaling to calibrate the teacher’s output.
  2. Boundary Uncertainty (BU): Recognizing that human experts often disagree on the exact borders of tissues, the authors incorporate Spatially Varying Label Smoothing (SVLS). This method assigns higher uncertainty to pixels near object boundaries in the ground-truth labels, acknowledging the inherent ambiguity of these regions.

The Pacing Function and Self-Distillation

P-CD utilizes a dynamic pacing function to manage the training workload. At the start of training (Epoch 0), the model is only penalized for errors on the "easiest" pixels—those with the lowest combined uncertainty. As training progresses, the uncertainty threshold is gradually lowered according to a predefined schedule. By the end of the curriculum, the model is trained on 100% of the pixels, including the most challenging boundaries.

This process is implemented via self-distillation, where a student network is trained to mimic the teacher's logits while simultaneously fitting the ground-truth labels. Crucially, this loss is only calculated for the pixels deemed "ready" by the current stage of the curriculum.

Experimental Results

The P-CD framework was validated across two distinct and challenging medical imaging domains using three popular architectures: UNet, DeepLabV3+, and LinkNet.

Performance Benchmarks

  • Robotic Surgery (MICCAI18): In the task of segmenting surgical instruments (e.g., needles, clamps), P-CD achieved a 2-4% improvement in the Dice Similarity Coefficient (DSC). When integrated with DeepLabV3+, the method outperformed the top three participants of the original MICCAI18 challenge.
  • Breast Ultrasound (BUS): On a dataset of 780 ultrasound images for tumor segmentation, the curriculum approach consistently yielded higher accuracy than standard training, demonstrating its versatility across different imaging modalities.

Robustness to Image Corruption

One of the most significant findings was the model's resilience to "dataset shift." Medical environments are often unpredictable; surgical fields may be obscured by smoke, and ultrasound images are frequently degraded by speckle noise. The researchers tested P-CD against 15 types of image perturbations, including motion blur, Gaussian noise, and simulated fog. In every case, the P-CD-trained models maintained higher performance levels as corruption severity increased compared to models trained with standard methods.

Discussion and Future Directions

While P-CD offers a robust framework for improving segmentation, it introduces specific hyper-parameters that require tuning: the initial ratio of easy pixels ($\beta$) and the duration of the curriculum ($\gamma$). The study found that the optimal starting point varied significantly by task—90% for the relatively structured surgical images versus 50% for the noisier ultrasound data.

Future research aims to further refine the Boundary Uncertainty component. By integrating "multi-rater" data—labels provided by multiple different clinicians—the model could learn to navigate areas of genuine clinical disagreement more effectively. This would move the curriculum from a purely mathematical estimation of difficulty toward one informed by real-world expert ambiguity.

Conclusion

Paced-Curriculum Distillation represents a shift in how we train segmentation models, moving away from uniform pixel weighting toward a more nuanced, human-like learning progression. By explicitly accounting for both model uncertainty and label ambiguity, P-CD produces models that are not only more accurate but significantly more reliable in the face of the noise and artifacts common in clinical practice.

Summary updated

BibTeX
@article{islam2023paced,
  author = {Mobarakol Islam and Lalithkumar Seenivasan and S. P. Sharan and V. K. Viekash and Bhavesh Gupta and Ben Glocker and Hongliang Ren},
  title = {{Paced-curriculum distillation with prediction and label uncertainty for image segmentation}},
  journal = {Int. J. Comput. Assist. Radiol. Surg.},
  year = {2023},
  doi = {10.1007/s11548-023-02847-9},
}
Figure