Calibrating the Dice Loss to Handle Neural Network Overconfidence for Biomedical Image Segmentation

Michael Yeung, Leonardo Rundo, Yang Nan, Evis Sala, Carola-Bibiane Schönlieb, Guang Yang

Introduction

In biomedical imaging, tasks often involve identifying sparse structures like tumors or vessels within large volumes of healthy tissue. This extreme class imbalance makes standard Cross-Entropy (CE) loss ineffective, as models can achieve high accuracy by simply predicting the background class. While the Dice Similarity Coefficient (DSC) loss has become the gold standard for these tasks by focusing on the overlap between predictions and ground truth, it suffers from a critical flaw: poor calibration.

Standard Dice loss encourages models to output probabilities near 0 or 1, even in ambiguous regions. In clinical settings, such "overconfident" predictions can provide false reassurance by missing lesions without indicating uncertainty. This paper introduces DSC++, a reformulation of the Dice loss designed to produce well-calibrated models where probability scores accurately reflect the model's confidence.

Methodology

The authors propose DSC++ as a drop-in replacement for the standard Dice loss. The core innovation involves a "focal" parameter applied to the error components of the Dice formula.

  1. Reformulation: The standard Dice loss is rewritten in terms of True Positives (TP), False Positives (FP), and False Negatives (FN).
  2. Focal Weighting: Inspired by Focal Loss, an exponential weighting parameter, $\gamma$, is applied to the FP and FN components.
  3. Selective Penalization: By setting $\gamma > 1$, the loss function more heavily penalizes incorrect predictions made with high confidence. This forces the model to "hedge its bets" at boundaries or in ambiguous regions, leading to more realistic probability outputs.
  4. Threshold Tuning: Because DSC++ outputs are better calibrated, clinicians can adjust the softmax threshold during post-processing to prioritize either recall (sensitivity) or precision (specificity) without retraining the model.
Comparison of the standard DSC training and deployment pipeline versus the proposed DSC++ approach. DSC++ results in well-calibrated predictions that provide useful uncertainty information to human operators.

Experimental Evaluation

The effectiveness of DSC++ was validated across six diverse biomedical datasets covering various modalities and dimensions:

  • DRIVE: 2D retinal vessel segmentation.
  • BUS2017: 2D breast tumor segmentation (ultrasound).
  • 2018DSB: 2D cell nuclei segmentation (microscopy).
  • ISIC2018: 2D skin lesion segmentation.
  • CVC-ClinicDB: 2D colorectal polyp segmentation.
  • KiTS19: 3D kidney and kidney tumor segmentation (CT scans).

Results and Discussion

The study found that a focal parameter of $\gamma = 2$ consistently provided the optimal balance between segmentation accuracy and calibration error across nearly all datasets.

  • Calibration Gains: On the DRIVE dataset, DSC++ reduced the Negative Log Likelihood (NLL)—a standard metric for calibration—from 0.204 to 0.041 compared to standard Dice loss.
  • Performance Maintenance: Unlike post-hoc calibration methods that often sacrifice accuracy, DSC++ maintained or even improved segmentation performance. For instance, on the BUS2017 dataset, the Dice score increased from 0.784 to 0.842.
  • 3D Segmentation: In the challenging KiTS19 task, DSC++ significantly outperformed standard Dice and Focal losses, achieving a tumor Dice score of 0.429 compared to 0.232 for standard Dice.
  • Uncertainty Visualization: Well-calibrated models generate heatmaps that highlight regions of uncertainty, providing crucial context for medical professionals that "all-or-nothing" models obscure.

Conclusion

DSC++ addresses the inherent overconfidence of the standard Dice loss by integrating calibration directly into the training objective. While the method significantly improves reliability in clinical deployment, the authors note that it inherits the gradient instability issues of the original Dice loss when denominators are small. Future work will focus on stabilizing these gradients and testing the model's robustness against out-of-distribution data.

Summary updated

Abstract

The Dice similarity coefficient (DSC) is both a widely used metric and loss function for biomedical image segmentation due to its robustness to class imbalance. However, it is well known that the DSC loss is poorly calibrated, resulting in overconfident predictions that cannot be usefully interpreted in biomedical and clinical practice. Performance is often the only metric used to evaluate segmentations produced by deep neural networks, and calibration is often neglected. However, calibration is important for translation into biomedical and clinical practice, providing crucial contextual information to model predictions for interpretation by scientists and clinicians. In this study, we provide a simple yet effective extension of the DSC loss, named the DSC++ loss, that selectively modulates the penalty associated with overconfident, incorrect predictions. As a standalone loss function, the DSC++ loss achieves significantly improved calibration over the conventional DSC loss across six well-validated open-source biomedical imaging datasets, including both 2D binary and 3D multi-class segmentation tasks. Similarly, we observe significantly improved calibration when integrating the DSC++ loss into four DSC-based loss functions. Finally, we use softmax thresholding to illustrate that well calibrated outputs enable tailoring of recall-precision bias, which is an important post-processing technique to adapt the model predictions to suit the biomedical or clinical task. The DSC++ loss overcomes the major limitation of the DSC loss, providing a suitable loss function for training deep learning segmentation models for use in biomedical and clinical practice. Source code is available at https://github.com/mlyg/DicePlusPlus .
BibTeX
@article{yeung2023calibrating,
  author = {Michael Yeung and Leonardo Rundo and Yang Nan and Evis Sala and Carola-Bibiane Schönlieb and Guang Yang},
  title = {{Calibrating the Dice Loss to Handle Neural Network Overconfidence for Biomedical Image Segmentation}},
  journal = {J. Digit. Imaging},
  year = {2023},
  doi = {10.1007/s10278-022-00735-3},
}
Figure