Label-Set Loss Functions for Partial Supervision: Application to Fetal Brain 3D MRI Parcellation

Lucas Fidon, Michael Aertsen, Doaa Emam, Nada Mufti, Frédéric Guffens, Thomas Deprest, Philippe Demaerel, Anna L. David, Andrew Melbourne, Sébastien Ourselin, Jan Deprest, Tom Vercauteren

TL;DR

This paper introduces a theoretical framework for "label-set loss functions" to enable deep learning models to learn from partially annotated medical images. They propose a novel "Leaf-Dice" loss function that achieves state-of-the-art performance in segmenting fetal brain MRI, demonstrating the power of their principled approach to partial supervision.

What Problem Does This Paper Address?

Training deep learning models for medical image segmentation, like parcellating fetal brain MRI, typically requires a large number of fully annotated (segmented) images. However, manual annotation is incredibly time-consuming and requires specialized expertise. This creates a bottleneck, as valuable datasets with only partial annotations are underutilized. The paper addresses the challenge of effectively training segmentation models when only some regions of interest are labeled in the training data, a scenario known as partially supervised learning.

Methods

The core contribution is the axiomatic definition of label-set loss functions. This provides a theoretical foundation for how loss functions should behave when dealing with "label-sets" (where a voxel might belong to multiple possible labels) rather than single "leaf-labels."

Key methodological aspects include:

  • Label-set Loss Axiom: The paper defines a crucial property that loss functions must satisfy to be compatible with partial supervision. This axiom ensures that the loss function behaves consistently regardless of how the ambiguity in a label-set is resolved.
  • Marginalization Function (Φ): They formalize the process of converting a probabilistic prediction for multiple possible labels into a single, coherent prediction that aligns with the label-set.
  • Leaf-Dice Loss: A novel generalization of the Dice loss is proposed. This "Leaf-Dice" loss is specifically designed to handle missing leaf-label annotations and is proven to satisfy the label-set loss axiom.
  • Canonical Conversion Method: The authors prove that there's a unique, theoretically sound way to convert existing fully supervised loss functions into label-set loss functions that adhere to their axiom. This validates and formalizes previous ad-hoc approaches.

Datasets

The study utilizes a combination of datasets for training and evaluation:

  • Training Data:
    • 18 fully annotated control fetal brain MRI volumes (from [12]).
  • An additional 146 partially annotated fetal brain MRI volumes from a private dataset.
  • Testing Data:
    • 100 fetal brain 3D MRI volumes from a multi-centric testing set, including volumes from University Hospital Leuven and the publicly available FeTA dataset.

The paper highlights that the partially supervised training dataset combines fully annotated data with a larger set of partially annotated data, which is a key aspect of their experimental setup.

Key Insights and Ideas

  • Theoretically Grounded Partial Supervision: The paper moves beyond empirical ad-hoc methods by providing a formal, axiomatic framework for designing loss functions in partially supervised learning. This is crucial for building robust and reliable models.
  • Leaf-Dice: A Natural Generalization: The Leaf-Dice loss is presented as a principled extension of the Dice loss, specifically tailored for scenarios with missing labels. Its effectiveness suggests that direct generalization is more powerful than simple marginalization of existing losses.
  • Uniqueness of Conversion: The proof that there's only one way to convert fully supervised losses to label-set losses is a significant theoretical result, providing clarity and guidance for future research.
  • Partial Supervision for Fetal Brain MRI: This work pioneers the application of partial supervision to the challenging domain of fetal brain MRI parcellation, opening up new avenues for leveraging under-annotated medical imaging data.

Key Results

  • State-of-the-Art Performance: The proposed Leaf-Dice loss, when applied to fetal brain 3D MRI segmentation, achieved state-of-the-art results compared to three baseline methods (fully supervised, soft-target Dice, and marginal Dice).
  • Quantitative Improvements: In Table 2, the Leaf-Dice method consistently outperformed other partially supervised methods and the fully supervised baseline across most tissue types, showing higher Dice scores and lower Hausdorff distances. For example, it achieved a Dice score of 91.5% for white matter and 90.7% for ventricles.
  • Handling Scarce Annotations: Notably, Leaf-Dice demonstrated superior performance even for tissue types with very few annotations in the training data (e.g., cortical gray matter, deep gray matter, brainstem), a challenge for other methods.
  • Qualitative Success: Figure 2 visually demonstrates that the Leaf-Dice loss provides satisfactory segmentations for all tissue types, even in complex cases like open spina bifida, where other methods struggled.

Limitations and Future Work

  • Focus on Missing Labels: The current theoretical framework and the Leaf-Dice loss are primarily designed for scenarios where the partial supervision arises from missing leaf-label annotations. While this is a common and important case, other forms of partial supervision might require further theoretical development.
  • Generalization to Other Modalities/Tasks: While successful for fetal brain MRI segmentation, the applicability and performance of label-set loss functions on different medical imaging modalities or other segmentation tasks (e.g., lesion segmentation) would be an interesting area for future exploration.
  • Exploration of Other Label-Set Loss Functions: The paper proves the existence and uniqueness of a conversion method for existing losses but also proposes Leaf-Dice as a direct generalization. Further research could explore other novel label-set loss functions that might offer additional benefits.

Summary updated

Abstract

Deep neural networks have increased the accuracy of automatic segmentation, however their accuracy depends on the availability of a large number of fully segmented images. Methods to train deep neural networks using images for which some, but not all, regions of interest are segmented are necessary to make better use of partially annotated datasets. In this paper, we propose the first axiomatic definition of label-set loss functions that are the loss functions that can handle partially segmented images. We prove that there is one and only one method to convert a classical loss function for fully segmented images into a proper label-set loss function. Our theory also allows us to define the leaf-Dice loss, a label-set generalisation of the Dice loss particularly suited for partial supervision with only missing labels. Using the leaf-Dice loss, we set a new state of the art in partially supervised learning for fetal brain 3D MRI segmentation. We achieve a deep neural network able to segment white matter, ventricles, cerebellum, extra-ventricular CSF, cortical gray matter, deep gray matter, brainstem, and corpus callosum based on fetal brain 3D MRI of anatomically normal fetuses or with open spina bifida. Our implementation of the proposed label-set loss functions is available at https://github.com/LucasFidon/label-set-loss-functions.
BibTeX
@inproceedings{fidon2021label,
  author = {Lucas Fidon and Michael Aertsen and Doaa Emam and Nada Mufti and Frédéric Guffens and Thomas Deprest and Philippe Demaerel and Anna L. David and Andrew Melbourne and Sébastien Ourselin and Jan Deprest and Tom Vercauteren},
  title = {{Label-Set Loss Functions for Partial Supervision: Application to Fetal Brain 3D MRI Parcellation}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  url = {https://openreview.net/forum?id=wXIOitNKw4},
}
Figure