Compositional Representation Learning for Brain Tumour Segmentation

Xiao Liu, Antanas Kascenas, Hannah Watson, Sotirios A. Tsaftaris, Alison Q. O'Neil

Introduction

Deep learning models have demonstrated remarkable success in medical image segmentation, often reaching or exceeding human-expert performance. However, these models are notoriously "data-hungry," requiring vast quantities of pixel-level annotations to achieve high accuracy. In the context of neuro-oncology, generating these labels is a labor-intensive process, requiring radiologists to manually delineate tumour boundaries across hundreds of MRI slices. This "annotation bottleneck" remains a significant barrier to the deployment of robust AI tools in clinical settings.

This paper introduces vMFBrain, a framework designed to address the low-annotated data regime. By leveraging compositional representation learning, vMFBrain mimics the human ability to recognize anatomical structures as configurations of distinct parts. The framework utilizes a mix of limited full labels, unlabeled data, and easily obtainable "weak" supervision to learn robust representations of both pathological and healthy brain structures.

Methodology: The vMFBrain Framework

The authors adapt the vMFNet framework specifically for neuroimaging tasks. The architecture is composed of three primary components:

  1. Feature Extractor ($F_\psi$): A standard 2D UNet backbone serves as the foundation, extracting high-dimensional feature maps from multi-modal MRI scans (T1, T1Gd, T2, and FLAIR).
  2. Compositional Layer: This module represents the core innovation. Extracted features are modeled using von-Mises-Fisher (vMF) distributions. The model learns a set of "vMF kernels" that function as cluster centers in the feature space. Each kernel is encouraged to represent a specific visual component, such as tumour tissue, cerebrospinal fluid (CSF), or grey/white matter.
  3. Task and Weak Supervision Networks: The task network ($T_\theta$) generates the final segmentation masks. Simultaneously, a weak supervision network ($W_\omega$) is trained to predict the presence or absence of a tumour within a specific slice.

Mixed-Supervision Training

The training objective is a multi-faceted loss function designed to maximize information gain from disparate data sources. It incorporates a Clustering Loss to organize the feature space by grouping similar anatomical features around vMF kernels, a standard Dice Loss applied to the small fraction of available pixel-level annotations, and a Weak Supervision Loss utilizing slice-level labels. These weak labels are significantly faster to collect than full segmentations, often derived from simple 2-point annotations indicating the vertical extent of a tumour.

Experimental Setup and Datasets

The framework was evaluated using the BraTS 2021 (Brain Tumour Segmentation) challenge dataset, which includes MRI volumes from 1,251 patients. The dataset provides annotations for three distinct tumour sub-regions: peritumoural edema (ED), GD-enhancing tumour (ET), and necrotic/non-enhancing tumour (NE).

To test the model's robustness in data-scarce environments, the authors simulated scenarios where only 0.1%, 0.5%, and 1% of the total pixel-level labels were available for training.

Results and Performance

The experimental results demonstrate that vMFBrain is exceptionally efficient in low-data regimes:

  • High Efficiency: With only 1% of pixel-level labels, vMFBrain achieved a Dice score of 89.04%. This performance is remarkably close to a standard UNet trained on 100% of the labels (90.84%), representing a 100x reduction in required annotation effort for comparable results.
  • Robustness at the Extreme: In the 0.1% label regime, vMFBrain maintained a Dice score of 85.64%, significantly outperforming the baseline UNet (80.66%) and other established semi-supervised methods like SDNet (79.20%).
  • Sub-region Granularity: The model also showed improvements in segmenting specific tumour sub-regions when provided with type-specific weak labels, although this specialized focus slightly reduced its attention to general anatomy.

Key Insights

Emergent Anatomy Learning

One of the most significant findings is the "emergent" learning of healthy anatomy. Even when the model was only explicitly supervised to find tumours, the internal vMF kernels naturally organized themselves to represent healthy structures like CSF and white/grey matter. This suggests that the compositional constraint forces the model to understand the entire image context to better isolate the pathology.

The Power of Parsimony

The authors discovered that simplifying the architecture—specifically by removing the image reconstruction sub-task present in the original vMFNet—actually improved performance for brain tumours. This indicates that for complex and heterogeneous structures like tumours, focusing on discriminative features rather than full image reconstruction prevents the model from being distracted by irrelevant noise.

Interpretability by Design

By design, the vMF kernels provide a degree of transparency. Because each kernel corresponds to a visual component, researchers can visualize which parts of the image activate specific kernels. This offers insight into the model's decision-making process and ensures it is focusing on relevant anatomical features rather than background artifacts.

Limitations and Future Directions

While vMFBrain represents a significant step forward, several areas for improvement remain:

  • Dimensionality: The current implementation operates on 2D slices. Extending the framework to 3D would allow for better capture of volumetric spatial context, likely improving performance further.
  • Hyperparameter Optimization: The number of vMF kernels and distribution variances currently require empirical tuning. Automating this process would make the framework more adaptable to diverse medical imaging tasks.
  • Supervision Balance: There is a delicate trade-off between task-specific performance and general representation learning. Future work will focus on balancing these objectives to maintain high diagnostic accuracy without losing the benefits of general anatomical understanding.

Conclusion

vMFBrain demonstrates that compositional representation learning, combined with mixed supervision, can dramatically reduce the reliance on expensive pixel-level annotations in medical imaging. By achieving near-state-of-the-art performance with only 1% of labeled data, this approach paves the way for more scalable and efficient AI development in healthcare.

Summary updated

BibTeX
@inproceedings{liu2023compositional,
  author = {Xiao Liu and Antanas Kascenas and Hannah Watson and Sotirios A. Tsaftaris and Alison Q. O'Neil},
  title = {{Compositional Representation Learning for Brain Tumour Segmentation}},
  booktitle = {DART @ MICCAI},
  year = {2023},
  doi = {10.1007/978-3-031-45857-6\_5},
}
Figure