Inherent Brain Segmentation Quality Control from Fully ConvNet Monte Carlo Sampling

Abhijit Guha Roy, Sailesh Conjeti, Nassir Navab, Christian WachingerarXiv 2018 · arXiv

Abstract

Automated whole-brain MRI segmentation is essential for studying neuroanatomical changes, yet its reliability is often compromised by domain shifts and imaging artifacts. This paper introduces a Bayesian Fully Convolutional Neural Network (F-CNN) framework that leverages Monte Carlo (MC) dropout at inference to provide built-in quality control. By generating multiple segmentation samples, the model derives structure-specific uncertainty metrics—most notably the MC Dice agreement ($d^{MC}_s$)—which correlate strongly with actual segmentation accuracy. We demonstrate that using these uncertainty scores to weight clinical group analyses significantly improves the detection of disease-related structural changes in large-scale datasets like ADNI and ABIDE.

Introduction

Whole-brain segmentation is a cornerstone of neuroimaging research, enabling the quantification of structural changes in aging and disease. Despite the success of deep learning models, their performance often degrades when applied to out-of-distribution data, such as scans from different MRI manufacturers or pediatric populations. Traditional quality control (QC) relies on labor-intensive manual inspection or separate "evaluator" models. This work addresses the need for an efficient, inherent uncertainty measure that allows a segmentation model to "know what it doesn't know" without requiring additional ground truth or complex secondary architectures.

Methodology: Bayesian F-CNN and Monte Carlo Sampling

The proposed method utilizes a standard 3D Fully Convolutional Neural Network (F-CNN) modified with dropout layers after each encoder and decoder block. Unlike standard practice where dropout is disabled during testing, this approach maintains dropout during inference. By performing $N$ forward passes for a single MRI volume, the network generates a distribution of $N$ Monte Carlo segmentations.

Overview of the Monte Carlo sampling pipeline and derived uncertainty metrics.

From these samples, three primary structure-wise uncertainty metrics are calculated:

  1. Coefficient of Variation ($CV_s$): Measures the variability in predicted volumes across MC samples.
  2. Mean Uncertainty ($U_s$): The average voxel-wise entropy within the predicted region.
  3. MC Dice Agreement ($d^{MC}_s$): The average overlap between all pairs of MC samples.

The $d^{MC}_s$ metric serves as a proxy for segmentation quality: high agreement across samples indicates high confidence, while low agreement suggests potential errors.

Experimental Results

The model was evaluated on several challenging datasets, including ADNI (elderly/Alzheimer's), CANDI (pediatric), and IBSR (low resolution). Across all datasets, the $d^{MC}_s$ metric demonstrated a remarkably high correlation with the actual Dice scores (ground truth accuracy), with correlation coefficients ranging from $r=0.78$ to $r=0.90$.

Correlation between structure-wise uncertainty metrics and actual Dice scores for the CANDI-13 dataset.

Impact on Clinical Group Analysis

A key application of these uncertainty metrics is in clinical group analysis. By using structure-wise uncertainty scores as weights in linear regression, the influence of poor-quality segmentations is minimized. In the ADNI dataset, uncertainty-weighted regression yielded effect sizes for hippocampal atrophy that were closer to manual gold standards than standard regression. Furthermore, in the large-scale ABIDE dataset, this method successfully identified significant structural differences in the basal ganglia of autistic subjects—findings that were missed by standard regression but are consistent with large-scale independent studies.

Conclusion

This work demonstrates that Monte Carlo dropout provides a computationally efficient and reliable mechanism for automated quality control in brain MRI segmentation. By deriving uncertainty directly from the model's own variance, researchers can better handle large, uncurated datasets and improve the statistical power of clinical studies. This framework represents a significant step toward the robust translation of deep learning tools into clinical practice.

Summary updated

Abstract

We introduce inherent measures for effective quality control of brain segmentation based on a Bayesian fully convolutional neural network, using model uncertainty. Monte Carlo samples from the posterior distribution are efficiently generated using dropout at test time. Based on these samples, we introduce next to a voxel-wise uncertainty map also three metrics for structure-wise uncertainty. We then incorporate these structure-wise uncertainty in group analyses as a measure of confidence in the observation. Our results show that the metrics are highly correlated to segmentation accuracy and therefore present an inherent measure of segmentation quality. Furthermore, group analysis with uncertainty results in effect sizes closer to that of manual annotations. The introduced uncertainty metrics can not only be very useful in translation to clinical practice but also provide automated quality control and group analyses in processing large data repositories.
BibTeX
@misc{roy2018inherent,
  author = {Abhijit Guha Roy and Sailesh Conjeti and Nassir Navab and Christian Wachinger},
  title = {{Inherent Brain Segmentation Quality Control from Fully ConvNet Monte Carlo Sampling}},
  year = {2018},
  eprint = {1804.07046},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1804.07046},
}
Figure