Handling missing MRI sequences in deep learning segmentation of brain metastases: a multicenter study

Endre Grøvik, Darvin Yi, Michael Iv, Elizabeth Tong, Line Brennhaug Nilsen, Anna Latysheva, Cathrine Saxhaug, Kari Dolven Jacobsen, Åslaug Helland, Kyrre Eeg Emblem, Daniel L. Rubin, Greg ZaharchukNpj Digit. Medicine 2021 · OpenReview

Introduction

The accurate detection and segmentation of brain metastases are critical for treatment planning, particularly in stereotactic radiosurgery. While deep learning (DL) models have shown great promise in automating this task, they often rely on a fixed set of multi-sequence MRI inputs. In clinical practice, however, imaging protocols vary across institutions, and patients may miss specific sequences due to time constraints or safety concerns. This "brittleness" of standard DL models—requiring a complete set of input channels—presents a significant barrier to widespread clinical deployment.

Representative MRI slice showing a brain metastasis, the target for automated segmentation.

Methodology: Input-Level Dropout (ILD)

To address this challenge, researchers developed a robust segmentation tool based on the DeepLab V3 architecture. The core innovation is the Input-Level Dropout (ILD) layer. Unlike standard dropout, which randomly zeros out individual neurons, the ILD layer stochastically "drops out" entire MRI sequences during the training phase.

By training the network on every possible subset of the four input sequences, the model learns to identify metastases regardless of which specific sequences are available at inference time. The four sequences utilized in the training set included:

  1. Pre-gadolinium (Gd) T1-weighted 3D fast spin echo
  2. Post-Gd T1-weighted 3D fast spin echo (CUBE/SPACE)
  3. Post-Gd T1-weighted inversion recovery (IR) prepped (BRAVO/MPRAGE)
  4. 3D fluid-attenuated inversion recovery (FLAIR)

Multicenter Evaluation

The study employed a retrospective dataset of 165 patients. The model was trained on data from 100 patients at Hospital A and tested on 55 patients from Hospital B. Crucially, the test set from Hospital B lacked the IR-prepped sequence (BRAVO). This setup directly tested the model's ability to generalize to a different clinical protocol with missing data, comparing its performance against a standard DeepLab V3 model trained only on the three available sequences.

Key Results

The ILD model demonstrated superior performance and robustness compared to the state-of-the-art baseline:

  • Segmentation Quality: The ILD model achieved a significantly higher Dice score (0.795 vs. 0.774, p=0.017) and IoU-score (0.561 vs. 0.492, p<0.001).
  • False Positive Reduction: For lesions larger than 10 mm³, the ILD model nearly halved the average false positive rate, reducing it from 7.0 to 3.6 per patient (p<0.001).
  • Detection Accuracy: Both models maintained a high Area Under the Curve (AUC) of 0.989.
An axial MRI slice of the lower brain region. The ILD model significantly reduced false positives in these complex anatomical areas compared to standard architectures.

Qualitatively, the ILD strategy acted as a powerful regularizer. By forcing the network to learn from incomplete information, it became less dependent on specific sequences and better at distinguishing metastases from vascular structures, particularly in the lower brain regions where false positives are common.

Discussion and Limitations

The study highlights the potential for ILD-trained models to be deployed "out-of-the-box" across different medical centers without site-specific fine-tuning. However, some limitations remain. Most false positives still occur near vascular structures that mimic enhancing metastases. Additionally, the test cohort consisted of patients scheduled for radiosurgery, who typically have larger lesions; further validation is needed in more diverse clinical populations. Future work may involve incorporating additional sequences, such as diffusion-weighted imaging (DWI), to further refine detection of small lesions.

Summary updated

Abstract

The purpose of this study was to assess the clinical value of a deep learning (DL) model for automatic detection and segmentation of brain metastases, in which a neural network is trained on four distinct MRI sequences using an input-level dropout layer, thus simulating the scenario of missing MRI sequences by training on the full set and all possible subsets of the input data. This retrospective, multicenter study, evaluated 165 patients with brain metastases. The proposed input-level dropout (ILD) model was trained on multisequence MRI from 100 patients and validated/tested on 10/55 patients, in which the test set was missing one of the four MRI sequences used for training. The segmentation results were compared with the performance of a state-of-the-art DeepLab V3 model. The MR sequences in the training set included pre-gadolinium and post-gadolinium (Gd) T1-weighted 3D fast spin echo, post-Gd T1-weighted inversion recovery (IR) prepped fast spoiled gradient echo, and 3D fluid attenuated inversion recovery (FLAIR), whereas the test set did not include the IR prepped image-series. The ground truth segmentations were established by experienced neuroradiologists. The results were evaluated using precision, recall, Intersection over union (IoU)-score and Dice score, and receiver operating characteristics (ROC) curve statistics, while the Wilcoxon rank sum test was used to compare the performance of the two neural networks. The area under the ROC curve (AUC), averaged across all test cases, was 0.989 ± 0.029 for the ILD-model and 0.989 ± 0.023 for the DeepLab V3 model (p = 0.62). The ILD-model showed a significantly higher Dice score (0.795 ± 0.104 vs. 0.774 ± 0.104, p = 0.017), and IoU-score (0.561 ± 0.225 vs. 0.492 ± 0.186, p &lt; 0.001) compared to the DeepLab V3 model, and a significantly lower average false positive rate of 3.6/patient vs. 7.0/patient (p &lt; 0.001) using a 10 mm3 lesion-size limit. The ILD-model, trained on all possible combinations of four MRI sequences, may facilitate accurate detection and segmentation of brain metastases on a multicenter basis, even when the test cohort is missing input MRI sequences.
BibTeX
@inproceedings{grvik2021handling,
  author = {Endre Grøvik and Darvin Yi and Michael Iv and Elizabeth Tong and Line Brennhaug Nilsen and Anna Latysheva and Cathrine Saxhaug and Kari Dolven Jacobsen and Åslaug Helland and Kyrre Eeg Emblem and Daniel L. Rubin and Greg Zaharchuk},
  title = {{Handling missing MRI sequences in deep learning segmentation of brain metastases: a multicenter study}},
  booktitle = {Npj Digit. Medicine},
  year = {2021},
  url = {https://openreview.net/forum?id=vLI8aCklBG},
}
Figure