Deep Learning for Automatic Segmentation of Vestibular Schwannoma: A Retrospective Study from Multi-Centre Routine MRI

Aaron Kujawa, Reuben Dorent, Steve Connor, Suki Thomson, Marina Ivory, Ali Vahedi, Emily Guilhem, Navodini Wijethilake, Robert Bradford, Neil Kitchen, Sotirios Bisdas, Sébastien Ourselin, Tom Vercauteren, Jonathan ShapeymedRxiv 2022 · DOI

Abstract

Automatic segmentation of vestibular schwannoma (VS) from routine clinical MRI has potential to improve clinical workflow, facilitate treatment decisions, and assist patient management. Previous work demonstrated reliable automatic segmentation performance on datasets of standardised MRI images acquired for stereotactic surgery planning. However, diagnostic clinical datasets are generally more diverse and pose a larger challenge to automatic segmentation algorithms, especially when post-operative images are included. In this work, we show for the first time that automatic segmentation of VS on routine MRI datasets is also possible with high accuracy. We acquired and publicly release a curated multi-centre routine clinical (MC-RC) dataset of 160 patients with a single sporadic VS. For each patient up to three longitudinal MRI exams with contrast-enhanced T1-weighted (ceT1w) (n=124) and T2-weighted (T2w) (n=363) images were included and the VS manually annotated. Segmentations were produced and verified in an iterative process: 1) initial segmentations by a specialized company; 2) review by one of three trained radiologists; and 3) validation by an expert team. Inter- and intra-observer reliability experiments were performed on a subset of the dataset. A state-of-the-art deep learning framework was used to train segmentation models for VS. Model performance was evaluated on a MC-RC hold-out testing set, another public VS datasets, and a partially public dataset. The generalizability and robustness of the VS deep learning segmentation models increased significantly when trained on the MC-RC dataset. Dice similarity coefficients (DSC) achieved by our model are comparable to those achieved by trained radiologists in the inter-observer experiment. On the MC-RC testing set, median DSCs were 86.2(9.5) for ceT1w, 89.4(7.0) for T2w and 86.4(8.6) for combined ceT1w+T2w input images. On another public dataset acquired for Gamma Knife stereotactic radiosurgery our model achieved median DSCs of 95.3(2.9), 92.8(3.8), and 95.5(3.3), respectively. In contrast, models trained on the Gamma Knife dataset did not generalise well as illustrated by significant underperformance on the MC-RC routine MRI dataset, highlighting the importance of data variability in the development of robust VS segmentation models. The MC-RC dataset and all trained deep learning models were made available online.

Introduction

Vestibular Schwannoma (VS) is a benign tumor of the vestibulocochlear nerve. Managing these tumors requires precise volumetric measurements to track growth over time, particularly for patients under "watchful waiting" or those undergoing post-operative monitoring. While deep learning has shown promise in medical imaging, most existing VS segmentation models are trained on highly standardized datasets, such as those acquired for Gamma Knife (GK) radiosurgery planning. These images are typically high-resolution and uniform.

In routine clinical practice, however, imaging protocols vary significantly across medical centers, utilizing different MRI manufacturers (Siemens, Philips, GE), field strengths (1.5T vs 3.0T), and resolutions. This "domain shift" often causes models trained on standardized data to fail when applied to "messy" routine clinical data. This study addresses this gap by developing a robust segmentation model trained on a diverse, multi-center routine clinical dataset.

Methods

The researchers utilized the nnU-Net framework, a self-configuring deep learning pipeline based on the 3D U-Net architecture. nnU-Net automatically adapts hyperparameters like patch size and batch size to the specific characteristics of the provided dataset.

To ensure high-quality ground truth for training, a rigorous three-stage curation and annotation pipeline was implemented. Initial segmentations were generated by a specialized medical imaging company, followed by review and editing by trained radiologists. Final validation for difficult or ambiguous cases was performed by a senior expert team consisting of neuroradiologists and a neurosurgeon.

The iterative curation and annotation pipeline used to generate high-quality ground truth segmentations.

Datasets

The study compared two primary datasets to evaluate model generalizability:

  1. MC-RC (Multi-Centre Routine Clinical): A new, curated dataset of 160 patients from 10 different medical sites, including longitudinal scans and both contrast-enhanced T1-weighted (ceT1w) and T2-weighted (T2w) images. This dataset captures the inherent variability of real-world clinical practice.
  2. SC-GK (Single-Centre Gamma Knife): A publicly available dataset of 242 patients scanned using a standardized protocol on a single machine.
Comparison of slice thickness, voxel volume, and intensity distributions between the standardized SC-GK and the diverse MC-RC datasets.

Key Results and Discussion

The primary performance metric was the Dice Similarity Coefficient (DSC). The study revealed several critical insights regarding model robustness:

  • The Generalization Gap: Models trained only on the "perfect" SC-GK data failed significantly when applied to routine clinical scans, with DSC dropping to as low as 48.5% for T2w images. Conversely, models trained on the diverse MC-RC dataset performed excellently on both clinical data and the clean GK data (achieving a DSC of 91.0% on the latter).
  • Clinical Performance: On the MC-RC testing set, the model achieved median DSCs of 86.2 for ceT1w and 89.4 for T2w images. These results are comparable to inter-observer reliability between human radiologists, suggesting the tool is ready for clinical assistance.
  • Modality Flexibility: While performance was slightly higher on ceT1w images, the high accuracy on T2w images is vital for patients who cannot receive gadolinium contrast agents.

Limitations

Despite its robustness, the model faced challenges in specific scenarios. It occasionally struggled with large cystic tumors (missing fluid-filled components) and post-operative cases where surgical fat grafts or scar tissue could be mistaken for residual tumor. Additionally, the study focused on sporadic tumors, excluding patients with Neurofibromatosis type 2 (NF2), who present with more complex multi-tumor anatomy.

Conclusion

This work demonstrates that training on diverse, multi-center routine clinical data is essential for developing robust medical AI. By moving beyond standardized benchmarks, the researchers have provided a tool capable of handling the variability of everyday hospital practice, facilitating more efficient and accurate tumor volumetry in clinical workflows.

Summary updated

Abstract

ABSTRACT Automatic segmentation of vestibular schwannoma (VS) from routine clinical MRI has potential to improve clinical workflow, facilitate treatment decisions, and assist patient management. Previous work demonstrated reliable automatic segmentation performance on datasets of standardised MRI images acquired for stereotactic surgery planning. However, diagnostic clinical datasets are generally more diverse and pose a larger challenge to automatic segmentation algorithms, especially when post-operative images are included. In this work, we show for the first time that automatic segmentation of VS on routine MRI datasets is also possible with high accuracy. We acquired and publicly release a curated multi-centre routine clinical (MC-RC) dataset of 160 patients with a single sporadic VS. For each patient up to three longitudinal MRI exams with contrast-enhanced T1-weighted (ceT1w) (n=124) and T2-weighted (T2w) (n=363) images were included and the VS manually annotated. Segmentations were produced and verified in an iterative process: 1) initial segmentations by a specialized company; 2) review by one of three trained radiologists; and 3) validation by an expert team. Inter- and intra-observer reliability experiments were performed on a subset of the dataset. A state-of-the-art deep learning framework was used to train segmentation models for VS. Model performance was evaluated on a MC-RC hold-out testing set, another public VS datasets, and a partially public dataset. The generalizability and robustness of the VS deep learning segmentation models increased significantly when trained on the MC-RC dataset. Dice similarity coefficients (DSC) achieved by our model are comparable to those achieved by trained radiologists in the inter-observer experiment. On the MC-RC testing set, median DSCs were 86.2(9.5) for ceT1w, 89.4(7.0) for T2w and 86.4(8.6) for combined ceT1w+T2w input images. On another public dataset acquired for Gamma Knife stereotactic radiosurgery our model achieved median DSCs of 95.3(2.9), 92.8(3.8), and 95.5(3.3), respectively. In contrast, models trained on the Gamma Knife dataset did not generalise well as illustrated by significant underperformance on the MC-RC routine MRI dataset, highlighting the importance of data variability in the development of robust VS segmentation models. The MC-RC dataset and all trained deep learning models were made available online.
BibTeX
@misc{kujawa2022deep,
  author = {Aaron Kujawa and Reuben Dorent and Steve Connor and Suki Thomson and Marina Ivory and Ali Vahedi and Emily Guilhem and Navodini Wijethilake and Robert Bradford and Neil Kitchen and Sotirios Bisdas and Sébastien Ourselin and Tom Vercauteren and Jonathan Shapey},
  title = {{Deep Learning for Automatic Segmentation of Vestibular Schwannoma: A Retrospective Study from Multi-Centre Routine MRI}},
  year = {2022},
  doi = {10.1101/2022.08.01.22278193},
  howpublished = {medRxiv},
}
Figure