Deep learning for automatic segmentation of vestibular schwannoma: a retrospective study from multi-center routine MRI

Aaron Kujawa, Reuben Dorent, Steve Connor, Suki Thomson, Marina Ivory, Ali Vahedi, Emily Guilhem, Navodini Wijethilake, Robert Bradford, Neil Kitchen, Sotirios Bisdas, Sébastien Ourselin, Tom Vercauteren, Jonathan ShapeyFrontiers Comput. Neurosci 2024 · DOI · OpenReview

Introduction

Vestibular Schwannoma (VS) is a benign tumor of the balance nerve, the management of which necessitates precise volumetric measurements for growth monitoring and treatment planning. While deep learning (DL) has shown significant promise in segmenting these tumors, existing state-of-the-art models have primarily been trained on highly standardized datasets, such as those acquired for Gamma Knife (GK) stereotactic radiosurgery planning.

In routine clinical practice, MRI scans vary significantly across scanner manufacturers, field strengths (1.0T to 3.0T), and pulse sequences. Furthermore, post-operative cases—often excluded from research—present unique challenges due to surgical scars and anatomical alterations. This study addresses the "generalization gap," where models trained on uniform data fail when confronted with the diversity of routine clinical practice.

Methods

The researchers utilized the nnU-Net (v2) framework, a self-configuring deep learning pipeline that serves as a benchmark for medical image segmentation. A critical component of the methodology was the data curation and annotation pipeline. To establish high-quality "ground truth" labels across a diverse dataset, a three-stage iterative process was implemented:

  1. Initial segmentations were generated by a specialized company.
  2. These were reviewed and corrected by one of three trained radiologists.
  3. Final validation was performed by an expert team comprising consultant neuroradiologists and a neurosurgeon.
The three-stage iterative annotation pipeline ensures high-quality ground truth labels for diverse clinical data.

Models were trained for three input scenarios: contrast-enhanced T1-weighted (ceT1w) images, T2-weighted (T2w) images, and a multi-modal combination.

Datasets

The study leveraged three primary datasets to evaluate performance and generalization:

  • MC-RC (Multi-Center Routine Clinical): A newly released dataset of 160 patients (427 exams) from 10 UK medical sites, including longitudinal and post-operative cases.
  • SC-GK (Single-Center Gamma Knife): A public dataset of 484 image pairs acquired using a rigid, standardized protocol.
  • T-SC-GK (Tilburg Single-Center): An independent testing set from the Netherlands used to assess generalization to entirely unseen centers.
Comparison of image characteristics (slice thickness, voxel volume, and intensity) between standardized (SC-GK) and routine clinical (MC-RC) datasets, highlighting the increased variability in real-world data.

Key Insights and Results

The study revealed that standardization is a double-edged sword. Models trained exclusively on the "perfect" SC-GK dataset saw their performance (Dice Similarity Coefficient, DSC) plummet from ~95% to as low as ~16% when applied to routine clinical data. In contrast, models trained on the diverse MC-RC dataset maintained high accuracy on standardized data, demonstrating that training on "noisy" and variable data fosters robustness.

On the MC-RC test set, the models achieved median DSCs of 86.2% for ceT1w and 89.4% for T2w. The best overall performance was achieved by a model trained on a combination of both datasets (MC-RC + SC-GK), reaching a DSC of 93.6% on the independent Tilburg dataset. Notably, the model's performance was comparable to the inter-observer variation between human experts.

Discussion and Limitations

While the results are promising, challenges remain. Detecting very small post-operative residuals (<30 mm³) is difficult, particularly on T2w scans. Additionally, the current study excluded patients with Neurofibromatosis type 2 (NF2), whose complex bilateral tumors require further investigation. Future work will focus on integrating these cases and evaluating the models' utility in real-time multidisciplinary team (MDT) meetings to reduce clinician workload.

Summary updated

Abstract

p>Automatic segmentation of vestibular schwannoma (VS) from routine clinical MRI has potential to improve clinical workflow, facilitate treatment decisions, and assist patient management. Previous work demonstrated reliable automatic segmentation performance on datasets of standardized MRI images acquired for stereotactic surgery planning. However, diagnostic clinical datasets are generally more diverse and pose a larger challenge to automatic segmentation algorithms, especially when post-operative images are included. In this work, we show for the first time that automatic segmentation of VS on routine MRI datasets is also possible with high accuracy. We acquired and publicly release a curated multi-center routine clinical (MC-RC) dataset of 160 patients with a single sporadic VS. For each patient up to three longitudinal MRI exams with contrast-enhanced T1-weighted (ceT1w) (<italic>n</italic> = 124) and T2-weighted (T2w) (<italic>n</italic> = 363) images were included and the VS manually annotated. Segmentations were produced and verified in an iterative process: (1) initial segmentations by a specialized company; (2) review by one of three trained radiologists; and (3) validation by an expert team. Inter- and intra-observer reliability experiments were performed on a subset of the dataset. A state-of-the-art deep learning framework was used to train segmentation models for VS. Model performance was evaluated on a MC-RC hold-out testing set, another public VS datasets, and a partially public dataset. The generalizability and robustness of the VS deep learning segmentation models increased significantly when trained on the MC-RC dataset. Dice similarity coefficients (DSC) achieved by our model are comparable to those achieved by trained radiologists in the inter-observer experiment. On the MC-RC testing set, median DSCs were 86.2(9.5) for ceT1w, 89.4(7.0) for T2w, and 86.4(8.6) for combined ceT1w+T2w input images. On another public dataset acquired for Gamma Knife stereotactic radiosurgery our model achieved median DSCs of 95.3(2.9), 92.8(3.8), and 95.5(3.3), respectively. In contrast, models trained on the Gamma Knife dataset did not generalize well as illustrated by significant underperformance on the MC-RC routine MRI dataset, highlighting the importance of data variability in the development of robust VS segmentation models. The MC-RC dataset and all trained deep learning models were made available online.</p>
BibTeX
@article{kujawa2024deep,
  author = {Aaron Kujawa and Reuben Dorent and Steve Connor and Suki Thomson and Marina Ivory and Ali Vahedi and Emily Guilhem and Navodini Wijethilake and Robert Bradford and Neil Kitchen and Sotirios Bisdas and Sébastien Ourselin and Tom Vercauteren and Jonathan Shapey},
  title = {{Deep learning for automatic segmentation of vestibular schwannoma: a retrospective study from multi-center routine MRI}},
  journal = {Frontiers Comput. Neurosci.},
  year = {2024},
  doi = {10.3389/fncom.2024.1365727},
}
Figure