Towards Longitudinal Glioma Segmentation: Evaluating combined pre- and post-treatment MRI training data for automated tumor segmentation using nnU-Net

Sara Ranjbar, Kyle Singleton, Lee Curtin, Lisa E. Paulson, Kamala Clark-Swanson, Andrea Hawkins‐Daarud, J. Ross Mitchell, Pamela Jackson, Kristin R. SwansonmedRxiv 2023 · DOI

Abstract

Identification of key phenotypic regions such as necrosis, contrast enhancement, and edema on magnetic resonance imaging (MRI) is important for understanding disease evolution and treatment response in patients with glioma. Manual delineation is time intensive and not feasible for a clinical workflow. Automating phenotypic region segmentation overcomes many issues with manual segmentation, however, current glioma segmentation datasets focus on pre-treatment, diagnostic scans, where treatment effects and surgical cavities are not present. Thus, existing automatic segmentation models are not applicable to post-treatment imaging that is used for longitudinal evaluation of care. Here, we present a comparison of three-dimensional convolutional neural networks (nnU-Net architecture) trained on large temporally defined pre-treatment, post-treatment, and mixed cohorts. We demonstrate that training a combined model can be as effective as models trained on just one temporal group. The results highlight the importance of a diverse training set, that includes images from the course of disease and with effects from treatment, in the creation of a model that can accurately segment glioma MRIs at multiple treatment time points.

The Challenge of Post-Treatment MRI

In neuro-oncology, "segmentation"—the process of outlining tumor regions like edema (ED), necrosis (NEC), and enhancing tissue (CE)—is vital for monitoring therapy response. While deep learning has revolutionized research-grade segmentation, a significant gap remains: most public datasets, such as BraTS, focus almost exclusively on pre-treatment diagnostic scans.

Post-treatment MRIs present a "messier" reality. They feature resection cavities, blood degradation products, and inflammation from radiation or chemotherapy. Models trained only on diagnostic scans often mistake these treatment-induced artifacts for active tumor. This study addresses the need for a robust model capable of handling the entire patient journey, from diagnosis through multiple rounds of treatment.

Methods and Data

The researchers employed the nnU-Net architecture, a self-configuring 3D convolutional neural network. To evaluate the impact of temporal data, three models were trained:

  1. Pre Model: Trained only on diagnostic (pre-surgical) scans.
  2. Post Model: Trained only on scans taken after surgery/therapy.
  3. Mixed Model: Trained on a combined dataset of both.

The study utilized a massive dataset of 1,563 imaging timepoints from 854 patients across 13 institutions. Notably, the models used only two MRI sequences—T1Gd (contrast-enhanced) and FLAIR—to reflect clinical scenarios where full four-sequence protocols might be unavailable.

Key Insights: Bridging the Generalization Gap

A model trained only on pre-treatment data performs significantly worse on post-treatment scans, and vice versa. For instance, the "Pre" model often struggles to distinguish between a necrotic tumor core and a surgical resection cavity.

Visual comparison of segmentation performance across Pre, Post, and Mixed models for both pre-treatment and post-treatment cases.

The "Mixed" model offers a distinct advantage: it performs nearly as well as specialized models on their respective domains. This suggests that diagnostic accuracy does not need to be sacrificed to gain post-treatment versatility. Furthermore, the Mixed model showed high longitudinal consistency, making it suitable for tracking a single patient over time.

Quantitative Results

The primary metric used was the Dice Coefficient (measuring overlap between AI prediction and human expert labels).

Dice scores for different tumor regions (NEC, CE, ED, WT) across the three models, evaluated on both pre-treatment and post-treatment data.

Key findings include:

  • Whole Tumor (WT): The Mixed Model achieved a Dice of 0.85 on pre-treatment and 0.80 on post-treatment data. Interestingly, on post-treatment data, the Mixed model outperformed the specialized "Post" model (p=0.038), suggesting that exposure to "clean" diagnostic scans helps the model better interpret "messy" post-treatment scans.
  • Necrotic Core (NEC): This was the most challenging region. The Pre model's performance on post-treatment NEC was poor (Dice 0.28), while the Mixed model improved this to 0.35, matching the specialized Post model.

Limitations and Future Directions

While promising, the study notes that "ground truth" labels were generated by trained technicians rather than senior neuroradiologists, potentially missing expert-level nuances. Additionally, while using only two sequences is practical, adding T2-weighted images might further improve the differentiation between resection cavities and active tumors. The ultimate goal is to integrate these "Mixed" models into clinical "Response Assessment" workflows to provide faster, data-driven treatment decisions.

Summary updated

Abstract

Identification of key phenotypic regions such as necrosis, contrast enhancement, and edema on magnetic resonance imaging (MRI) is important for understanding disease evolution and treatment response in patients with glioma. Manual delineation is time intensive and not feasible for a clinical workflow. Automating phenotypic region segmentation overcomes many issues with manual segmentation, however, current glioma segmentation datasets focus on pre-treatment, diagnostic scans, where treatment effects and surgical cavities are not present. Thus, existing automatic segmentation models are not applicable to post-treatment imaging that is used for longitudinal evaluation of care. Here, we present a comparison of three-dimensional convolutional neural networks (nnU-Net architecture) trained on large temporally defined pre-treatment, post-treatment, and mixed cohorts. We used a total of 1563 imaging timepoints from 854 patients curated from 13 different institutions as well as diverse public data sets to understand the capabilities and limitations of automatic segmentation on glioma images with different phenotypic and treatment appearance. We assessed the performance of models using Dice coefficients on test cases from each group comparing predictions with manual segmentations generated by trained technicians. We demonstrate that training a combined model can be as effective as models trained on just one temporal group. The results highlight the importance of a diverse training set, that includes images from the course of disease and with effects from treatment, in the creation of a model that can accurately segment glioma MRIs at multiple treatment time points.
BibTeX
@misc{ranjbar2023towards,
  author = {Sara Ranjbar and Kyle Singleton and Lee Curtin and Lisa E. Paulson and Kamala Clark-Swanson and Andrea Hawkins‐Daarud and J. Ross Mitchell and Pamela Jackson and Kristin R. Swanson},
  title = {{Towards Longitudinal Glioma Segmentation: Evaluating combined pre- and post-treatment MRI training data for automated tumor segmentation using nnU-Net}},
  year = {2023},
  doi = {10.1101/2023.05.31.23290537},
  howpublished = {medRxiv},
}
Figure