Deep Learning for Longitudinal Gross Tumor Volume Segmentation in MRI-Guided Adaptive Radiotherapy for Head and Neck Cancer

Xin Tie, Weijie Chen, Zachary Huemann, Brayden Schott, N. Liu, Tyler BradshawarXiv 2024 · DOI · arXiv

TL;DR

This paper presents deep learning models for automatically segmenting gross tumor volumes (GTVs) in head and neck cancer patients undergoing MRI-guided adaptive radiotherapy. By leveraging longitudinal MRI data and introducing novel mask-aware attention mechanisms, their approach significantly improves segmentation accuracy, outperforming baseline methods and achieving first place in a challenge.

What Problem Does This Paper Address?

Accurate segmentation of the Gross Tumor Volume (GTV) is crucial for effective radiation therapy, especially in head and neck cancer (HNC) where tumors can change significantly during treatment. MRI-guided adaptive radiotherapy (MRgART) offers superior soft-tissue contrast for this purpose. However, manual GTV segmentation on pre- and mid-treatment MRI scans is a time-consuming and error-prone process, subject to inter-observer variability. This variability can lead to suboptimal treatment plans. The paper addresses the need for automated, accurate, and consistent GTV segmentation across different stages of treatment to streamline MRgART workflows.

Methods

The core of the proposed solution is the SegResNet architecture, a convolutional encoder-decoder model known for its strong performance in medical image segmentation. For both pre-radiotherapy (pre-RT) and mid-radiotherapy (mid-RT) segmentation tasks, SegResNet with deep supervision was employed.

For Task 1 (pre-RT segmentation), the model was trained using a combined dataset of pre-RT and mid-RT MRI data, which surprisingly improved performance compared to using only pre-RT data.

For Task 2 (mid-RT segmentation), a novel mask-aware attention module was introduced. This module allows the model to leverage information from pre-RT GTV masks to guide the segmentation of mid-RT GTVs. It works by first applying channel attention to emphasize important features and then spatial attention, which incorporates the pre-RT GTV masks to refine the features. This approach helps the mid-RT model focus on relevant regions informed by the earlier tumor state. The modified architecture is referred to as MA-SegResNet.

An ensemble of 10 models was used for final predictions to further enhance robustness.

Datasets

The study utilized a retrospective dataset of 150 HNC patients. Each patient provided:

  • Pre-radiotherapy (pre-RT) T2-weighted MRI scans and corresponding GTV labels (primary GTV - GTVp, and metastatic lymph nodes - GTVn).
  • Mid-radiotherapy (mid-RT) T2-weighted MRI scans and corresponding GTV labels.

For Task 2, pre-RT data was deformably registered to mid-RT MRI for spatial alignment. The data was provided in NIfTI format. The dataset was split into five folds for cross-validation, with four folds used for training and one for validation.

Key Insights and Ideas

  • Longitudinal Data Synergy: Surprisingly, using both pre-RT and mid-RT MRI data for training the pre-RT segmentation model (Task 1) yielded better results than using only pre-RT data. This suggests that information from later stages can inform earlier segmentation.
  • Mask-Aware Attention for Temporal Context: The introduction of mask-aware attention in Task 2 effectively leverages the spatial information from pre-RT GTV masks to guide mid-RT GTV segmentation. This is a crucial step for handling tumor changes over time.
  • Ensemble Power: Combining predictions from multiple trained models (an ensemble of 10) significantly boosted performance, highlighting the benefit of model diversity.
  • GTVp vs. GTVn Performance Discrepancy: The models consistently performed better on metastatic lymph nodes (GTVn) than on the primary tumor (GTVp). This is likely due to the more defined and consistent boundaries of lymph nodes compared to the often irregular and infiltrative nature of primary tumors.

Key Results

  • Task 1 (pre-RT segmentation): The ensemble of 10 models achieved an average Dice Similarity Coefficient (DSCagg) of 0.794 on the hold-out test set, with 0.745 for GTVp and 0.844 for GTVn.
  • Task 2 (mid-RT segmentation): The ensemble of 10 MA-SegResNet models achieved an average DSCagg of 0.733 on the hold-out test set, with 0.607 for GTVp and 0.859 for GTVn. This performance secured 1st place in the HNTS-MRG 2024 challenge.
  • Ablation Study (Task 1): Including paired mid-RT data in training improved DSCagg from 0.776 (pre-RT only) to 0.799 (mid-RT & pre-RT), a statistically significant improvement.
  • Ablation Study (Task 2): Integrating pre-RT GTV masks as input improved DSCagg from 0.588 (mid-RT only) to 0.704. Further improvements were seen with the MA-SegResNet architecture and MPDR post-processing, reaching 0.725 and 0.733 respectively.

Limitations and Future Work

The authors acknowledge several limitations:

  • Lack of Co-registered Pre-RT MRI: Co-registered pre-RT MRI images were not used as an additional input channel for mid-RT GTV segmentation, which could potentially further improve results.
  • Generalizability of Attention: The mask-aware attention modules were only tested with the SegResNet backbone. Their effectiveness with other architectures remains to be explored.
  • Clinical Applicability Uncertainty: While the models show strong performance, their direct impact on clinical efficiency and patient outcomes is yet to be rigorously evaluated through prospective studies.

Future work could focus on incorporating co-registered pre-RT MRI, exploring attention mechanisms with diverse backbones, and conducting clinical validation studies to assess the real-world impact of these automated segmentation tools.

Summary updated

Abstract

Accurate segmentation of gross tumor volume (GTV) is essential for effective MRI-guided adaptive radiotherapy (MRgART) in head and neck cancer. However, manual segmentation of the GTV over the course of therapy is time-consuming and prone to interobserver variability. Deep learning (DL) has the potential to overcome these challenges by automatically delineating GTVs. In this study, our team, $\textit{UW LAIR}$, tackled the challenges of both pre-radiotherapy (pre-RT) (Task 1) and mid-radiotherapy (mid-RT) (Task 2) tumor volume segmentation. To this end, we developed a series of DL models for longitudinal GTV segmentation. The backbone of our models for both tasks was SegResNet with deep supervision. For Task 1, we trained the model using a combined dataset of pre-RT and mid-RT MRI data, which resulted in the improved aggregated Dice similarity coefficient (DSCagg) on an internal testing set compared to models trained solely on pre-RT MRI data. In Task 2, we introduced mask-aware attention modules, enabling pre-RT GTV masks to influence intermediate features learned from mid-RT data. This attention-based approach yielded slight improvements over the baseline method, which concatenated mid-RT MRI with pre-RT GTV masks as input. In the final testing phase, the ensemble of 10 pre-RT segmentation models achieved an average DSCagg of 0.794, with 0.745 for primary GTV (GTVp) and 0.844 for metastatic lymph nodes (GTVn) in Task 1. For Task 2, the ensemble of 10 mid-RT segmentation models attained an average DSCagg of 0.733, with 0.607 for GTVp and 0.859 for GTVn, leading us to $\textbf{achieve 1st place}$. In summary, we presented a collection of DL models that could facilitate GTV segmentation in MRgART, offering the potential to streamline radiation oncology workflows. Our code and model weights are available at https://github.com/xtie97/HNTS-MRG24-UWLAIR.
BibTeX
@misc{tie2024deep,
  author = {Xin Tie and Weijie Chen and Zachary Huemann and Brayden Schott and N. Liu and Tyler Bradshaw},
  title = {{Deep Learning for Longitudinal Gross Tumor Volume Segmentation in MRI-Guided Adaptive Radiotherapy for Head and Neck Cancer}},
  year = {2024},
  eprint = {2412.00663},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.00663},
}
Figure