A transformer-based multi-task deep learning model for simultaneous T-stage identification and segmentation of nasopharyngeal carcinoma

Kaifan Yang, Xiuyu Dong, Fan Tang, Feng Ye, Bei Chen, Shujun Liang, Yu Zhang, Yikai XuFrontiers in Oncology 2024 · DOI

TL;DR

Researchers have developed a multi-task deep learning model that simultaneously segments tumor volumes and identifies the "T-stage" (tumor extent) of nasopharyngeal carcinoma (NPC) from MRI scans. By using a hybrid architecture that combines a 3D U-Net with a Transformer-based prediction network, the model leverages the natural synergy between tumor shape and clinical staging to outperform traditional single-task models.

What Problem Does This Paper Address?

Nasopharyngeal carcinoma (NPC) is a malignant tumor located in a complex anatomical region near critical neural and vascular structures. For effective radiation therapy, clinicians must perform two grueling tasks:

  1. Segmentation: Delineating the Gross Tumor Volume (GTV) to ensure radiation hits the cancer while sparing healthy tissue.
  2. T-staging: Determining the severity and extent of the tumor (T1–T4), which dictates the overall treatment strategy.

Currently, these tasks are performed manually, which is time-consuming and prone to inter-observer variability. While previous AI research has tackled these tasks individually, this paper addresses the "silo" problem: segmentation and staging are inherently related, yet most models ignore this correlation. The authors aim to bridge this gap with a unified model that performs both tasks better by sharing information between them.

Methods

The authors propose a transformer-based multi-task deep learning model. The architecture consists of three main components:

  • 3D U-Net for Segmentation: A modified U-shaped network that extracts spatial features to create a 3D map of the tumor.
  • Transformer-style Prediction Network: Instead of a standard CNN for classification, the authors used a "Bottleneck Transformer." This architecture replaces the final convolutional blocks of a ResNet with Multi-Head Self-Attention (MHSA) layers, which are better at capturing long-range dependencies and global context—crucial for understanding how a tumor relates to surrounding anatomy.
  • Feature-Fusion-Aware (FFA) Module: This is the "connective tissue" of the model. It uses attention mechanisms to fuse features from the segmentation task into the staging task, ensuring that the staging prediction is informed by the precise shape and location of the segmented tumor.

The model was trained using a hybrid loss function combining Jaccard loss and Focal loss for segmentation, and cross-entropy loss for staging.

Datasets

The study utilized two primary datasets consisting of contrast-enhanced T1-weighted images (CE-T1WI):

  • Dataset 1 (Internal): 320 NPC patients collected between 2017 and 2020. This was used for three-fold cross-validation.
  • Dataset 2 (Independent Test Set): 150 patients used for external validation to ensure the model generalizes to new data.
  • Imaging Protocols: The data came from various MRI platforms (GE, Philips, Siemens) with varying resolutions, increasing the model's robustness to real-world clinical variability.

Key Insights and Ideas

  • The Power of Synergy: The most significant insight is that segmentation and staging are mutually beneficial. The ablation study showed that training the tasks separately resulted in a significant drop in staging performance (AUC dropped from 0.85 to 0.78).
  • Transformers Over CNNs for Staging: While CNNs are excellent at "seeing" local textures, Transformers are better at "understanding" the global relationship between the tumor and the skull base. This global context is vital for T-staging, where the invasion of distant structures defines the stage.
  • Attention as a Bridge: The Feature-Fusion-Aware module acts as a filter, allowing the model to ignore irrelevant signals and focus on the most "expressive" representations of the tumor's extent.

Key Results

The multi-task model demonstrated strong performance across both domains:

  • Segmentation: Achieved a median Dice Similarity Coefficient (DSC) of 0.74 and an Average Surface Distance (ASD) of 0.97 mm. It statistically outperformed standard models like V-Net and Med3D.
  • T-Staging: Achieved an AUC of 0.85 and an accuracy of 70%.
  • Clinical Consistency: The model was particularly accurate at identifying early stages (T1-T2) with 76% accuracy, though it struggled slightly more with advanced stages (T3-T4) at 64% accuracy.
  • Superiority: In head-to-head comparisons, the multi-task approach consistently beat single-task benchmarks (like ResNet-50 or DenseNet) in every metric, including sensitivity and specificity.

Limitations and Future Work

Despite the promising results, the authors acknowledge several areas for improvement:

  • Single Modality: The model currently relies only on CE-T1WI images. Future iterations could benefit from multi-modal data (integrating T2-weighted or PET images) to provide a more comprehensive view of the tumor.
  • Data Imbalance: There was a relative scarcity of T1-stage cases in the dataset, which may have led to weaker performance in identifying the earliest stage of the disease.
  • Incomplete TNM Staging: The model only addresses "T" (tumor) staging. A complete clinical tool would also need to automate "N" (node) and "M" (metastasis) staging to fully align with the AJCC/UICC 8th edition guidelines.
  • Clinical Integration: The next step involves expanding the model to include nodal gross tumor volume (GTVn) and testing the tool in a live clinical workflow to measure actual time savings for radiologists.

Summary updated

Abstract

This study emphasized the potential of multi-task model for simultaneously delineating the tumor contour and identifying T-stage. The multi-task model harnesses the synergy between these interrelated learning tasks, leading to improvements in the performance of both tasks. The performance demonstrates the potential of our work for delineating the tumor contour and identifying T-stage and suggests that it can be a practical tool for supporting clinical precision radiation therapy.
BibTeX
@inproceedings{yang2024transformer,
  author = {Kaifan Yang and Xiuyu Dong and Fan Tang and Feng Ye and Bei Chen and Shujun Liang and Yu Zhang and Yikai Xu},
  title = {{A transformer-based multi-task deep learning model for simultaneous T-stage identification and segmentation of nasopharyngeal carcinoma}},
  booktitle = {Frontiers in Oncology},
  year = {2024},
  doi = {10.3389/fonc.2024.1377366},
}
Figure