Respiratory Prediction Based on Multi-Scale Temporal Convolutional Network for Tracking Thoracic Tumor Movement

Lijuan Shi, Shuai Han, Jian Zhao, Zhejun Kuang, Weipeng Jing, Yuqing Cui, Zhanpeng ZhuFrontiers in Oncology 2022 · DOI

Abstract

Radiotherapy is one of the important treatments for malignant tumors. The precision of radiotherapy is affected by the respiratory motion of human body, so real-time motion tracking for thoracoabdominal tumors is of great significance to improve the efficacy of radiotherapy. This paper aims to establish a highly precise and efficient prediction model, thus proposing to apply a depth prediction model composed of multi-scale enhanced convolution neural network and temporal convolutional network based on empirical mode decomposition (EMD) in respiratory prediction with different delay times. First, to enhance the precision, the unstable original sequence is decomposed into several intrinsic mode functions (IMFs) by EMD, and then, a depth prediction model of parallel enhanced convolution structure and temporal convolutional network with the characteristics specific to IMFs is built, and finally training on the respiratory motion dataset of 103 patients with malignant tumors is conducted. The prediction precision and time efficiency of the model are compared at different levels with those of the other three depth prediction models so as to evaluate the performance of the model. The result shows that the respiratory motion prediction model determined in this paper has superior prediction performance under different lengths of input data and delay time, and, furthermore, the network update time is shortened by about 60%. The method proposed in this paper will greatly improve the precision of radiotherapy and shorten the radiotherapy time, which is of great application value.

Introduction: Addressing Motion Latency in Radiotherapy

In radiotherapy, precision is paramount. However, thoracic and abdominal tumors, such as those in the lungs or liver, are dynamic targets that move with a patient’s breathing. While modern hardware can track this motion, there is an inherent system latency—ranging from 50ms to over 400ms—between detecting movement and adjusting the radiation beam.

Multi-modal tumor tracking visualization showing fluoroscopic (A) and CT (B) imaging used to identify and track moving thoracic targets.

If the system fails to accurately predict the tumor's future position, radiation may miss the malignancy and damage surrounding Organs at Risk (OARs). Traditional models like LSTMs (Long Short-Term Memory) often struggle with the non-linear, noisy nature of human breathing and are frequently too slow to update in real-time. This research introduces a hybrid deep learning model, EMD-SEnet-TCN, designed to predict respiratory motion with high precision and significantly reduced latency.

Methodology: The EMD-SEnet-TCN Framework

The proposed pipeline utilizes a three-stage approach to process and predict respiratory signals:

  1. Empirical Mode Decomposition (EMD): The model decomposes raw, unstable respiratory data into several "Intrinsic Mode Functions" (IMFs). This simplifies complex waves into manageable components based on different time scales, facilitating easier pattern recognition.
  2. Multi-Scale CNN with SEnet: The architecture employs a parallel Convolutional Neural Network (CNN) with three kernel sizes (3x1, 5x3, and 7x5). This allows the network to capture both fine-grained local changes and broader temporal trends simultaneously. A Squeeze-and-Excitation (SEnet) block provides an attention mechanism to prioritize the most relevant data channels.
  3. Temporal Convolutional Network (TCN): By using causal and dilated convolutions, the TCN processes historical data in parallel. This avoids the sequential processing bottlenecks of RNNs, increasing speed without sacrificing the "memory" of previous respiratory cycles.
Detailed view of the tracking algorithm identifying tumor movement within a localized region of interest in fluoroscopic imaging.

Experimental Setup and Dataset

The study utilized a dataset from the Institute of Robotics and Cognitive Systems at the University of Lübeck, Germany, encompassing 306 respiratory motion trajectories from 103 patients. Motion was recorded via three markers on the chest and abdomen using an optical tracking system at a sampling frequency of 26 Hz.

Results: Efficiency and Accuracy Gains

The EMD-SEnet-TCN model was benchmarked against CNN-BiLSTM and standard TCN architectures, yielding significant improvements:

  • Computational Efficiency: Network update times were reduced by approximately 60% compared to BiLSTM-based models, cutting epoch times from roughly 10 seconds to 2 seconds.
  • Prediction Accuracy: At a clinical delay of 400ms, the model outperformed CNN-TCN by 13.7% in Mean Absolute Error (MAE) and 9.2% in Root Mean Square Error (RMSE).
  • Robustness: The model maintained high stability, with R² scores consistently exceeding 0.90 even as look-ahead times were extended to 520ms.

Discussion and Future Clinical Integration

While the EMD-SEnet-TCN model demonstrates superior performance, two primary hurdles remain for clinical adoption. First, the current model predicts external marker movement; future iterations must better correlate these external signals with internal tumor positions. Second, the system requires validation within specialized radiotherapy software environments (e.g., CyberKnife or Varian MLC) to ensure theoretical speedups translate to real-world latency reduction. Finally, the authors emphasize the necessity of robust medical analysis platforms that strictly adhere to patient privacy laws during deployment.

Summary updated

Abstract

Radiotherapy is one of the important treatments for malignant tumors. The precision of radiotherapy is affected by the respiratory motion of human body, so real-time motion tracking for thoracoabdominal tumors is of great significance to improve the efficacy of radiotherapy. This paper aims to establish a highly precise and efficient prediction model, thus proposing to apply a depth prediction model composed of multi-scale enhanced convolution neural network and temporal convolutional network based on empirical mode decomposition (EMD) in respiratory prediction with different delay times. First, to enhance the precision, the unstable original sequence is decomposed into several intrinsic mode functions (IMFs) by EMD, and then, a depth prediction model of parallel enhanced convolution structure and temporal convolutional network with the characteristics specific to IMFs is built, and finally training on the respiratory motion dataset of 103 patients with malignant tumors is conducted. The prediction precision and time efficiency of the model are compared at different levels with those of the other three depth prediction models so as to evaluate the performance of the model. The result shows that the respiratory motion prediction model determined in this paper has superior prediction performance under different lengths of input data and delay time, and, furthermore, the network update time is shortened by about 60%. The method proposed in this paper will greatly improve the precision of radiotherapy and shorten the radiotherapy time, which is of great application value.
BibTeX
@inproceedings{shi2022respiratory,
  author = {Lijuan Shi and Shuai Han and Jian Zhao and Zhejun Kuang and Weipeng Jing and Yuqing Cui and Zhanpeng Zhu},
  title = {{Respiratory Prediction Based on Multi-Scale Temporal Convolutional Network for Tracking Thoracic Tumor Movement}},
  booktitle = {Frontiers in Oncology},
  year = {2022},
  doi = {10.3389/fonc.2022.884523},
}
Figure