Spatiotemporal correlation enhanced real-time 4D-CBCT imaging using convolutional LSTM networks

Hua Zhang, Kai Chen, Xiaotong Xu, Tao You, Wenzheng Sun, Jun DangFrontiers in Oncology 2024 · DOI

Abstract

The spatiotemporal correlation-based respiration motion modeling supplied a potential solution for accurate real-time 4D-CBCT reconstruction.

Introduction

In Stereotactic Body Radiotherapy (SBRT), high precision is essential for effective treatment. However, tumors located in the thorax and upper abdomen are subject to significant respiratory motion. While 3D Cone Beam CT (CBCT) is a standard for image guidance, it often results in motion-blurred images due to the "long exposure" nature of the acquisition. Traditional 4D-CBCT methods attempt to mitigate this by sorting projections into respiratory phases, but these methods are often slow and increase radiation exposure. Recent deep-learning approaches for "single-projection" estimation often treat X-rays as isolated snapshots, failing to account for the vital temporal context—the "flow" of respiration—between consecutive frames.

Methodology

This research proposes a deep-learning framework that leverages Convolutional Long Short-Term Memory (ConvLSTM) networks to capture spatiotemporal correlations in sequential X-ray projections.

Motion Modeling and Data Augmentation

To simplify the learning task, Principal Component Analysis (PCA) was applied to Deformation Vector Fields (DVFs) derived from patient 4D-CT scans. The researchers determined that the first three PCA coefficients capture over 97% of the motion information. To ensure model robustness, the dataset was expanded by simulating 900 distinct respiratory patterns through random sampling of motion amplitudes.

ConvLSTM Architecture

The core of the framework is the ConvLSTM network. Unlike standard Convolutional Neural Networks (CNNs) that focus solely on spatial features, the ConvLSTM processes a sequence of projections. This allows the model to utilize temporal context, effectively "remembering" previous frames to determine the direction of motion (e.g., distinguishing between inhalation and exhalation).

Reconstruction Pipeline

The predicted PCA coefficients are used to deform a high-quality planning CT into a real-time 3D volume that reflects the patient's current respiratory state.

Overview of the proposed framework, detailing the training stage (top) and the application stage (bottom) for real-time 4D-CBCT imaging.

Results and Discussion

The framework was evaluated using both the XCAT digital human phantom and pilot clinical data from 4D-CT scans.

Performance Metrics

The ConvLSTM approach was compared against CNN, U-Net, and ResNet architectures. It consistently demonstrated superior performance:

  • XCAT Phantom: Achieved a Mean Absolute Percentage Error (MAPE) of 0.0459 and a Peak Signal-to-Noise Ratio (PSNR) of 64.6742.
  • Clinical Data: Maintained high accuracy with a MAPE of 0.0934 and a PSNR of 63.7294.

Efficiency and Fidelity

The model is highly efficient, predicting motion labels in approximately 0.006 seconds per projection. Visual analysis of difference images confirmed that ConvLSTM significantly reduces residuals, particularly in high-motion areas like the diaphragm and lung boundaries.

Conclusion and Future Work

By integrating temporal context via ConvLSTM and utilizing PCA for dimensionality reduction, this framework provides a robust solution for real-time 4D-CBCT imaging. Current limitations include a dependency on simulated training data and the assumption of consistent breathing patterns. Future research will focus on hardware acceleration to reduce the 3D volume reconstruction time (currently ~5 seconds) and improving robustness against real-world CBCT artifacts.

Summary updated

Abstract

The spatiotemporal correlation-based respiration motion modeling supplied a potential solution for accurate real-time 4D-CBCT reconstruction.
BibTeX
@inproceedings{zhang2024spatiotemporal,
  author = {Hua Zhang and Kai Chen and Xiaotong Xu and Tao You and Wenzheng Sun and Jun Dang},
  title = {{Spatiotemporal correlation enhanced real-time 4D-CBCT imaging using convolutional LSTM networks}},
  booktitle = {Frontiers in Oncology},
  year = {2024},
  doi = {10.3389/fonc.2024.1390398},
}
Figure