TL;DR
This paper introduces RSTAR4D-Net, a novel 4D deep learning model for reducing rotational streak artifacts in 4D Cone-Beam CT (CBCT) used in image-guided radiation therapy. By treating 4D CBCT as a dynamic spatiotemporal problem and employing separable 4D convolutions and a unique "Tetris" training strategy, RSTAR4D-Net achieves superior artifact reduction and image quality compared to existing methods, even with limited training data.
What Problem Does This Paper Address?
Four-dimensional Cone-Beam Computed Tomography (4D CBCT) is crucial for image-guided radiation therapy (IGRT) as it provides respiration-resolved images, allowing for more precise targeting of tumors that move with breathing. However, the process of sorting projection data into different respiratory phases leads to sparse and non-uniform sampling, resulting in severe streak artifacts in the reconstructed 4D CBCT images. These artifacts degrade image quality and can hinder accurate tumor localization and treatment delivery. Existing deep learning methods often treat 4D CBCT as a collection of 2D images, neglecting the inherent dynamic and spatiotemporal nature of the data, which limits their effectiveness in addressing these artifacts.
Methods
The core innovation is the RSTAR4D-Net, a fully 4D neural network designed to process the entire 4D CBCT image (spatiotemporal volume) at once.
- Separable 4D Convolutions: To overcome the computational burden of traditional 4D convolutions, RSTAR4D-Net decomposes them into a series of lower-dimensional convolutions: a 2D convolution for the x-y plane, a 1D convolution for the z-axis, and a 1D convolution for the temporal dimension. This significantly reduces parameters and computational cost while still capturing spatiotemporal information.
- Rotational Streak Artifact (RSA) Insight: The authors identify that streak artifacts in 4D CBCT exhibit a unique "rotational motion" pattern distinct from respiratory motion. This insight motivates the development of a model that can differentiate and remove these specific artifacts.
- Tetris Training Strategy: To address the challenge of limited 4D training data, a two-stage training strategy is proposed:
- Stage I (2D+T): The model is trained on slice-by-slice (axial view) data, effectively increasing the number of samples. During this stage, the z-axis convolution is frozen, focusing on learning to reduce rotational streak artifacts.
- Stage II (4D Blocks): The model is then finetuned using 4D image blocks of varying sizes. The z-axis convolution is unfrozen to capture volumetric consistency and respiratory motion. This strategy acts as data augmentation and helps mitigate overfitting.
- Model Architecture: The RSTAR4D-Net is built upon the U-Net architecture, adapted with the proposed separable 4D convolutions.
Datasets
- Simulated Datasets:
- Training/Validation: 45 high-quality 4D CT images from Shanghai Chest Hospital were used. These were then used to simulate 4D CBCT images. Breathing signals from 20 real clinical 4D CBCT scans were also utilized. The simulated 4D CBCT images contained 10 volumes per phase.
- Test: 11 high-quality 4D CT images from the 4D-Lung Cancer Imaging Archive were used. Simulated 4D CBCT scans were generated from these, with original 4D CT images serving as ground truth.
- Real Clinical Dataset: 12 real clinical CBCT scans from Shanghai Chest Hospital were used for qualitative evaluation. Breathing signals were estimated from projection data.
Key Insights and Ideas
- Rotational Streak Artifacts (RSA): The identification of a distinct "rotational motion" pattern for streak artifacts, separate from respiratory motion, is a key insight. This suggests that these artifacts can be modeled and removed by considering their unique spatiotemporal behavior.
- 4D Processing is Crucial: Treating 4D CBCT as a truly spatiotemporal problem, rather than a collection of 2D slices, is essential for effectively capturing the dynamic nature of artifacts and anatomical structures.
- Separable Convolutions for Efficiency: Decomposing 4D convolutions into lower-dimensional operations is a practical solution to make full 4D network architectures computationally feasible for medical imaging tasks.
- "Tetris" Training for Data Scarcity: The proposed Tetris training strategy effectively leverages limited 4D data by progressively training the model, first on 2D+T data to learn artifact reduction and then on 4D blocks to learn volumetric consistency.
Key Results
- Quantitative Evaluation (Simulated Data): RSTAR4D-Net achieved the best SSIM (Structural Similarity Index) and RMSE (Root Mean Squared Error) across tumor, lung, and global regions compared to Gated FDK, PICCS, MSD-GAN, CycN-Net, and PriorNet. For example, it achieved the highest SSIM-Tumor at 84.33% and the lowest RMSE-Global at 42.54 HU.
- Qualitative Evaluation (Simulated Data): Visual results showed that RSTAR4D-Net significantly reduced streak artifacts while better preserving tumor structures compared to other methods.
- Qualitative Evaluation (Real Clinical Data): RSTAR4D-Net demonstrated superior volumetric consistency in diaphragm regions and better recovery of moving organ positions compared to other deep learning methods.
- Projection Domain Evaluation (Real Clinical Data): Quantitative analysis in the projection domain using SSIM and NCC showed that RSTAR4D-Net maintained better consistency of projection data across different respiratory phases.
- Efficiency: The separable convolution strategy makes RSTAR4D-Net efficient, with significantly fewer parameters and lower computational cost compared to other deep learning models (e.g., 2.35 million parameters vs. 113.77 million for MSD-GAN).
Limitations and Future Work
- Generalization: The authors acknowledge that clinical data often follow a long-tailed distribution, and generalizing the method to more complex cases requires further investigation.
- Future Work:
- Extend RSTAR4D-Net to other 4D imaging modalities like dynamic MR, PET, and SPECT.
- Explore incorporating advanced modules like temporal transformers or spatiotemporal implicit neural representations to further enhance performance.
- Combine the neural network model with traditional CT reconstruction techniques to enhance robustness and interpretability.
- Investigate the application of the model to scenarios like image-guided percutaneous lung biopsy, potentially enabling free-breathing scans.
Summary updated