TL;DR
This paper introduces JSR-Net, a novel deep learning architecture that tackles the challenging problem of reconstructing CT images from incomplete data (sparse views or limited angles). By jointly learning to reconstruct both the image and its Radon domain projections, and employing a hybrid loss function, JSR-Net significantly improves image quality and preserves structural details compared to existing methods.
What Problem Does This Paper Address?
Medical imaging, particularly Computed Tomography (CT), often faces the challenge of acquiring complete data. This can be due to patient movement, safety concerns (reducing radiation dose), or physical limitations of the scanner. Reconstructing usable CT images from such incomplete datasets (sparse-view or limited-angle CT) is crucial for accurate diagnosis but is inherently difficult. Traditional methods like Filtered Back-Projection (FBP) struggle with artifacts and poor image quality in these scenarios. While iterative reconstruction methods offer improvements, they often fail to fully leverage the vast amounts of data available for training deep learning models. This paper aims to bridge this gap by developing a deep learning approach that can effectively reconstruct high-quality CT images from incomplete data.
Methods
The core innovation of this paper is the JSR-Net, a deep convolutional neural network (CNN) designed to unroll and approximate an existing iterative optimization algorithm known as the Joint Spatial-Radon (JSR) model. The JSR model itself is designed to improve reconstruction by considering both the spatial domain of the CT image and its corresponding Radon domain projections, ensuring data consistency in both.
JSR-Net achieves this by:
- Unrolling the JSR Algorithm: Instead of running the full iterative algorithm, JSR-Net learns to approximate key steps within a feed-forward network. This allows for faster inference while retaining the benefits of the iterative approach.
- CNNs for Operator Approximation: Specific CNNs are trained to approximate complex operators within the JSR algorithm, such as matrix inversions and thresholding operations. This replaces computationally intensive steps with learned, efficient neural network modules.
- Hybrid Loss Function: To further enhance image quality and preserve important structures, a novel hybrid loss function is introduced. This function combines:
- SSIM Loss: Measures structural similarity between the reconstructed and ground truth images.
- MSE Loss: Standard mean squared error in both spatial and Radon domains.
- Semantic Loss: A novel component that focuses on preserving image content by comparing level sets of the reconstructed and ground truth images. This is particularly important for preventing over-smoothing and maintaining sharp features.
Datasets
The paper utilizes two types of datasets:
- Simulated Data: CT imaging system with 1024 detectors and fan beam geometry was simulated for both sparse-view (90 views over 360°) and limited-angle (150 views with 1° scanning step) scenarios. Additive white Gaussian noise with a variance of $\sigma = 0.05$ was added to the projection data.
- Real Clinical Data: The "2016 NIH-AAPM-Mayo Clinic Low Dose CT Grand Challenge" dataset was used for training and validation. This dataset consists of abdomen CT images from five patients, totaling 1684 slices of 512x512 pixels. A subset of 50 slices was used for validation, and the rest for training.
Key Insights and Ideas
- Joint Spatial-Radon Reconstruction is Powerful: The paper reinforces the idea that considering both spatial and Radon domains simultaneously is crucial for robust reconstruction from incomplete data, as it enforces data consistency across different representations.
- Deep Unrolling for Optimization: Unrolling iterative optimization algorithms into deep neural networks is a highly effective strategy for solving inverse problems, offering a powerful blend of model-based priors and data-driven learning.
- Hybrid Loss for Structural Preservation: The proposed hybrid loss function, particularly the inclusion of a semantic loss component, is key to preventing over-smoothing and preserving fine details and sharp structures in the reconstructed images, which is a common pitfall in deep learning-based reconstruction.
- JSR-Net Outperforms State-of-the-Art: The JSR-Net, especially when trained with the hybrid loss, demonstrates superior performance compared to existing methods, including other deep learning approaches and traditional iterative techniques.
Key Results
- Quantitative Improvements: JSR-Net with the SS2 hybrid loss consistently achieved higher SSIM and PSNR values, and lower MSE and NRMSE values compared to FBP, PD-Net (another deep learning method), and the original JSR model. For sparse-view CT, JSR-Net with SS2 loss achieved an SSIM of 0.9081 and PSNR of 31.59.
- Qualitative Improvements: Visual inspection of the reconstructed images (Figures 1 and 2) shows that JSR-Net produces images with significantly fewer artifacts and better preservation of anatomical details, especially in challenging regions, compared to other methods. The error maps further highlight the superior performance of JSR-Net.
- Competitive Performance: For limited-angle CT, JSR-Net achieved comparable results to PD-Net, indicating its robustness across different types of incomplete data.
Limitations and Future Work
- Limitations: The paper notes that the semantic loss component, while beneficial, might not be crucial for all sparse-view problems, and its impact can be mild for methods lacking a joint spatial-Radon recovery mechanism like PD-Net.
- Future Work: The authors suggest exploring more reliable loss functions that are even more effective at preserving details. They also propose extending the JSR-Net architecture and its applications to other areas, such as interior/exterior CT reconstruction. The use of more complex noise models (e.g., Poisson-Gaussian) is also mentioned as a future direction.
Summary updated