TL;DR
This paper introduces a novel three-stage self-supervised framework for denoising fluoroscopy videos, which are inherently noisy due to low-dose X-ray imaging and challenging to process due to motion artifacts. Their method effectively stabilizes the video, decomposes it into background and foreground components, and then denoises each part independently, achieving significant improvements in image quality.
What Problem Does This Paper Address?
Fluoroscopy is a crucial medical imaging technique that provides real-time X-ray visualization of internal structures, aiding surgeons during interventions. However, to minimize radiation exposure, low-dose X-rays are used, resulting in significant noise in the acquired images. This noise severely degrades image quality, making it difficult to discern pathological structures and tissue functions. Compounding this issue is the inherent motion present in fluoroscopy procedures, where the X-ray detector can move relative to the patient, leading to unstable backgrounds and further complicating denoising efforts. Traditional denoising methods struggle with these combined challenges, especially in the absence of clean ground truth data for training.
Methods
The authors propose a three-stage self-supervised framework designed to tackle the specific challenges of fluoroscopy denoising:
- Stabilize: This first stage addresses the motion artifacts caused by detector movement. It computes optical flow between consecutive frames to estimate the background motion. Using a kernel density estimator (KDE) on the optical flow field, they identify the dominant background movement and construct a stabilized "world view canvas" by compensating for these translations. This ensures temporal consistency and a stable background for subsequent processing.
- Decompose: Building upon the stabilized video, this stage employs a novel mask-based Robust Principal Component Analysis (RPCA) method. RPCA is a standard technique for separating low-rank background components from sparse foreground elements. Their proposed method incorporates a mask to better handle the specific characteristics of fluoroscopy videos, allowing for a more accurate separation of the static background from dynamic foreground structures (like instruments or anatomical changes). This decomposition is designed to align with how medical experts typically interpret fluoroscopy images.
- Denoise: In the final stage, both the decomposed low-rank background and sparse foreground are denoised independently using a self-supervised learning strategy. The paper specifically mentions using the "Self2Self" network, a known self-supervised denoising approach. For the foreground, an additional spatiotemporal bilateral filter is applied to further enhance denoising by leveraging temporal information and spatial coherence. Finally, the denoised background and foreground are fused to produce the final output.
Datasets
- Fluoroscopy Dataset: The authors curated a new dataset consisting of 27 fluoroscopy videos (1,568 frames total). This dataset includes 20 videos with static backgrounds and 7 with non-static backgrounds. For evaluation, they added Gaussian noise with variances of 0.001, 0.003, and 0.005 to clean images to simulate corrupted data. This dataset serves as a benchmark for their method.
- Clinical Dataset: They also collected 60 groups of real-world corrupted fluoroscopy samples acquired with low-dose X-rays. Each group contains an original noisy image and denoised versions from their method and other competing methods.
- Training Data: For training the self-supervised denoiser (Self2Self), they used the X-ray Coronary Angiograms dataset (XCA), comprising 22 videos.
Key Insights and Ideas
- Motion as a Feature, Not Just Noise: Instead of treating detector motion solely as an artifact to be removed, the paper leverages it to stabilize the video and then uses the motion characteristics to inform the decomposition process.
- Domain-Specific Decomposition: The mask-based RPCA is tailored to fluoroscopy, acknowledging that foreground elements are sparse and dynamic, while the background, even with detector motion, has underlying low-rank properties.
- Self-Supervised Power for Medical Imaging: The paper demonstrates the effectiveness of self-supervised learning for medical imaging denoising, a field where acquiring large, clean, paired datasets is often challenging.
- Synergy of Stages: The three stages are designed to work in concert. Stabilization prepares the video for accurate decomposition, and the decomposed components allow for more targeted and effective denoising.
Key Results
- Quantitative Improvements: The proposed framework achieves significant improvements in denoising performance, reporting an increase of about 5-6 dB in PSNR compared to other self-supervised methods.
- Superior Performance: In comparisons with state-of-the-art self-supervised methods like Noise2Void, Noise2Self, and Self2Self, their method consistently achieved better PSNR and SSIM scores across different noise levels.
- Expert Validation: A radiologist with 20 years of experience rated the denoised images from the clinical dataset. Their framework received a significantly higher average rating (4.883/5) compared to other methods, indicating superior perceived image quality and preservation of micro-structures.
- Stabilization Benefits: Ablation studies showed that the Stabilize stage preserves more background information, leading to better overall denoising performance.
Limitations and Future Work
- Complex Motion: The paper mentions that the current framework is designed for relatively straightforward motion. For more complex circumstances, such as affine transformations or non-rigid deformations, they plan to explore using deep learning models to estimate affine parameters.
- Computational Cost: While the framework shows strong results, the paper notes that the Inc-PCP algorithm is implemented on GPU, and some parts (like filling blank areas) are handled separately. Future work might focus on optimizing the computational efficiency.
Summary updated