TL;DR
This paper introduces the first deep learning network, DISN, for multi-contrast compressed sensing MRI reconstruction. By sharing features across different MRI contrasts and using dense connections, DISN significantly improves reconstruction accuracy and efficiency compared to existing methods, with potential benefits for medical image analysis and robustness to real-world data imperfections.
What Problem Does This Paper Address?
Magnetic Resonance Imaging (MRI) is a crucial tool for medical diagnosis, but its slow acquisition speed is a major bottleneck. Compressed Sensing (CS) offers a solution by allowing reconstruction from fewer k-space measurements, thus accelerating MRI. However, traditional CS-MRI methods, especially those dealing with multiple MRI contrasts (e.g., T1, T2, PD weighted images), face several challenges:
- Limited Information Sharing: Models often treat each contrast independently, failing to leverage the inherent structural similarities between them.
- Computational Cost: Optimization-based methods can be very time-consuming.
- Shallow Models: Traditional approaches struggle to capture the complex patterns present in large MRI datasets.
- Strict Assumptions: Many methods rely on strong assumptions about sparsity in specific transform domains, which may not always hold true, especially in real-world, non-ideal scenarios.
This paper aims to overcome these limitations by developing a deep learning framework that effectively shares information across multiple MRI contrasts to achieve faster and more accurate reconstructions.
Methods
The authors propose a novel deep learning architecture called the Deep Information Sharing Network (DISN). The core ideas are:
- Feature Sharing Unit: This unit is designed to exploit the structural similarities present across different MRI contrasts. Instead of processing each contrast in isolation, it learns shared feature representations. This significantly reduces the number of parameters compared to having separate networks for each contrast.
- Data Fidelity Unit: This unit ensures that the reconstructed images remain consistent with the acquired k-space measurements, a standard component in CS reconstruction.
- Inference Block: Each block combines a feature sharing unit and a data fidelity unit.
- Dense Connections: Inspired by DenseNet, the DISN cascades these inference blocks, but with a crucial difference: each block receives inputs from all preceding blocks. This allows for efficient information flow and reuse across different depths of the network, preventing information loss as the network gets deeper. This dense connection strategy, applied to intermediate reconstructions rather than raw feature maps, keeps the parameter growth manageable.
The paper also introduces two baseline deep models:
- Deep Independent Reconstruction Network (DIRN): This model uses parallel subnetworks for each contrast, with no information sharing between them.
- Deep Feature Sharing Network (DFSN): This model incorporates the feature sharing unit but lacks the dense connections of DISN.
Datasets
The proposed DISN model was trained and evaluated on three multi-contrast MRI datasets:
- SRI24 Atlas: A dataset of normal adult brain MRIs with T1-weighted, T2-weighted, and PD-weighted contrasts.
- MRBrainS: A benchmark dataset of brain MRIs from patients with varying degrees of white matter lesions, including T1-weighted, T1-weighted inversion recovery, and T2-FLAIR contrasts.
- NeoBrainS: A dataset of neonatal brain MRIs with T1-weighted and T2-weighted contrasts.
The authors randomly split these datasets into training and testing sets, using standard under-sampling patterns (1D Cartesian masks) with varying ratios (e.g., 10%, 20%).
Key Insights and Ideas
- Leveraging Multi-Contrast Similarity is Key: The paper strongly advocates for exploiting the structural similarities between different MRI contrasts. This is a fundamental insight that traditional methods often underutilize.
- Feature Sharing Reduces Parameters Dramatically: By sharing feature representations, DISN achieves comparable or better performance with significantly fewer parameters than independent reconstruction networks. This is crucial for model efficiency and generalization.
- Dense Connections Enhance Information Flow: The dense connection strategy is vital for DISN's success. It ensures that information from earlier stages of reconstruction is effectively propagated to later stages, leading to progressively better results. This addresses the problem of information loss in deeper network layers.
- Deep Learning Outperforms Traditional Methods: The experiments consistently show that DISN surpasses both single-contrast and multi-contrast optimization-based methods in terms of reconstruction quality and speed.
- Robustness to Real-World Imperfections: A significant finding is DISN's robustness to non-registration (slight shifts) between MRI contrasts, a common issue in clinical practice. This makes it more practical for real-world applications than methods that rely on perfect alignment.
- Improved Reconstruction Aids Downstream Tasks: The paper demonstrates that better MRI reconstructions directly translate to improved performance in medical image analysis tasks like segmentation, highlighting the practical impact of their work.
Key Results
- Accuracy: DISN consistently achieved state-of-the-art performance in terms of PSNR and SSIM across all tested datasets, outperforming methods like BCS, PANO, GBRWT, and FCSA-MT. For example, on the SRI24 dataset, DISN achieved higher PSNR and SSIM values than DFSN and DIRN.
- Efficiency: The proposed deep learning models (DIRN, DFSN, DISN) are significantly faster at testing time compared to optimization-based methods. DISN achieved reconstruction in milliseconds (e.g., 0.18s for 5 blocks) compared to minutes or hours for methods like BCS.
- Parameter Efficiency: DFSN-5B had 101K parameters, significantly fewer than DIRN-5B (286K), while DISN-5B had slightly more (106K) due to dense connections. Even with more parameters, DISN-5B with 32 feature maps outperformed DFSN-5B with 64 feature maps.
- Segmentation Improvement: When reconstructed images were used for segmentation with a U-Net model, DISN-reconstructed images led to higher Dice Coefficients (e.g., 76.41% for GM, 85.03% for WM, 80.97% for CSF) compared to other CS-MRI methods, approaching the performance of full-sampled MRIs.
- Robustness: DISN showed superior robustness to pixel shifts compared to FCSA-MT, maintaining good PSNR performance even with misregistered data.
Limitations and Future Work
- Dataset Size: While experiments were conducted on multiple datasets, the authors acknowledge that larger and more diverse datasets could further improve generalization.
- Network Depth: While DISN's dense connections allow for deeper networks, the paper notes that performance gains diminish after a certain depth (e.g., 5 blocks), suggesting an optimal depth exists.
- Future Work: The authors suggest exploring more advanced network architectures, investigating different under-sampling patterns (e.g., 2D random masks), and further validating the model on a wider range of clinical applications and patient populations. The potential for DISN to benefit medical image analysis tasks is highlighted as a key area for future research.
Summary updated