Abstract
Compared with 2D MRI, 3D MRI provides superior volumetric spatial resolution and signal-to-noise ratio. However, it is more challenging to reconstruct 3D MRI images. Current methods are mainly based on convolutional neural networks (CNN) with small kernels, which are difficult to scale up to have sufficient fitting power for 3D MRI reconstruction due to the large image size and GPU memory constraint. Furthermore, MRI reconstruction is a deconvolution problem, which demands long-distance information that is difficult to capture by CNNs with small convolution kernels. The multi-layer perceptron (MLP) can model such long-distance information, but it requires a fixed input size. In this paper, we proposed Recon3DMLP, a hybrid of CNN modules with small kernels for low-frequency reconstruction and adaptive MLP (dMLP) modules with large kernels to boost the high-frequency reconstruction, for 3D MRI reconstruction. We further utilized the circular shift operation based on MRI physics such that dMLP accepts arbitrary image size and can extract global information from the entire FOV. We also propose a GPU memory efficient data fidelity module that can reduce >50% memory. We compared Recon3DMLP with other CNN-based models on a high-resolution (HR) 3D MRI dataset. Recon3DMLP improves HR 3D reconstruction and outperforms several existing CNN-based models under similar GPU memory consumption, which demonstrates that Recon3DMLP is a practical solution for HR 3D MRI reconstruction.
Introduction
High-resolution 3D MRI is essential for clinical diagnosis but presents significant reconstruction challenges. Reconstructing images from under-sampled k-space data is fundamentally a deconvolution problem. Conventional deep learning approaches, primarily based on Convolutional Neural Networks (CNNs), face two critical limitations: the locality problem, where small kernels struggle to capture global dependencies inherent in MRI aliasing, and the memory wall, where massive 3D volumes exceed typical GPU capacities.
Methods: The Recon3DMLP Architecture
To address these issues, the researchers introduced Recon3DMLP, a cascaded architecture that hybridizes local and global processing.
Adaptive MLP (dMLP)
The core innovation is the dynamic/adaptive MLP (dMLP). Unlike traditional MLPs that require fixed input dimensions, the dMLP module employs circular padding, patching, and circular window shifts. This design allows the model to process arbitrary image sizes and extract global information across the entire Field of View (FOV) by shifting patches. This approach leverages the periodicity of the Discrete Fourier Transform (DFT), ensuring the model respects the underlying physics of MRI acquisition.
Memory-Efficient Data Fidelity (eDF)
Standard Data Fidelity (DF) layers are memory-intensive because they process all receiver coils simultaneously. The authors proposed an efficient Data Fidelity (eDF) module that processes coils sequentially. This mathematically equivalent approach reduces memory consumption by over 50%, enabling the reconstruction of massive 3D volumes on standard hardware.
Experimental Setup
The model was evaluated on a high-resolution (HR) 3D multi-coil brain MRI dataset consisting of 812 images. The resolution of these volumes reached up to $336 \times 336 \times 192$ with 32 coils, representing a significantly higher computational load than standard 2D benchmarks.
Key Results
Recon3DMLP demonstrates superior performance in both quantitative metrics and computational efficiency.
- Quantitative Superiority: Recon3DMLP achieved a PSNR of 41.19, outperforming standard CNN baselines (40.27). As shown in the loss curves, Recon3DMLP (dashed blue line) achieves the lowest L1 loss and faster convergence compared to other variants.
- Memory Efficiency: By combining half-precision (FP16) training with the eDF module, the memory footprint was reduced from over 40GB to approximately 11.5GB, making high-resolution 3D reconstruction feasible on mid-range professional GPUs.
- High-Frequency Recovery: Qualitative analysis through error maps indicates that the dMLP module is particularly effective at recovering high-frequency details and fine structures that small-kernel CNNs typically blur.
Conclusion
Recon3DMLP provides a practical and powerful solution for high-resolution 3D MRI reconstruction. By combining the local feature extraction of CNNs with the global context of adaptive MLPs, the model overcomes the locality limitations of traditional architectures while maintaining a manageable memory footprint. Future work will explore multi-scale dMLP modules to further enhance feature capture across varying anatomical structures.
Summary updated