Boosting ViT-based MRI Reconstruction from the Perspectives of Frequency Modulation, Spatial Purification, and Scale Diversification

Yucong Meng, Zhiwei Yang, Yonghong Shi, Zhijian SongAAAI 2025 · DOI

TL;DR

This paper introduces FPS-Former, a novel Vision Transformer (ViT) architecture designed to significantly improve Magnetic Resonance Imaging (MRI) reconstruction from undersampled k-space data. By addressing ViTs' inherent limitations in capturing high-frequency details, handling irrelevant token interactions, and modeling multi-scale information, FPS-Former achieves state-of-the-art performance with lower computational costs.

What Problem Does This Paper Address?

Magnetic Resonance Imaging (MRI) is a crucial diagnostic tool, but its long scan times can be burdensome for patients and lead to motion artifacts. Accelerating MRI acquisition by undersampling k-space data is a common solution, but it introduces artifacts that degrade image quality. Reconstructing high-quality MRI images from this undersampled data is a challenging ill-posed inverse problem. While Vision Transformers (ViTs) have shown promise in this area, they suffer from three key issues: (1) difficulty in capturing high-frequency details essential for texture and edge restoration, (2) inefficiently processing all image patches (tokens) regardless of their relevance, leading to noise and high computational burden, and (3) a lack of effective multi-scale feature representation. This paper aims to overcome these limitations to boost ViT-based MRI reconstruction performance.

Methods

The proposed FPS-Former framework is built upon a ViT architecture and introduces three novel modules to address the aforementioned challenges:

  • Frequency Modulation Attention Module (FMAM): This module enhances the self-attention mechanism by recalibrating frequency information using a Laplacian pyramid. It adaptively adjusts the importance of different frequency components, effectively counteracting the low-pass filtering tendency of standard ViTs and preserving crucial high-frequency details for better texture and edge reconstruction.
  • Spatial Purification Attention Module (SPAM): To tackle the issue of irrelevant token interactions and computational burden, SPAM groups similar image patches (tokens) based on their content. The self-attention mechanism is then applied within these groups, significantly reducing noise from unrelated tokens and lowering computational complexity. This allows for more precise self-similarity capturing.
  • Scale Diversification Feed-forward Network (SDFN): Recognizing the importance of multi-scale information for image restoration, SDFN incorporates two parallel depth-wise convolution paths with different kernel sizes (3x3 and 5x5) to effectively extract and fuse multi-scale features.

Additionally, the framework includes a Hybrid Experts Feature Refinement (HEFR) module, which uses a self-attention mechanism to adaptively emphasize different CNN-based "experts" for fine-grained feature refinement, further improving reconstruction quality. The overall architecture is a hierarchical encoder-decoder framework that stacks these FPS blocks and HEFR modules.

Datasets

The proposed FPS-Former was evaluated on three publicly available datasets:

  • CC359: A raw brain MRI dataset.
  • fastMRI: A dataset of complex-valued single-coil coronal proton density-weighted knee MRI scans.
  • SKM-TEA: A dataset of complex-valued multi-coil T2-weighted knee MRI scans.

The paper also mentions using various undersampling patterns (e.g., Cartesian, radial, random) and acceleration factors (AF) of 4x, 8x, 5x, and 10x for evaluation.

Key Insights and Ideas

  • ViTs are not inherently good at high frequencies: The paper highlights that standard ViTs, due to their MSA mechanism, act as low-pass filters, losing critical high-frequency information vital for MRI details. FMAM directly addresses this by actively recalibrating frequency content.
  • Global attention is computationally wasteful for MRI: Processing all image patches globally in ViTs is inefficient and introduces noise when many patches are contextually unrelated. SPAM's grouping strategy offers a more targeted and efficient approach.
  • Multi-scale processing is key for restoration: The paper emphasizes that a single-scale approach is insufficient for complex image restoration tasks like MRI reconstruction, and SDFN's multi-scale fusion is crucial.
  • Combining specialized modules is more effective: The synergistic effect of FMAM, SPAM, and SDFN within the FPS block, along with HEFR, leads to superior performance compared to individual component improvements or standard ViT blocks.

Key Results

  • State-of-the-art performance: FPS-Former consistently outperforms existing CNN-based and Transformer-based methods on single-coil (CC359, fastMRI) and multi-coil (SKM-TEA) datasets across various acceleration factors.
  • Significant PSNR gains: For instance, on CC359 under 4x AF, FPS-Former achieved a 2.37 dB improvement over the CNN-based DCRCN and 0.22 dB over ReconFormer. At 8x AF, it surpassed AST by 1.70 dB.
  • Robustness to different masks and acceleration: The method demonstrates strong performance even with challenging radial and random undersampling patterns and higher acceleration factors (5x and 10x).
  • Lower computational cost: Despite its superior performance, FPS-Former achieves this with significantly fewer FLOPs and parameters compared to other advanced ViT-based methods like ReconFormer and AST.

Limitations and Future Work

  • Hyperparameter sensitivity: While the paper mentions analyzing hyper-parameters, the optimal settings for FMAM's pyramid layers (M=3), SPAM's groups (N), and SDFN's expansion ratio (r) were determined empirically. Further investigation into more adaptive or automated hyperparameter tuning could be beneficial.
  • Generalizability to other modalities: The current work focuses on MRI. Extending FPS-Former to other medical imaging modalities or even natural image restoration tasks could be a promising future direction.
  • Further exploration of HEFR: While HEFR contributes to performance, exploring a wider range of "experts" or more sophisticated gating mechanisms within HEFR could potentially yield further improvements.

Summary updated

BibTeX
@inproceedings{meng2025boosting,
  author = {Yucong Meng and Zhiwei Yang and Yonghong Shi and Zhijian Song},
  title = {{Boosting ViT-based MRI Reconstruction from the Perspectives of Frequency Modulation, Spatial Purification, and Scale Diversification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i6.32656},
}
Figure