Coarse-to-Fine 3D MRI Reconstruction via Resolution-Agnostic Neural Operators

Jiayun Wang, Ram Daftari, Weijie Gan, Valentin Duruisseaux, Ruibo Wang, Armeet Singh Jatyani, Boris Bonev, Anima AnandkumararXiv 2026 · arXiv

TL;DR

KIR-NO is a new framework for 3D MRI reconstruction that solves the "memory nightmare" of training high-resolution 3D models. By using resolution-agnostic neural operators, it can be trained on memory-efficient low-resolution volumes and deployed "zero-shot" to high-resolution ones, outperforming state-of-the-art CNNs in both reconstruction fidelity and training speed.

What Problem Does This Paper Address?

While 3D MRI provides crucial volumetric context for diagnosis—especially when lesions span multiple slices—it is computationally expensive. Training deep learning models on full-resolution 3D volumes often exceeds the memory limits of even high-end GPUs.

To cope, researchers usually resort to 2D models that process MRI volumes slice-by-slice. However, this ignores the "through-plane" structure, leading to inconsistent reconstructions along the stacking axis. Standard 3D Convolutional Neural Networks (CNNs) are another option, but they are "grid-locked": a model trained at one resolution typically fails when applied to another. This prevents a "coarse-to-fine" strategy where one might train quickly on low-res data and deploy on high-res clinical scans.

Methods

The authors introduce KIR-NO (k-space-to-Image Reconstruction Neural Operator). The core innovation is the move from discrete pixels to continuous functions using Neural Operators (NOs).

1. Discrete-Continuous (DISCO) Convolutions

Instead of using fixed, discrete kernels (like a standard 3x3x3 filter), KIR-NO uses DISCO convolutions. These kernels are parameterized in a continuous basis—the authors designed two new 3D versions: piecewise-linear "hat" functions and 3D Morlet wavelets. Because the filters are continuous, they can be sampled at any voxel resolution. This makes the model "discretization-agnostic."

2. Dual-Domain Architecture

KIR-NO operates in both the raw data domain (k-space) and the image domain:

  • k-space NO Head: Refines the initial undersampled Fourier measurements to recover missing spectral content.
  • Image-space NO Cascades: After an inverse Fourier transform, a series of neural operator blocks further refine the image, interleaved with physics-based "data consistency" steps to ensure the reconstruction stays true to the original measurements.
The KIR-NO architecture combines a k-space refinement head with image-space cascades and physics-based data consistency.

Datasets

The model was evaluated on two major 3D MRI benchmarks:

  • SKM-TEA: A large-scale dataset of quantitative knee MRI scans.
  • BraTS 2021: A brain tumor segmentation benchmark, where the authors synthesized multi-coil k-space data to test reconstruction.

Key Insights and Ideas

  • Resolution Agnosticism as a Superpower: By decoupling the model weights from the voxel grid, KIR-NO enables a "train-low, deploy-high" paradigm. This bypasses the memory bottleneck of 3D training without sacrificing high-resolution performance.
  • Faster Convergence: KIR-NO isn't just more flexible; it's more efficient. It reaches high-quality reconstruction thresholds significantly faster than standard 3D CNNs.
  • The Importance of k-space: Ablation studies showed that the k-space head is the primary driver of high-resolution gains, effectively "pre-filling" missing data before the image-space cascades take over.
Figure 1: (a–c) Three deep learning architectures for 3D MRI reconstruction. (a) 2D CNN: GPU- memory limits force slice-wise 2D models that stack into a volume, losing inter-slice consistency. (b) 3D CNN: fixed discrete kernels are tied to the training grid, so cross-resolution transfer is un- reliable. (c) 3D Neural Operator (KIR-NO, ours): discretization-agnostic — trained on memory- friendly coarse volumes and deployed at higher resolutions without retraining. (d) Resolution robustness: PSNR at quarter/half/full resolution on SKM-TEA (quarter-trained); KIR-NO leads throughout. (e) Convergence (SKM-TEA, half-res, 4×): first epoch to reach each PSNR threshold; KIR-NO reaches 34 dB ∼6× faster than the 3D CNN. (f) Qualitative: a knee slice with error maps for 2D CNN, 3D CNN, and KIR-NO; KIR-NO has the lowest residual.

Key Results

  • Superior Fidelity: On the SKM-TEA knee dataset, KIR-NO improved over a state-of-the-art 3D CNN by 2.2 dB PSNR at half-resolution.
  • Zero-Shot Super-Resolution: When trained at half-resolution and deployed at full-resolution, KIR-NO achieved a +3 dB improvement in PSNR and significantly higher structural similarity (SSIM 0.940 vs 0.870) compared to the best CNN baseline.
  • 6x Faster Training: KIR-NO reached a target PSNR of 34 dB roughly 6 times faster than the 3D CNN baseline (5 epochs vs 31 epochs).
  • Robustness: The gains persisted across different acceleration factors (4x, 8x, and 16x) and various k-space sampling patterns (Gaussian, Poisson-disc, radial).

Limitations and Future Work

While KIR-NO solves the training memory bottleneck, inference at full resolution still requires significant GPU memory (e.g., ~72 GiB for a 512³ volume). The authors suggest that "tiled execution" could mitigate this in the future.

Additionally, the current evaluation uses single-seed runs and doesn't fully account for real-world "distribution shifts" like patient motion or different scanner hardware between training and deployment. Future work could involve integrating uncertainty quantification and testing on more diverse, multi-contrast clinical datasets.

Summary updated

Abstract

Deep-learning-based 3D MRI reconstruction is limited in practice by the GPU memory and compute cost of full-volume training; 3D acquisitions are thus most often reconstructed by 2D models applied slice-by-slice and then stacked, which ignores through-plane (stacking-axis) structure. We reconstruct in full 3D and ask whether a model can be trained at low resolution (fast and memory-feasible) yet deployed at higher resolution. This coarse-to-fine strategy needs reliable cross-resolution performance, but standard networks are resolution-specific and degrade substantially when evaluated outside the training setting. We therefore use resolution-agnostic neural operators (NOs) and propose KIR-NO (k-space-to-Image Reconstruction Neural Operator), a discretization-agnostic framework for accelerated 3D MRI reconstruction. KIR-NO is trained with full backpropagation on low-resolution volumes and applied zero-shot to higher-resolution volumes at inference. Its core building block is a 3D neural operator based on discrete-continuous convolutions: convolutional filters are parameterized in a continuous basis and sampled at arbitrary voxel resolutions while preserving the local inductive bias of convolution. KIR-NO integrates these learned operators with physics-based consistency: a k-space neural-operator head first refines the undersampled Fourier measurements, and image-space neural-operator cascades then refine the reconstruction through interleaved data-consistency updates. Across 3D knee and brain MRI benchmarks, KIR-NO improves over classical and CNN baselines: it improves over a state-of-the-art 3D CNN by 2.2 dB PSNR at half resolution under 4x acceleration on SKM-TEA, with gains persisting at 8x and 16x. These results show that resolution-agnostic operators make coarse-to-fine 3D MRI reconstruction - cheap low-resolution training with high-resolution deployment - practical across anatomies.
BibTeX
@misc{wang2026coarse,
  author = {Jiayun Wang and Ram Daftari and Weijie Gan and Valentin Duruisseaux and Ruibo Wang and Armeet Singh Jatyani and Boris Bonev and Anima Anandkumar},
  title = {{Coarse-to-Fine 3D MRI Reconstruction via Resolution-Agnostic Neural Operators}},
  year = {2026},
  eprint = {2611.14979},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.14979},
}
Figure