In an ideal world, three-dimensional medical imaging scans—such as Magnetic Resonance Imaging (MRI) or Computed Tomography (CT)—would consist of perfectly isotropic voxels, offering the same high level of detail whether viewed from the front, side, or top. Accurate 3D anatomical maps are vital for clinical interventions and automated morphometric analysis (Accelerated free breathing ECG triggered contrast enhanced pulmonary vein magnetic resonance angiography using compressed sensing).
However, physical and clinical realities force a stark compromise. In practice, scans are highly anisotropic: they possess sharp, high-resolution details within a given 2D slice, but are separated by large, blurry gaps along the perpendicular slice-selection axis.
To bridge this through-plane resolution gap, researchers have developed a variety of computational frameworks. Rather than relying on simple geometric interpolation, which introduces severe blurring and stair-step artifacts, modern techniques exploit physical redundancies, continuous coordinate spaces, and generative priors to synthesize missing high-frequency anatomical details.
The Slice-Direction Resolution Bottleneck
Why does anisotropy occur in the first place? The root cause lies in a fundamental physical "tug-of-war" between spatial resolution, acquisition speed, and the signal-to-noise ratio (SNR).
As explained by Peeters et al. in A Superresolution Framework for fMRI Sequences and Its Impact on Resulting Activation Maps, the data redundancy inherent in Fourier-based in-plane acquisition makes super-resolution mathematically unnecessary in two dimensions. The primary bottleneck for spatial detail exists almost entirely in the slice-selection direction, where physical constraints limit the density of information compared to the in-plane grid.
To maintain adequate signal levels and prevent scans from becoming prohibitively long—which would risk severe patient motion artifacts—technicians routinely acquire images with increased slice thickness (A 3D Conditional Diffusion Model for Image Quality Transfer - An Application to Low-Field MRI). This clinical prioritization of acquisition speed results in "thick-slice" scans (MRI Super-Resolution using Multi-Channel Total Variation).
Anisotropic Voxel (Clinical Reality) Isotropic Voxel (Ideal)
+-------------------+ +---+
| | | |
| In-Plane | | |
| High Resolution | | |
| | +---+
+-------------------+ Equal resolution
Thick Slice-Selection Axis in all 3 axes
When these 2D slices are stacked into a 3D volume, the missing information along the low-resolution axis creates significant visual artifacts (SuperSlicing Frame Restoration for Anisotropic ssTEM and Video Data). This through-plane sparsity prevents accurate 3D volumetric analysis and limits the effectiveness of automated segmentation or mesh-refinement tools (Deformable biomechanical models: Application to 4D cardiac image analysis; SynthSR: A public AI tool to turn heterogeneous clinical brain scans into high-resolution T1-weighted images for 3D morphometry).
Leveraging Internal and Cross-Modality Priors
To resolve this bottleneck without forcing patients to spend hours inside a scanner, algorithms must infer what the anatomy looks like between the acquired slices. The core intuition behind these methods is that the missing high-frequency information often exists elsewhere in the patient's own data.
One strategy is to use a high-resolution structural scan to guide the reconstruction of a lower-resolution scan from the same subject. In early work, Penney et al. demonstrated this in Registration-Based Interpolation Using a High-Resolution Image for Guidance by tracking anatomical features across slices in a high-resolution image and transferring those pathways to a low-resolution scan.
Modern variants of this idea pool information across different co-registered image contrasts (such as T1 and T2 weighted images). For instance, Brudfors et al. developed a generative model utilizing a multi-channel total variation prior to ensure that anatomical edges align precisely across different scans, sharpening the through-plane boundaries without requiring external training data (MRI Super-Resolution using Multi-Channel Total Variation).
Alternatively, the high-resolution in-plane data itself can serve as its own training signal. Because human anatomy is oriented arbitrarily relative to the scan grid, the structural patterns observed within a sharp 2D slice can teach a model how to fill in the blurry through-plane gaps. Peng et al. leveraged this concept in DA-VSR: Domain Adaptable Volumetric Super-Resolution For Medical Images, using high-resolution in-plane data as a self-supervision signal at test time to bridge the domain gap between different organs and scanners.
From Discrete Grids to Continuous Representations
Traditional super-resolution methods treat 3D volumes as discrete grids of voxels. This imposes a major limitation: the model must be trained for a specific, fixed upsampling factor (e.g., doubling the slice density), and it struggles to adapt when encountering scans with varying physical slice thicknesses.
To overcome this, recent architectures shift the problem into a continuous domain using Implicit Neural Representations (INRs). Instead of predicting a discrete grid, the network learns a continuous function:
$$f(x, y, z) \to \text{Voxel Intensity}$$
By parameterizing the volume this way, you can query the neural network at any arbitrary, non-integer coordinate along the slice axis, effectively reducing the inter-slice spacing to zero.
Discrete Grid Upsampling Implicit Neural Representation
(Fixed Factors) (Continuous Coordinate Query)
Slice 1 [ Voxel ] Slice 1 [ Voxel ]
↓ (Fixed 2x) │
Synthesized Grid Layer ├─ Query coordinate (z = 1.34)
↑ │
Slice 2 [ Voxel ] Slice 2 [ Voxel ]
Peng et al. introduced this coordinate-based approach to slice synthesis with SAINT, a spatially aware meta-network that accounts for physical voxel spacing to handle arbitrary upsampling factors efficiently (SAINT: Spatially Aware Interpolation NeTwork for Medical Slice Synthesis).
Building on this, Wang et al. combined coordinate queries with a spatial attention mechanism to dynamically focus on relevant intra-slice features, allowing for arbitrary, non-integer reduction of MRI slice spacing (Spatial attention-based implicit neural representation for arbitrary reduction of MRI slice spacing).
Because INRs can operate on a single subject's data, they eliminate cohort biases. Li et al. demonstrated that a subject-specific INR can successfully map multi-contrast relationships to synthesize high-resolution isotropic volumes using only the patient's available anisotropic scans (Single-subject Multi-contrast MRI Super-resolution via Implicit Neural Representations).
Generative Priors and Diffusion Models
When the slice gap becomes exceptionally wide—such as an $8\times$ reduction in resolution—deterministic regression models hit a mathematical wall. Because multiple high-frequency structures could theoretically fit within a large gap, standard regression models minimize their loss by averaging all possibilities, resulting in a blurry, safe guess.
To recover sharp, realistic anatomical details without manual hallucination, researchers deploy deep generative priors. Instead of asking "what is the exact pixel value here?", these models ask "what is the most anatomically plausible structure that matches the surrounding slices?".
One prominent approach frames the missing data as a signal reconstruction problem. Remedios et al. modeled the missing high-frequency information as "detail coefficients" within a perfect reconstruction filter bank, learning these coefficients directly from the scan's internal data (Deep filter bank regression for super-resolution of anisotropic MR brain images).
More recently, conditional diffusion models have emerged as a powerful tool for this task. By treating slice synthesis as a denoising process conditioned on the low-resolution slices, they generate high-fidelity structural details.
- Wang et al. developed HiFi-Diff, which utilizes a hierarchical feature conditional diffusion framework to systematically resolve the "gappy" nature of clinical 2D MRI scans (Arbitrary Reduction of MRI Inter-slice Spacing Using Hierarchical Feature Conditional Diffusion).
- Kim et al. introduced DiffusionIQT, a 3D conditional diffusion model featuring a cross-batch attention mechanism designed to capture global context and upgrade low-field, thick-slice scans to high-field quality (A 3D Conditional Diffusion Model for Image Quality Transfer - An Application to Low-Field MRI).
To further constrain these generative models and ensure clinical accuracy, researchers often combine them with structural transformers or temporal flow priors. For example, Huang et al. designed TransMRSR, which pairs the long-range modeling capabilities of Transformers with a pre-trained generative adversarial network (GAN) prior, using a self-distilled truncation trick to safely upsample thick-slice scans up to a factor of $\times8$ (TransMRSR: Transformer-based Self-Distilled Generative Prior for Brain MRI Super-Resolution).
Taking a different angle on prior learning, Wang et al. demonstrated that models pre-trained on natural videos can learn generalized spatial flow and continuity, which can then be fine-tuned via self-supervision on a patient's scan to achieve near-supervised performance in turning thick 2D slices into continuous 3D volumes (Inter-slice Super-resolution of Magnetic Resonance Images by Pre-training and Self-supervised Fine-tuning).
Through this evolution from basic interpolation to generative coordinate spaces, resolving through-plane anisotropy has shifted from a geometric impossibility to a showcase of how deep neural networks can exploit latent physical and structural redundancies.
Key Foundational Work
Registration-Based Interpolation Using a High-Resolution Image for Guidance (2004) — contributed 1 concept elements to this direction.
Deformable biomechanical models: Application to 4D cardiac image analysis (2003) — contributed 1 concept elements to this direction.
A Superresolution Framework for fMRI Sequences and Its Impact on Resulting Activation Maps (2003) — contributed 1 concept elements to this direction.