TL;DR
The paper introduces Neural Image Registration (NIR), a novel framework that uses coordinate-based neural fields to perform medical image registration via pair-wise optimization. By representing deformations as continuous functions parameterized by lightweight MLPs, NIR achieves state-of-the-art alignment accuracy and topology preservation while running significantly faster and using less memory than traditional optimization methods.
What Problem Does This Paper Address?
Medical image registration—the process of aligning two or more 3D medical scans (like MRIs) into a common coordinate space—is foundational for tasks like disease tracking and atlas-based segmentation.
Historically, this has been solved using optimization-based methods (e.g., SyN, NiftyReg) that iteratively find the best transformation for a specific pair of images. While highly accurate and reliable, they are computationally slow and rely on rigid, hand-crafted modeling assumptions. Conversely, recent learning-based methods (e.g., VoxelMorph) train deep convolutional networks to predict transformations instantly. However, these models often generalize poorly to unseen image domains or unique patient anatomy not represented in the training data.
The authors bridge this gap by asking: Can we leverage the expressive power of deep neural networks within an optimization-based framework?
Methods
NIR reformulates registration by modeling the transformation between two images as a continuous function using Neural Fields (also known as Implicit Neural Representations). Instead of predicting a dense deformation grid all at once, a lightweight Multi-Layer Perceptron (MLP) takes a single 3D coordinate as input and outputs its corresponding deformation vector.
Key technical components include:
- Coordinate Encoding & SIREN: Input coordinates are mapped to a higher-dimensional space using Fourier features, and the MLP utilizes sinusoidal activation functions (SIREN). This combination allows the network to overcome the "spectral bias" of standard networks and effectively capture high-frequency, fine-grained structural deformations.
- Two Registration Modes:
- Neural Displacement Field: Directly outputs displacement vectors for general deformable registration.
- Neural Velocity Field: Outputs a stationary velocity field, which is integrated over time using a Differentiable Ordinary Differential Equation (Neural ODE) solver. This guarantees a diffeomorphic transformation, meaning the mapping is smooth, invertible, and preserves topology (preventing anatomical structures from folding in on themselves).
- Hybrid Coordinate Sampler: Optimizing over every voxel in a 3D volume is memory-intensive. The authors discovered a fundamental trade-off between different stochastic sampling strategies: "downsize sampling" yields high alignment accuracy but poor local regularity, while "mini-patch sampling" preserves topology but struggles with global alignment. To solve this, they propose a two-phase hybrid scheme using two stacked neural fields ($NF_1$ and $NF_2$) optimized sequentially.
Datasets
The framework was evaluated on two public 3D brain MR image datasets:
- Mindboggle101: Used to evaluate surface alignment accuracy across 31 cortical regions.
- OASIS: Consisting of 416 T1-weighted brain MRIs, used to evaluate volumetric alignment accuracy across 27 anatomical structures.
Key Insights and Ideas
- Overfitting as a Feature: Rather than training a network to generalize across an entire dataset, NIR optimizes the network parameters specifically for each image pair. The network effectively "overfits" to the unique deformations of that pair, bypassing the generalization pitfalls of standard deep learning.
- The Sampling Trade-off: The choice of how you sample coordinates during stochastic gradient descent (SGD) acts as an implicit regularizer. Uniformly downsampling coordinates across the volume helps the model capture global context quickly, whereas sampling dense local patches forces the model to respect local spatial derivatives (Jacobian determinants), preserving topology.
Key Results
NIR was compared against both learning-based baselines (VoxelMorph, SYM_Net) and traditional optimization tools (SyN, NiftyReg).
- Accuracy & Regularity: The diffeomorphic variant, NIR-H-Diff (using the hybrid sampler), achieved state-of-the-art performance. On Mindboggle101, it achieved a Surface Dice score of 0.7904. On OASIS, it achieved a Volumetric Dice score of 0.8382 while maintaining an exceptionally low rate of topological folding ($J_{\le 0} = 4.55 \times 10^{-6}$).
- Efficiency: NIR is highly memory-efficient, requiring less than 3500MB of GPU memory during optimization—less than half of what learning-based models require for training.
- Speed: While slower than feed-forward deep learning inference, NIR is dramatically faster than traditional optimization methods, completing a 100-iteration optimization block in roughly 9 seconds for displacement fields and 64 seconds for velocity fields.
Limitations and Future Work
The primary limitation of NIR is its optimization runtime. Because it must optimize a new neural network from scratch for every single image pair, it cannot match the near-instantaneous inference speeds of fully supervised, feed-forward registration networks.
To address this, the authors suggest several future directions:
- Designing adaptive coordinate samplers that focus computation on hard-to-align regions with large errors.
- Using a learning-based model to provide a rapid initial global alignment, followed by NIR for fine-grained, patient-specific optimization.
- Integrating shape/surface-based registration metrics alongside intensity-based metrics within the same neural field framework.
Summary updated