Abstract
We introduce an end-to-end deep-learning framework for 3D medical image registration. In contrast to existing approaches, our framework combines two registration methods: an affine registration and a vector momentum-parameterized stationary velocity field (vSVF) model. Specifically, it consists of three stages. In the first stage, a multi-step affine network predicts affine transform parameters. In the second stage, we use a Unet-like network to generate a momentum, from which a velocity field can be computed via smoothing. Finally, in the third stage, we employ a self-iterable map-based vSVF component to provide a non-parametric refinement based on the current estimate of the transformation map. Once the model is trained, a registration is completed in one forward pass. To evaluate the performance, we conducted longitudinal and cross-subject experiments on 3D magnetic resonance images (MRI) of the knee of the Osteoarthritis Initiative (OAI) dataset. Results show that our framework achieves comparable performance to state-of-the-art medical image registration approaches, but it is much faster, with a better control of transformation regularity including the ability to produce approximately symmetric transformations, and combining affine and non-parametric registration.
Introduction
Image registration—the process of overlaying two images to find spatial correspondences—is a cornerstone of medical AI, used for everything from tracking tumor growth to population studies. Historically, researchers faced a trade-off between traditional optimization methods and early deep learning approaches. Traditional optimization is highly accurate and mathematically "well-behaved" (smooth), but painfully slow, often taking minutes or hours per image pair. Conversely, early deep learning models are fast but often require images to be "pre-aligned" manually or via separate tools and struggle with the massive memory requirements of 3D volumes.
This paper addresses the "pre-alignment" bottleneck and the memory wall by introducing AVSM (Affine-vSVF-Mapping), an end-to-end framework that performs both global (affine) and local (non-parametric) 3D medical image registration in a single forward pass.
Methods
The proposed AVSM pipeline consists of three primary stages designed to handle large misalignments and local deformations efficiently.
- Multi-step Affine Network: Instead of predicting global alignment (rotation, scaling, translation) in one go, the network iteratively refines affine parameters over several steps. This recurrent approach provides stability for large initial misalignments.
- Momentum Generation (U-Net): The model predicts a "momentum" field at a lower resolution to save memory. By predicting momentum rather than a full displacement field, physics-based equations can be used to ensure the final transformation is smooth.
- vSVF Refinement: The momentum is converted into a vector momentum-parameterized stationary velocity field (vSVF). This component solves an advection equation to produce a diffeomorphic (topology-preserving) map. This stage can be iterated to fine-tune local warping.
A key innovation is the use of map compositions. Instead of warping the image multiple times, which causes blurring, the framework composes mathematical maps and only warps the source image once at the end.
Experimental Setup
The model was evaluated using the Osteoarthritis Initiative (OAI) dataset, consisting of 3D MRIs of human knees. Two tasks were performed:
- Longitudinal Registration: Registering images of the same patient taken at different times.
- Cross-subject Registration: Registering images from two different people, a more challenging task due to anatomical variation.
The model was trained on thousands of unlabeled image pairs and tested on 176 labeled pairs using femoral and tibial cartilage labels.
Results
AVSM matches or outperforms established methods like SyN, Demons, and NiftyReg while being significantly faster.
- Accuracy: On cross-subject registration, AVSM achieved a Dice score of 68.40, outperforming SyN (65.71) and NiftyReg (67.92).
- Speed: AVSM completes a 3D registration in 0.83 seconds on a single GPU. In contrast, SyN takes approximately 1,330 seconds on a CPU.
- Regularity: The transformations are smoother and show fewer "folds" compared to other deep learning baselines like VoxelMorph.
Discussion and Conclusion
The success of AVSM stems from several key insights. Iterative refinement within the network architecture (multi-step) significantly boosts performance over one-shot methods. The use of an inverse consistency loss ensures that registering image A to B gives the inverse of B to A, leading to more biologically plausible deformations. Furthermore, computing momentum at a lower resolution saves 7/8ths of memory and computation time without sacrificing accuracy.
While AVSM is highly effective, limitations remain, such as "folds" in regions with fundamental anatomical differences and boundary artifacts at image edges. Future work includes automating the selection of regularization parameters and testing the framework on other organs like the brain and lungs.
Summary updated