FireANTs: Adaptive Riemannian Optimization for Multi-Scale Diffeomorphic Registration

Rohit Jena, Pratik Chaudhari, James GeearXiv 2024 · OpenReview

Abstract

Diffeomorphic Image Registration is a critical part of the analysis in various imaging modalities and downstream tasks like image translation, segmentation, and atlas building. Registration algorithms based on optimization have stood the test of time in terms of accuracy, reliability, and robustness across a wide spectrum of modalities and acquisition settings. However, these algorithms converge slowly, are prohibitively expensive to run, and their usage requires a steep learning curve, limiting their scalability to larger clinical and scientific studies. In this paper, we develop multi-scale Adaptive Riemannian Optimization algorithms for diffeomorphic image registration. We demonstrate compelling improvements on image registration across a spectrum of modalities and anatomies by measuring structural and landmark overlap of the registered image volumes. Our proposed framework leads to a consistent improvement in performance, and from 300x up to 2000x speedup over existing algorithms. Our modular library design makes it easy to use and allows customization via user-defined cost functions.

Introduction

In medical and biological imaging, registration is the process of aligning two images—such as a patient's MRI to a standard brain atlas—to facilitate comparative analysis. Diffeomorphic matching is the gold standard for this task because it ensures transformations are smooth and invertible, preserving biological plausibility by preventing anatomical "tearing" or "folding."

Historically, researchers faced a trade-off between flexibility and speed. Traditional optimization tools like ANTs are highly generalizable but computationally expensive, often requiring hours for a single scan. Conversely, modern Deep Learning (DL) models offer rapid alignment but are often "brittle," requiring extensive training data and failing when applied to novel species or modalities. FireANTs bridges this gap, providing a training-free, GPU-accelerated framework that matches the speed of DL while maintaining the robustness of classical optimization.

Methods

FireANTs introduces Adaptive Riemannian Optimization for dense image matching, treating the space of deformations as a Lie Group. Key technical innovations include:

  • Eulerian Descent: By defining descent directions from the identity transform, the framework avoids the high computational cost of parallel transport typically required in manifold optimization.
  • Adaptive Riemannian Optimization: The authors adapted the Adam optimizer for non-Euclidean spaces, enabling the algorithm to handle ill-conditioned problems where local update requirements vary significantly across the image.
  • Jacobian-Free Approximation: A simplified descent version skips expensive matrix multiplications, significantly boosting runtime without compromising empirical accuracy.
  • Diffeomorphism-Preserving Interpolation: The framework utilizes bilinear/trilinear interpolation during multi-scale processing to preserve diffeomorphic properties, as standard bicubic interpolation can introduce anatomical folds.

Datasets and Generalization

The framework was evaluated on 14 diverse datasets comprising over 15,000 image pairs. These covered three organs (brain, lung, abdomen), seven modalities (including MRI, CT, and LSFM), and six species (human, macaque, mouse, rat, zebrafish, and sheep).

Examples of diverse imaging modalities and anatomies supported by FireANTs, including rodent brains, abdomen MR/CT, and primate neuroimaging.

Notable benchmarks included human neuroimaging (OASIS, LPBA40), lung CT (EMPIRE10), and high-resolution microscopy (fMOST mouse brain). Unlike DL models, FireANTs works "out of the box" for specialized data like larval zebrafish atlases without the need for costly retraining.

Key Results

FireANTs demonstrates that mathematically rigorous classical optimization can be highly competitive on modern hardware, achieving up to a 1200× speedup on a GPU compared to ANTs on a CPU.

  • Efficiency: Atlas generation for 55 mouse brains was reduced from 141 hours to just 22 minutes. Furthermore, the framework uses up to 10× less GPU memory than state-of-the-art DL methods like VoxelMorph, allowing for the registration of volumes 50× larger than what standard DL models can accommodate.
  • Accuracy: FireANTs secured first place on the RnR-ExM mouse brain leaderboard and outperformed baselines in fissure alignment for the EMPIRE10 lung challenge.
Quantitative comparison of registration accuracy (Dice Score) across various benchmarks.

Discussion and Limitations

While FireANTs offers significant improvements, the "Jacobian-free" method is an approximation that may be less effective in scenarios involving extreme local rotations. Additionally, while the speedup makes hyperparameter tuning feasible, users must still select parameters like learning rates and smoothing kernels. Future work may involve integrating FireANTs' geometric optimization with the feature-extraction capabilities of deep learning to create hybrid models.

Summary updated

Abstract

Diffeomorphic Image Registration is a critical part of the analysis in various imaging modalities and downstream tasks like image translation, segmentation, and atlas building. Registration algorithms based on optimization have stood the test of time in terms of accuracy, reliability, and robustness across a wide spectrum of modalities and acquisition settings. However, these algorithms converge slowly, are prohibitively expensive to run, and their usage requires a steep learning curve, limiting their scalability to larger clinical and scientific studies. In this paper, we develop multi-scale Adaptive Riemannian Optimization algorithms for diffeomorphic image registration. We demonstrate compelling improvements on image registration across a spectrum of modalities and anatomies by measuring structural and landmark overlap of the registered image volumes. Our proposed framework leads to a consistent improvement in performance, and from 300x up to 2000x speedup over existing algorithms. Our modular library design makes it easy to use and allows customization via user-defined cost functions.
BibTeX
@misc{jena2024fireants,
  author = {Rohit Jena and Pratik Chaudhari and James Gee},
  title = {{FireANTs: Adaptive Riemannian Optimization for Multi-Scale Diffeomorphic Registration}},
  year = {2024},
  url = {https://openreview.net/forum?id=NGIcEyCPIp},
}
Figure