A Multi-scale Optimization Learning Framework for Diffeomorphic Deformable Registration

Risheng Liu, Zi Li, Yuxi Zhang, Chenying Zhao, Hao Huang, Zhongxuan Luo, Xin Fan

Abstract

Deformable Image Registration (DIR) is a critical task in medical imaging, traditionally requiring a trade-off between the mathematical rigor of iterative optimization and the speed of deep learning. This paper introduces a novel framework that bridges this gap by unrolling a diffeomorphic optimization model into a multi-scale "propagative" network. By employing a bilevel training strategy to automatically tune hyper-parameters, the proposed method achieves state-of-the-art accuracy and topology-preserving deformations in a fraction of a second, effectively eliminating the need for manual parameter tuning across different anatomical domains.

Introduction

Deformable Image Registration (DIR) aims to establish dense spatial correspondences between images, such as aligning a patient's brain scan to a standardized atlas. Historically, this has been addressed through iterative optimization methods. While these methods are mathematically grounded and ensure diffeomorphism—smooth, invertible transformations that preserve anatomical topology—they are computationally expensive, often requiring minutes or hours per image pair.

Recent deep learning (DL) approaches have revolutionized the field by providing near-instantaneous registration. However, many DL models act as "black boxes," lacking the geometric guarantees of classical methods. Furthermore, they often require tedious manual tuning of hyper-parameters (e.g., regularization weights) which may not generalize across different modalities or organs. This paper addresses these challenges by integrating optimization principles directly into the neural network architecture and training process.

Methods

The proposed framework consists of two primary innovations: an optimization-inspired architecture and a bilevel self-tuned training strategy.

Optimization-Inspired Architecture

Instead of a generic U-Net, the authors develop a Deep Propagative Network. This architecture treats the network as a series of unrolled optimization steps, organized in a coarse-to-fine multi-scale hierarchy. It comprises four modular components:

  1. Feature Pyramid Extraction (F): Learns noise-invariant representations of the input images.
  2. Error-based Matching (M): Computes voxel-to-voxel correspondences.
  3. Context-based Regularization (R): Utilizes dilated convolutions to smooth the predicted flow fields.
  4. Constraint Module (C): Implements a "scaling and squaring" method to solve an Ordinary Differential Equation (ODE), ensuring the resulting deformation is diffeomorphic.
Overview of the proposed framework, including the feature space, optimization-inspired propagative network, and bilevel self-tuned training strategy.

Bilevel Self-Tuned Training

To solve the problem of manual hyper-parameter tuning, the authors utilize bilevel optimization. This "self-tuning" mechanism operates on two levels:

  • Lower-level: The network learns its internal weights ($w$) to minimize registration error on the training dataset.
  • Upper-level: A separate optimization process learns the optimal hyper-parameters ($\lambda$) by evaluating the model's performance on a validation set.

This approach allows the model to automatically adapt its regularization strength and similarity metrics based on the specific characteristics of the data (e.g., brain vs. liver), ensuring optimal performance without human intervention.

Experimental Setup

The framework was evaluated on several large-scale medical imaging benchmarks:

  • Brain MRI: 551 T1-weighted volumes from ADNI, ABIDE, PPMI, OASIS, and HCP for image-to-atlas registration.
  • Liver CT: 1,025 scans from MSD and BFH, with evaluation on SLIVER and LSPIG for image-to-image registration.
  • Multi-modal Testing: BraTS18 (brain tumors) and ISeg19 (infant brain) were used to test the model's robustness in fusion and data augmentation tasks.

Results and Discussion

Performance and Speed

The proposed method achieved the highest Dice scores (a measure of anatomical overlap) across nearly all tested brain structures, outperforming established tools like SyN, NiftyReg, and VoxelMorph. Remarkably, the model completes a 3D registration in approximately 0.36 seconds on a GPU. In comparison, the classical SyN algorithm requires roughly 80 minutes (4,799 seconds) for the same task.

Topology Preservation

A key advantage of the framework is its ability to produce physically plausible deformations. By embedding the ODE constraint directly into the $C$ module, the model produces nearly zero "folds" (indicated by negative Jacobian determinants). This is a significant improvement over other deep learning methods that often struggle with local singularities when deformations are large.

Comparison of registration results with and without the diffeomorphic constraint module. Note the grid folding in the "W/O Constraint" case.

Generalization and Limitations

The bilevel training strategy proved highly effective for generalization. The model transitioned from brain to liver data seamlessly, significantly outperforming "default" models that were not specifically tuned for liver tissue.

However, cross-modal registration (e.g., MRI to CT) remains a challenge. While the authors' "Ours-MIND" variant showed promise by using modality-independent descriptors, the significant appearance differences between modalities still present a hurdle compared to uni-modal cases. Future research will likely focus on complex clinical scenarios involving massive pathological changes, such as tumor resections.

Conclusion

This work demonstrates that integrating classical optimization theory with deep learning can yield registration models that are both extremely fast and mathematically robust. By automating hyper-parameter tuning through bilevel optimization, the framework provides a scalable and highly accurate solution for diverse medical imaging applications.

Summary updated

Abstract

Conventional deformable registration methods aim at solving a specifically designed optimization model on image pairs and offer a rigorous theoretical treatment. However, their computational costs are exceptionally high. In contrast, recent learning-based approaches can provide fast deformation estimation. These heuristic network architectures are fully data-driven and thus lack explicitly domain knowledge or geometric constraints, such as topology-preserving, which is indispensable to generate plausible deformations. To integrate the advantages and avoid the limitations of these two categories of approaches, we design a new learning-based framework to optimize a diffeomorphic model via multi-scale propagations. Specifically, we first introduce a generic optimization model to formulate diffeomorphic registration with both velocity and deformation fields. Then we propose a schematic optimization scheme with a nested splitting technique. Finally, a series of learnable architectures are utilized to obtain the final propagative updating in the coarse-to-fine feature spaces. We conduct two groups of image registration experiments on 3D adult and child brain MR volume datasets including image-to-atlas and image-to-image registrations. Extensive results demonstrate that the proposed method achieves state-of-the-art performance with diffeomorphic guarantee and extreme efficiency.
BibTeX
@misc{liu2020multi,
  author = {Risheng Liu and Zi Li and Yuxi Zhang and Chenying Zhao and Hao Huang and Zhongxuan Luo and Xin Fan},
  title = {{A Multi-scale Optimization Learning Framework for Diffeomorphic Deformable Registration}},
  year = {2020},
  url = {https://openreview.net/forum?id=o-gy1STggk},
}
Figure