CycleMorph: Cycle consistent unsupervised deformable image registration

Boah Kim, Dong Hwan Kim, Seong Ho Park, Jieun Kim, June-Goo Lee, Jong Chul Ye

TL;DR

CycleMorph introduces a cycle-consistent unsupervised learning framework for deformable image registration that ensures anatomical topology is preserved during transformation. By enforcing a "round-trip" constraint—requiring that a deformed image can be mapped back to its original state—the model provides highly accurate, physically plausible registrations at speeds orders of magnitude faster than traditional methods.

What Problem Does This Paper Address?

Image registration—the process of aligning two or more images of the same scene—is a cornerstone of medical imaging, used for everything from tracking tumor growth to atlas-based brain mapping. While traditional variational methods are accurate, they are computationally expensive and slow. Recent deep learning (DL) approaches have slashed processing times from minutes to seconds, but they often suffer from a "degeneracy" problem: the resulting deformation fields can be non-diffeomorphic, meaning they cause "folding" or tearing in the image that violates anatomical reality.

Existing DL methods often lack explicit constraints to guarantee that the transformation is reversible and topology-preserving. The authors of CycleMorph aim to bridge this gap by providing a fast, unsupervised method that maintains structural integrity without requiring ground-truth deformation fields for training.

Methods

The core of CycleMorph is the cycle consistency constraint, a concept popularized by CycleGAN but applied here to the geometry of image registration.

  1. Dual Networks: The system trains two convolutional neural networks (CNNs), $G_X$ and $G_Y$. $G_X$ learns the forward deformation field (moving image $X$ to target $Y$), while $G_Y$ learns the reverse.
  2. Cycle Loss: Unlike previous methods that enforce consistency on the deformation vectors themselves, CycleMorph enforces it on the pixels. It calculates a loss based on how well the "re-deformed" image (warped to the target and then warped back) matches the original source image. This acts as an implicit regularizer, preventing the network from taking "shortcuts" that result in non-physical folding.
  3. Identity Loss: To improve stability, the authors include an identity loss that penalizes the network if it tries to deform an image when the source and target are identical.
  4. Multi-scale Implementation: To handle large 3D volumes (like liver CT scans) that exceed GPU memory, the authors propose a two-stage approach: a global registration on downsampled volumes followed by a local registration on high-resolution patches.

Datasets

The authors evaluated CycleMorph across three diverse datasets to prove its flexibility:

  • Facial Expression Image (RaFD): A 2D dataset of 1,608 images used to test registration across different facial expressions (e.g., angry to happy).
  • Brain MRI (OASIS-3): A 3D dataset of 1,249 T1-weighted scans. This is a standard benchmark for atlas-based registration where individual brains are mapped to a common template.
  • Multiphase Liver CT: A challenging 3D dataset from Asan Medical Center (555 scans) featuring four different contrast-enhanced phases. This tests the model's ability to handle significant shape and contrast variations over time.

Key Insights and Ideas

  • Image-Level vs. Field-Level Consistency: The most significant "aha" moment is that imposing cycle consistency on the images is a more direct and effective way to preserve topology than imposing it on the deformation fields. It naturally forces the network to find a one-to-one mapping.
  • Implicit Diffeomorphism: While some methods use complex mathematical layers (like diffeomorphic integration) at inference time to prevent folding, CycleMorph achieves similar topological preservation through its loss function during training, keeping the inference phase lightweight.
  • Handling Contrast Variation: By using local cross-correlation (LCC) in the registration loss, the model remains robust even when the source and target images have different intensities (common in contrast-enhanced CT).

Key Results

  • Topology Preservation: CycleMorph significantly reduced "folding" compared to the baseline VoxelMorph. In brain MRI tasks, it achieved a lower percentage of non-positive Jacobian determinant values (a measure of folding) while maintaining higher accuracy.
  • Accuracy: On the OASIS-3 brain dataset, CycleMorph achieved higher Dice scores (a measure of overlap) across almost all anatomical structures compared to VoxelMorph and MS-DIRNet.
  • Clinical Utility: In the liver CT task, CycleMorph reduced the Target Registration Error (TRE) and provided more accurate measurements of tumor size differences than competing DL models.
  • Speed: While traditional methods like Elastix took ~20 minutes for 3D registration, CycleMorph’s global registration took only 4 seconds, with the full multi-scale pipeline completing in 41 seconds.

Limitations and Future Work

The authors note that while the multi-scale approach solves memory issues, it requires a two-stage training process (global then local). Future work could explore more integrated end-to-end multi-scale architectures. Additionally, while the cycle consistency provides a strong implicit regularizer, it does not mathematically guarantee a 100% fold-free transformation in every possible edge case, though it performs exceptionally well in practice. Expanding the framework to handle multi-modal registration (e.g., MRI to CT) is another logical next step.

Summary updated

Abstract

Image registration is a fundamental task in medical image analysis. Recently, deep learning based image registration methods have been extensively investigated due to their excellent performance despite the ultra-fast computational time. However, the existing deep learning methods still have limitation in the preservation of original topology during the deformation with registration vector fields. To address this issues, here we present a cycle-consistent deformable image registration. The cycle consistency enhances image registration performance by providing an implicit regularization to preserve topology during the deformation. The proposed method is so flexible that can be applied for both 2D and 3D registration problems for various applications, and can be easily extended to multi-scale implementation to deal with the memory issues in large volume registration. Experimental results on various datasets from medical and non-medical applications demonstrate that the proposed method provides effective and accurate registration on diverse image pairs within a few seconds. Qualitative and quantitative evaluations on deformation fields also verify the effectiveness of the cycle consistency of the proposed method.
BibTeX
@article{kim2021cyclemorph,
  author = {Boah Kim and Dong Hwan Kim and Seong Ho Park and Jieun Kim and June-Goo Lee and Jong Chul Ye},
  title = {{CycleMorph: Cycle consistent unsupervised deformable image registration}},
  journal = {Medical Image Analysis},
  year = {2021},
  url = {https://openreview.net/forum?id=4wLubvyBz7x},
}

Loading related content...

Figure