Unsupervised Deformable Image Registration Using Cycle-Consistent CNN

Boah Kim, Jieun Kim, June-Goo Lee, Dong Hwan Kim, Seong Ho Park, Jong Chul Ye

This paper, presented by researchers from KAIST and the University of Ulsan, tackles a fundamental challenge in medical imaging: how to accurately and quickly align 3D scans taken at different times or during different physiological phases.

TL;DR

The authors propose a novel unsupervised deep learning framework for 3D deformable image registration that utilizes cycle-consistency to ensure mappings are reversible and physically plausible. Their method achieves near state-of-the-art accuracy while being roughly 100 times faster than traditional classical methods, making it a strong candidate for real-time clinical applications.

What Problem Does This Paper Address?

Medical image registration is the process of overlaying two or more images (like CT scans) to track changes, such as tumor growth. This is difficult because patients move, breathe, and their internal organs deform.

Currently, there are two main approaches:

  1. Classical Variational Methods: These are highly accurate but computationally expensive, often taking many minutes to process a single pair of 3D volumes.
  2. Supervised Deep Learning: These are fast but require "ground truth" deformation fields for training—data that is nearly impossible to obtain accurately for real human anatomy.

Existing unsupervised deep learning methods (like VoxelMorph) solve the data problem but often suffer from "folding" (non-physical mappings where multiple points in one image map to a single point in another) and lack a guarantee that the registration is reversible.

Methods

The researchers built their framework on the VoxelMorph architecture (a U-Net style encoder-decoder) but introduced two critical components to improve performance:

  • Cycle-Consistent Learning: Instead of training one network to map Image A to Image B, they train two networks ($G_{AB}$ and $G_{BA}$) simultaneously. The "cycle" constraint dictates that if you warp Image A to match Image B, and then warp that result back using the second network, you should end up with the original Image A. This forces the networks to learn a one-to-one, reversible mapping.
  • Identity Loss: To ensure the network doesn't "hallucinate" movement in stationary parts of the body, they include a loss term that penalizes the network if it tries to deform an image when the source and target are identical.
  • Similarity Metric: They used cross-correlation as their primary loss function. This is crucial for multiphase CT scans where the contrast of tissues changes significantly (e.g., before and after a contrast agent is injected).

Datasets

The study used a substantial clinical dataset:

  • Source: Multiphase liver CT scans from 605 patients at Asan Medical Center, Seoul.
  • Phases: Each patient had four scans: unenhanced, arterial, portal, and 180-second delayed phases.
  • Split: 555 scans for training and 50 for testing.
  • Evaluation: Performance was measured using Target Registration Error (TRE) based on 20 anatomical landmarks identified by radiologists, as well as tumor size estimation.

Key Insights and Ideas

  • Cycle-Consistency as a Regularizer: The most significant "aha moment" is that cycle-consistency acts as a powerful topological regularizer. By requiring the transformation to be reversible, the network naturally avoids creating "folds" in the deformation field without needing complex mathematical constraints.
  • Speed-Accuracy Trade-off: While the method is slightly less accurate than the best classical methods (like Elastix), the trade-off is a massive gain in speed. In a clinical setting, waiting 10 seconds is vastly preferable to waiting 20 minutes.
  • Robustness to Contrast: By using cross-correlation rather than simple pixel-wise intensity differences, the model successfully ignores changes in brightness caused by contrast agents and focuses purely on the underlying anatomy.

Key Results

  • Speed: The proposed method takes only 10 seconds to register a 3D volume, compared to 19.6 minutes for the classical Elastix software.
  • Topology Preservation: The cycle-consistent model significantly reduced "folding" problems. It showed a much lower percentage of voxels with non-positive Jacobian determinants (0.0175%) compared to the baseline VoxelMorph (0.0327%).
  • Accuracy: The method achieved a TRE of 3.76–4.91 mm. While slightly higher than Elastix (~3 mm), it was a marked improvement over the standard unsupervised VoxelMorph (~5.3–6.6 mm).
  • Clinical Utility: The method provided more accurate estimations of cancer size in delayed-to-portal phase registrations compared to other deep learning baselines.

Limitations and Future Work

  • Accuracy Gap: There is still a small accuracy gap between this deep learning approach and the state-of-the-art classical variational methods.
  • Organ Specificity: The paper focuses exclusively on the liver. While the principles are general, the model's performance on other organs with different deformation characteristics (like the lungs or heart) remains to be seen.
  • Hyperparameter Sensitivity: The balance between registration loss, cycle loss, and identity loss requires careful tuning, which might vary depending on the specific medical imaging task.

Summary updated

Abstract

Medical image registration is one of the key processing steps for biomedical image analysis such as cancer diagnosis. Recently, deep learning based supervised and unsupervised image registration methods have been extensively studied due to its excellent performance in spite of ultra-fast computational time compared to the classical approaches. In this paper, we present a novel unsupervised medical image registration method that trains deep neural network for deformable registration of 3D volumes using a cycle-consistency. Thanks to the cycle consistency, the proposed deep neural networks can take diverse pair of image data with severe deformation for accurate registration. Experimental results using multiphase liver CT images demonstrate that our method provides very precise 3D image registration within a few seconds, resulting in more accurate cancer size estimation.
BibTeX
@inproceedings{kim2019unsupervised,
  author = {Boah Kim and Jieun Kim and June-Goo Lee and Dong Hwan Kim and Seong Ho Park and Jong Chul Ye},
  title = {{Unsupervised Deformable Image Registration Using Cycle-Consistent CNN}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2019},
  url = {https://openreview.net/forum?id=RxC3HL5zBv},
}

Loading related content...

Figure