Coordinate Translator for Learning Deformable Medical Image Registration

Yihao Liu, Lianrui Zuo, Shuo Han, Jerry L. Prince, Aaron Carass

TL;DR

This paper introduces "Coordinate Translator," a novel module that decouples feature extraction from coordinate system understanding in deep learning-based medical image registration. Their proposed network, "im2grid," uses this module to achieve state-of-the-art performance in unsupervised 3D MR image registration, outperforming existing methods both qualitatively and quantitatively.

What Problem Does This Paper Address?

Medical image registration, particularly deformable registration, is crucial for aligning medical scans (e.g., MRIs) to compare anatomical structures over time or across individuals. Traditional deep learning (DL) methods for this task often rely on Convolutional Neural Networks (CNNs) to directly predict displacement fields. However, these CNNs must simultaneously learn to extract image features and understand the spatial coordinate systems of the images. The authors argue that this dual responsibility is a significant challenge for CNNs, limiting their registration accuracy. They aim to alleviate this burden by separating these tasks, allowing CNNs to focus on feature extraction while a new module handles coordinate understanding.

Methods

The core innovation is the Coordinate Translator, a differentiable module designed to identify matched features between two images and output precise coordinate correspondences without requiring explicit training for this specific task. It essentially "unloads" the burden of understanding image coordinate systems from the CNN.

The authors then propose im2grid, a novel deformable registration network that leverages multiple Coordinate Translators. im2grid uses a CNN encoder to extract hierarchical features from both the fixed and moving images. These multi-level features are then fed into Coordinate Translators, which generate multi-scale sampling grids. These grids are composed in a coarse-to-fine manner to produce the final deformation field. A key aspect is that the CNN encoder in im2grid is trained solely for feature extraction, while the Coordinate Translator handles the matching and correspondence tasks.

A Positional Encoding Layer is also introduced, which combines positional information with image features. This layer helps initialize the Coordinate Translator to produce an identity grid at the start of training, ensuring that the initial output is a sensible, non-deforming transformation.

Datasets

The paper uses the following datasets for training and evaluation:

  • OASIS3: T1-weighted MR images of the human brain. Used for training, validation, and testing.
  • IXI: Brain MR images. Used for training, validation, and testing.
  • Learn2Reg 2021 Task 3: A benchmark dataset for inter-subject brain MRI registration, used for evaluation.

Key Insights and Ideas

  • Decoupling Feature Extraction and Coordinate Understanding: The fundamental insight is that forcing CNNs to learn both feature extraction and coordinate system understanding simultaneously is suboptimal for registration. Separating these tasks allows each component to excel at its intended function.
  • Coordinate Translator as a "Smart Matcher": The Coordinate Translator acts as a specialized module that can efficiently find corresponding points in feature maps, effectively translating feature similarities into precise spatial relationships.
  • Hierarchical Registration with Coordinate Correspondences: im2grid's coarse-to-fine strategy, where Coordinate Translators operate on multi-scale features, allows for efficient and accurate registration by progressively refining the deformation.
  • Generalizability of Coordinate Translator: The authors suggest that the Coordinate Translator is a general module that can be applied to various tasks requiring image input and coordinate output, not just medical image registration.

Key Results

  • State-of-the-Art Performance: im2grid significantly outperforms existing state-of-the-art DL and non-DL methods on both OASIS3 and IXI datasets in terms of Dice Similarity Coefficient (DSC), a measure of segmentation overlap.
  • Improved Diffeomorphism: The generated deformation fields from im2grid have fewer voxels with negative Jacobian determinants compared to other DL methods that directly output deformation fields, indicating a more regular and less topologically problematic transformation.
  • Quantitative Improvements: On the OASIS3 dataset, im2grid achieved a DSC of 0.908 ± 0.023, outperforming methods like Voxelmorph (0.883 ± 0.040) and ViT-V-Net (0.872 ± 0.042). Similar improvements were observed on the IXI dataset.
  • Learn2Reg Challenge Performance: im2grid-Lite achieved competitive results on the Learn2Reg 2021 Task 3 validation dataset, demonstrating its effectiveness on a challenging benchmark.

Limitations and Future Work

  • Not Explicitly Diffeomorphic: While im2grid generates more regular transformations than some DL methods, it does not have an explicit guarantee of being diffeomorphic. However, the authors note that even theoretically diffeomorphic algorithms can produce non-diffeomorphic transformations due to interpolation errors.
  • Potential for Broader Applications: The authors believe the Coordinate Translator module can be beneficial for many tasks beyond registration that involve image input and coordinate output, suggesting future work in exploring these applications.
  • Instance-Specific Optimization: The paper mentions that instance-specific optimization (as used in some comparison methods) could potentially boost performance, but their method focuses on training using only the training set, assuming such fine-tuning might not be available during deployment.

Summary updated

Abstract

The majority of deep learning (DL) based deformable image registration methods use convolutional neural networks (CNNs) to estimate displacement fields from pairs of moving and fixed images. This, however, requires the convolutional kernels in the CNN to not only extract intensity features from the inputs but also understand image coordinate systems. We argue that the latter task is challenging for traditional CNNs, limiting their performance in registration tasks. To tackle this problem, we first introduce Coordinate Translator, a differentiable module that identifies matched features between the fixed and moving image and outputs their coordinate correspondences without the need for training. It unloads the burden of understanding image coordinate systems for CNNs, allowing them to focus on feature extraction. We then propose a novel deformable registration network, im2grid, that uses multiple Coordinate Translator's with the hierarchical features extracted from a CNN encoder and outputs a deformation field in a coarse-to-fine fashion. We compared im2grid with the state-of-the-art DL and non-DL methods for unsupervised 3D magnetic resonance image registration. Our experiments show that im2grid outperforms these methods both qualitatively and quantitatively.
BibTeX
@inproceedings{liu2022coordinate,
  author = {Yihao Liu and Lianrui Zuo and Shuo Han and Jerry L. Prince and Aaron Carass},
  title = {{Coordinate Translator for Learning Deformable Medical Image Registration}},
  booktitle = {MMMI @ MICCAI},
  year = {2022},
  doi = {10.1007/978-3-031-18814-5\_10},
}
Figure