Fast Diffeomorphic Image Registration using Patch based Fully Convolutional Networks

Jiong Wu, Shuang Zhou, Lin Li, Xin Wang, Wenxue Tan

TL;DR

This paper introduces a novel unsupervised deep learning framework for fast and accurate diffeomorphic medical image registration. By focusing on image patches and incorporating a new differential operator, their method achieves superior registration accuracy and preserves topological information better than existing approaches, all while maintaining a promising speed.

What Problem Does This Paper Address?

Medical image registration is crucial for tasks like segmentation and fusion, ensuring that images from different times or modalities align correctly. A key requirement is "diffeomorphism," meaning the transformation is smooth, invertible, and preserves the topology of the anatomy. Traditional methods are accurate but computationally expensive. While deep learning (specifically Fully Convolutional Networks or FCNs) offers speed, existing unsupervised learning-based FCNs often extract features at the whole-image level, which can limit their accuracy, especially for complex deformations. This paper aims to bridge this gap by proposing a faster, more accurate, and topologically robust unsupervised registration method using FCNs.

Methods

The core of the proposed method is a patch-based Fully Convolutional Network (FCN) for registration. Instead of processing the entire image at once, the FCN learns to register smaller image patches. This patch-level feature extraction is hypothesized to capture local details more effectively.

A key novelty is the introduction of a differential operator integrated into the FCN's loss function. This operator, inspired by conventional Large Deformation Diffeomorphic Metric Mapping (LDDMM) methods, helps to enhance the smoothness and desirable diffeomorphic properties of the predicted velocity fields. The loss function balances image matching (using Normalized Cross-Correlation or NCC) with the smoothness of the velocity field, controlled by a hyperparameter a.

To handle large deformations, the authors employ a coarse-to-fine registration strategy. This involves multiple stages, where a is gradually decreased. This means the initial stages focus on smoother velocity fields (capturing large-scale deformations), and subsequent stages refine the registration with progressively less smooth fields (capturing finer details). The FCN architecture itself is a 3D UNet-style encoder-decoder with skip connections, utilizing 3D convolutions and Leaky ReLU activations.

Datasets

  • Training: 370 T1-weighted MRI images from the OASIS dataset were used, forming 362 training image pairs. 5 pairs were used for validation.
  • Testing:
    • Mindboggle-101 dataset (101 T1-weighted images with 50 manually delineated cortical structures).
  • MICCAI 2012 Multi-Atlas Labelling Challenge (MALC) dataset (35 T1-weighted images with 134 manually delineated brain structures).

The authors also mention that the datasets were preprocessed using FreeSurfer for skull-stripping, automatic segmentation, and affine spatial normalization.

Key Insights and Ideas

  • Patch-level processing is key for accuracy: By shifting from whole-image to patch-level feature extraction, the FCN can better capture local image nuances, leading to improved registration accuracy.
  • A novel differential operator enhances diffeomorphic properties: Integrating a specific differential operator into the loss function provides a more principled way to enforce smoothness and invertibility of the transformation, going beyond simple L2 regularization.
  • Coarse-to-fine with varying smoothness is effective for large deformations: The multi-stage approach, where the degree of smoothness is progressively reduced, allows the framework to handle both large and fine-grained deformations effectively.

Key Results

The proposed method was compared against three state-of-the-art approaches: a traditional method (SyN) and two unsupervised learning-based methods (Diff-VM and SYMNet).

  • Accuracy: The proposed method achieved superior registration accuracy, as measured by average Dice Similarity Coefficient (DSC), on both the MICCAI2012 and Mindboggle101 datasets. The improvement was 1.1% and 3.9% higher than Diff-VM, respectively.
  • Topology Preservation: The method maintained the number of voxels with non-positive Jacobian determinants close to zero, indicating excellent preservation of topology.
  • Speed: The average running time for the proposed method was 1.51 seconds, which is significantly faster than SyN (1039s) and competitive with other learning-based methods (Diff-VM: 0.517s, SYMNet: 0.414s).

Qualitatively, example MR slices showed that the deformed image from their method was the closest to the fixed image compared to the other methods.

Limitations and Future Work

  • Limitations: While the method is fast, the paper notes that the average running time of 1.51 seconds is slightly slower than some other learning-based methods (Diff-VM, SYMNet). The authors also mention that the accuracy improvement over Diff-VM is modest (1.1% and 3.9%), suggesting room for further enhancement.
  • Future Work: Not explicitly detailed in the provided text, but the success of the patch-based approach and the novel differential operator suggests potential for further exploration in different medical imaging modalities or more complex registration tasks. The authors' focus on unsupervised learning also points towards continued efforts in reducing reliance on ground truth data.

Summary updated

Abstract

Diffeomorphic image registration is a fundamental step in medical image analysis, owing to its capability to ensure the invertibility of transformations and preservation of topology. Currently, unsupervised learning-based registration techniques primarily extract features at the image level, potentially limiting their efficacy. This paper proposes a novel unsupervised learning-based fully convolutional network (FCN) framework for fast diffeomorphic image registration, emphasizing feature acquisition at the image patch level. Furthermore, a novel differential operator is introduced and integrated into the FCN architecture for parameter learning. Experiments are conducted on three distinct T1-weighted magnetic resonance imaging (T1w MRI) datasets. Comparative analyses with three state-of-the-art diffeomorphic image registration approaches including a typical conventional registration algorithm and two representative unsupervised learning-based methods, reveal that the proposed method exhibits superior performance in both registration accuracy and topology preservation.
BibTeX
@misc{wu2024fast,
  author = {Jiong Wu and Shuang Zhou and Lin Li and Xin Wang and Wenxue Tan},
  title = {{Fast Diffeomorphic Image Registration using Patch based Fully Convolutional Networks}},
  year = {2024},
  eprint = {2404.04244},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.04244},
}
Figure