Test-Time Training for Deformable Multi-Scale Image Registration

Wentao Zhu, Yufang Huang, Daguang Xu, Zhen Qian, Wei Fan, Xiaohui Xie

TL;DR

This paper introduces a novel "test-time training" approach for deep learning-based deformable image registration. By fine-tuning the registration model on each individual test image pair, they significantly improve generalization to noisy and complex medical images, outperforming existing methods in segmentation and dense tracking tasks.

What Problem Does This Paper Address?

Image registration, the process of aligning images, is crucial for many medical applications like surgery and analysis. Traditional methods are accurate but computationally expensive. Deep learning methods offer speed but often struggle with generalization to new, unseen data, especially when images are noisy or exhibit large deformations. This "domain shift" between training and testing data leads to a performance gap. The paper aims to bridge this gap by making deep learning registration models more robust to variations encountered in real-world medical imaging scenarios.

Methods

The core innovation is test-time training (TTT). Instead of training a model once and applying it directly, TTT fine-tunes the model's parameters on each specific test image pair. This allows the model to adapt to the unique characteristics of the input data, mitigating the domain shift problem.

To handle complex deformations, the authors employ a multi-scale strategy. This involves estimating the registration field in a coarse-to-fine manner. A U-Net architecture is used at each scale to predict a residual deformation field. These residual fields are then aggregated to form the final, accurate registration. This sequential optimization across scales helps capture both large and fine-grained movements. The framework is self-supervised, meaning it doesn't require manual ground truth registration fields for training.

Datasets

The paper evaluates their method on two main types of medical imaging data:

  • 3D Hippocampus MRI Dataset: Used for registration-based segmentation. This dataset was split into 208 training and 52 test images.
  • Echocardiograms: Used for myocardial tracking and cardiac blood flow tracking. This involved data from 19 patients for tracking and 71 patients for blood flow tracking, with a total of 3,052 and 11,462 frames respectively.

Key Insights and Ideas

  • Test-time training is a powerful tool for generalization: Fine-tuning on individual test images dramatically improves robustness to noise and deformation, a common pitfall for purely learning-based methods.
  • Multi-scale registration is essential for complex deformations: Decomposing the registration problem into multiple scales allows for better capture of both large and subtle anatomical movements, leading to more accurate results.
  • Self-supervision combined with TTT offers a practical solution: The approach avoids the need for expensive manual annotations while achieving high performance.
  • The "domain shift" is a significant challenge in medical image registration: The paper directly addresses this by adapting the model at inference time.

Key Results

  • Segmentation: The proposed method (SSMSR (1)) achieved the highest Dice scores on the hippocampus dataset, outperforming ANTs, NiftyReg, VoxelMorph, and NeurReg.
  • Dense Tracking (Myocardial and Cardiac Blood Flow): SSMSR (1) consistently outperformed ANTs and VoxelMorph in terms of both Mean Squared Error (MSE) and Normalized Local Cross-Correlation (NLCC). For instance, on myocardial tracking, SSMSR (1) achieved an MSE of 0.9206±0.3236 (compared to 15.5179±9.4637 for ANTs) and an NLCC of 5.0881±0.4107 (compared to 3.1597±1.3913 for ANTs).
  • Speed: While test-time training adds computational overhead per image pair, the overall process for a sequence was significantly faster than traditional methods like ANTs (e.g., 20x speedup over ANTs for echocardiograms).

Limitations and Future Work

  • Computational Cost of TTT: While faster than traditional methods, test-time training is inherently slower than a single forward pass of a pre-trained model like VoxelMorph. The paper notes that their multi-scale SSMSR takes around 9 minutes for a sequence, achieving a 20x speedup over ANTs.
  • Future Work Suggestions:
    • Enforcing consistent displacement fields between moving and fixed images.
  • Adapting iterative modules and improving efficiency by predicting the final registration field directly.
  • Integrating registration directly into segmentation pipelines with strategies for managing large GPU memory consumption.

Summary updated

Abstract

Registration is a fundamental task in medical robotics and is often a crucial step for many downstream tasks such as motion analysis, intra-operative tracking and image segmentation. Popular registration methods such as ANTs and NiftyReg optimize objective functions for each pair of images from scratch, which are time-consuming for 3D and sequential images with complex deformations. Recently, deep learning-based registration approaches such as VoxelMorph have been emerging and achieve competitive performance. In this work, we construct a test-time training for deep deformable image registration to improve the generalization ability of conventional learning-based registration model. We design multi-scale deep networks to consecutively model the residual deformations, which is effective for high variational deformations. Extensive experiments validate the effectiveness of multi-scale deep registration with test-time training based on Dice coefficient for image segmentation and mean square error (MSE), normalized local cross-correlation (NLCC) for tissue dense tracking tasks.
BibTeX
@inproceedings{zhu2021test,
  author = {Wentao Zhu and Yufang Huang and Daguang Xu and Zhen Qian and Wei Fan and Xiaohui Xie},
  title = {{Test-Time Training for Deformable Multi-Scale Image Registration}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  url = {https://openreview.net/forum?id=ep5QPrg437j},
}
Figure