4D‐CT deformable image registration using unsupervised recursive cascaded full‐resolution residual networks

Lei Xu, Ping Jiang, Tiffany Tsui, Junyan Liu, Xiping Zhang, Lequan Yu, Tianye NiuBioengineering 2023 · DOI

TL;DR

This paper introduces RCFRR-Net, a novel unsupervised deep learning model for registering 4D-CT scans, crucial for tracking organ motion during medical imaging. By using a cascaded architecture and a full-resolution residual network, it achieves state-of-the-art accuracy without needing ground truth deformation data, making it a promising tool for clinical applications.

What Problem Does This Paper Address?

Accurate medical image registration is vital for many clinical applications, especially in radiation oncology and diagnostics, where it helps track how organs move over time. Four-dimensional computed tomography (4D-CT) captures this motion, but registering these dynamic scans, particularly in the abdomen where motion can be complex and image contrast can be low, is challenging. Traditional registration methods are often slow and struggle with large deformations. While deep learning has shown promise, many methods require extensive labeled data (ground truth deformation fields), which are difficult and expensive to obtain. This paper tackles the problem of achieving accurate and efficient deformable image registration for 4D-CT scans using an unsupervised deep learning approach, eliminating the need for ground truth data.

Methods

The core of this work is the Recursive Cascaded Full-Resolution Residual Network (RCFRR-Net). This architecture is designed to handle large deformations progressively. It consists of three independent registration subnetworks, each building upon the output of the previous one. This "cascaded" approach allows the network to first capture global deformations and then refine them in subsequent stages.

What's novel here is the combination of:

  • Unsupervised Learning: The network is trained solely on pairs of fixed and moving images, without requiring pre-computed deformation vector fields (DVFs). This significantly reduces the burden of data annotation.
  • Full-Resolution Residual Network (FRRN): Each subnetwork utilizes an FRRN architecture. FRRNs are designed to process features at full resolution, preserving fine details and boundary information, which is crucial for accurate registration. They achieve this by maintaining two parallel streams: a residual stream for high-resolution details and a pooling stream for contextual information.
  • Recursive Cascading: The three subnetworks are trained jointly, allowing them to cooperate and perform progressive registration. The loss function, comprising image similarity and DVF regularization, is calculated based on the final warped image and the fixed image.

Datasets

The RCFRR-Net was trained and evaluated on a variety of datasets:

  • Internal 4D-CT Dataset: A retrospective dataset from 13 patients, acquired using Siemens SOMATOM Definition AS. This dataset was used for training. It included 11 phases per respiratory cycle.
  • DIRLAB 4D-CT Dataset: A public dataset used for testing generalization. It consists of 3D images from 3 cases, each with 10 phases.
  • 4D-CBCT Dataset: Another public dataset used for testing generalization, specifically from the "Sparse-view Reconstruction Challenge for Four-dimensional Cone-beam CT." These images have higher noise and artifacts compared to CT images.

Key Insights and Ideas

  • Unsupervised Learning is Viable for Complex Registration: The success of RCFRR-Net demonstrates that high-quality deformable image registration can be achieved without ground truth DVFs, opening doors for broader application and easier data acquisition.
  • Cascading for Progressive Refinement: Breaking down the registration into multiple stages (cascades) allows the network to tackle complex deformations more effectively than a single-stage approach. Each cascade refines the deformation from the previous one.
  • Full-Resolution Features are Key: The use of FRRNs is critical for preserving fine anatomical details and accurate boundary alignment, which is often lost in downsampling-then-upsampling architectures.
  • Generalization Across Modalities and Noise Levels: The model's strong performance on both CT and the more challenging 4D-CBCT datasets highlights its robustness and generalizability.

Key Results

The RCFRR-Net consistently outperformed established methods like the Demon algorithm, VoxelMorph, and a standard recursive cascaded network (RC-Net) across various metrics (RMSE, NCC, SSIM, Dice Score) on all tested datasets.

  • Significant RMSE Reduction: On the internal dataset, the average RMSE was reduced from 82.52 HU (before registration) to 10.88 HU with RCFRR-Net.
  • High Similarity Scores: NCC and SSIM scores reached near-perfect levels (e.g., NCC of 0.9999 ± 0.0001 and SSIM of 0.8762 ± 0.0258 for global images).
  • Robust ROI Performance: Even in challenging regions of interest (like the liver and stomach) with significant motion, RCFRR-Net achieved superior results, with Dice scores reaching up to 0.9579 ± 0.0111 for ROI-2.
  • Competitive Runtime: While slightly slower than VoxelMorph, RCFRR-Net's runtime (around 1.9 seconds per image on a GPU) is still orders of magnitude faster than traditional methods and acceptable for clinical use.

Limitations and Future Work

  • Patch-Based Training: The current training was performed on downsampled global images. The authors suggest that future work could explore more advanced patch extraction methods to potentially improve performance, especially for handling variations in organizational structure within paired image patches.
  • Loss Function Uniformity: The current loss function treats all regions of the image equally. Future work could incorporate adaptive weights for image similarity and DVF regularization, potentially considering the varying physiological motion characteristics of different anatomical regions (e.g., the lung).
  • Extending to Other DIR Tasks: The authors express confidence that the proposed architecture could be extended to other deformable image registration tasks beyond abdominal 4D-CT.

Summary updated

Abstract

A novel recursive cascaded full-resolution residual network (RCFRR-Net) for abdominal four-dimensional computed tomography (4D-CT) image registration was proposed. The entire network was end-to-end and trained in the unsupervised approach, which meant that the deformation vector field, which presented the ground truth, was not needed during training. The network was designed by cascading three full-resolution residual subnetworks with different architectures. The training loss consisted of the image similarity loss and the deformation vector field regularization loss, which were calculated based on the final warped image and the fixed image, allowing all cascades to be trained jointly and perform the progressive registration cooperatively. Extensive network testing was conducted using diverse datasets, including an internal 4D-CT dataset, a public DIRLAB 4D-CT dataset, and a 4D cone-beam CT (4D-CBCT) dataset. Compared with the iteration-based demon method and two deep learning-based methods (VoxelMorph and recursive cascaded network), the RCFRR-Net achieved consistent and significant gains, which demonstrated that the proposed method had superior performance and generalization capability in medical image registration. The proposed RCFRR-Net was a promising tool for various clinical applications.
BibTeX
@inproceedings{xu2023dct,
  author = {Lei Xu and Ping Jiang and Tiffany Tsui and Junyan Liu and Xiping Zhang and Lequan Yu and Tianye Niu},
  title = {{4D‐CT deformable image registration using unsupervised recursive cascaded full‐resolution residual networks}},
  booktitle = {Bioengineering},
  year = {2023},
  doi = {10.1002/btm2.10587},
}
Figure