Abstract
Multi-modality image registration is one of the most underlined processes in medical image analysis. Recently, convolutional neural networks (CNNs) have shown significant potential in deformable registration. However, the lack of voxel-wise ground truth challenges the training of CNNs for an accurate registration. In this work, we propose a cross-modality similarity metric, based on the KL-divergence of image variables, and implement an efficient estimation method using a CNN. This estimation network, referred to as KLDivNet, can be trained unsupervisedly. We then embed the KLDivNet into a registration network to achieve the unsupervised deformable registration for multi-modality images. We employed three datasets, i.e., AAL Brain, LiTS Liver and Hospital Liver, with both the intra- and inter-modality image registration tasks for validation. Results showed that our similarity metric was effective, and the proposed registration network delivered superior performance compared to the state-of-the-art methods.
Introduction
Image registration—the process of aligning two or more images into a single coordinate system—is a cornerstone of medical imaging applications, from surgical planning to longitudinal disease monitoring. While "intra-modality" registration (e.g., aligning two CT scans) is relatively straightforward due to linearly correlated image intensities, "multi-modality" registration (e.g., CT to MRI) presents a significant challenge. In these scenarios, a bright pixel in one modality might correspond to a dark pixel in another, rendering standard loss functions like Mean Squared Error (MSE) or Local Normalized Cross-Correlation (LNCC) ineffective.
Traditionally, researchers have relied on Mutual Information (MI) to address cross-modality alignment. However, MI is notoriously difficult to integrate into modern deep learning pipelines. Calculating its gradient typically requires computing joint intensity histograms, a process that is computationally expensive and often mathematically non-differentiable, making backpropagation difficult. This research introduces a differentiable, learning-based similarity metric that can be trained in an unsupervised manner, bypassing the need for voxel-wise ground truth.
Methodology: The DivRegNet Framework
The proposed framework, DivRegNet, consists of two primary neural components that work in tandem to achieve accurate deformable registration:
- RegNet: This registration network is based on the VoxelMorph architecture. It takes a "moving" image ($\mathcal{M}$) and a "fixed" image ($\mathcal{F}$) as input and predicts a Displacement Vector Field (DVF). This field is used to warp the moving image into the coordinate space of the fixed image.
- KLDivNet: This is the core innovation of the paper. Rather than using a hard-coded formula for similarity, the authors employ a second CNN to estimate the similarity between the warped moving image and the fixed image. Specifically, it estimates the lower bound of the Kullback-Leibler (KL) divergence between the image distributions using the Donsker-Varadhan representation.
By treating the similarity metric as an optimization problem solved by a CNN, the entire system can be trained end-to-end. During the training phase, RegNet attempts to warp the image to maximize the similarity score produced by KLDivNet, while KLDivNet is simultaneously updated to provide a more accurate estimation of the divergence.
Experimental Validation
The researchers validated DivRegNet across three distinct datasets to test its versatility in both intra- and inter-modality tasks:
- AAL Brain: A phantom dataset generated from the Anatomical Automatic Labeling template, used for T1-T2 and PD-T2 MR registration.
- LiTS Liver: An open-source dataset of 131 CT scans, utilized for single-modality inter-subject registration.
- Hospital Liver: A private clinical dataset containing 14 CT and 19 MR images, providing a challenging real-world multi-modality task.
Results and Discussion
The proposed DivRegNet outperformed several state-of-the-art methods in both accuracy and anatomical fidelity. On the AAL Brain T1-T2 task, the model achieved a Dice score of 0.9546, surpassing traditional MI-based registration and LNCC-based CNNs. In the more complex Hospital Liver (CT to MR) task, it achieved a Dice score of 0.8351, demonstrating superior performance in aligning soft tissue structures across different imaging modalities.
Key Insights
- Neural Estimation vs. Histograms: By utilizing KLDivNet, the authors eliminate the "joint histogram" bottleneck, making the similarity metric fully compatible with stochastic gradient descent.
- Robustness to Local Optima: Analysis of the metric surfaces revealed that while common metrics like LNCC often exhibit local optima that can trap optimizers, KLDivNet provides a smoother optimization path with a clear global optimum for multi-modality tasks.
- Unsupervised Versatility: Because the metric is based on intensity distributions rather than specific labels, the same architecture handles both intra-modality and inter-modality tasks effectively without requiring manual annotations.
Conclusion
While DivRegNet offers a powerful solution for multi-modality registration, the authors note that the simultaneous training of two networks (RegNet and KLDivNet) increases hyperparameter sensitivity. Future work may explore recursive or cascaded architectures to further refine the precision of the deformation fields and reduce the computational overhead during the training phase. Nevertheless, the transition from hard-coded metrics to neural similarity estimation represents a significant step forward in unsupervised medical image analysis.
Summary updated