TL;DR
This paper introduces KLDivNet, a novel unsupervised neural network for multi-modality image registration. By proposing a new cross-modality similarity metric based on KL-divergence and an efficient CNN-based estimator, they enable accurate registration without the need for ground truth data, outperforming existing methods.
What Problem Does This Paper Address?
Medical image analysis often requires aligning images from different sources or at different times, a process known as image registration. Deformable registration, which allows for complex, non-rigid transformations, is particularly crucial for capturing subtle anatomical changes. While deep learning (DL) has shown great promise in this area, a major hurdle for training DL models is the lack of voxel-wise ground truth for deformable registration. This is especially challenging for multi-modality registration (e.g., aligning CT and MRI scans) because the intensity values in different modalities don't directly correspond. Traditional similarity metrics like Sum of Squared Differences (SSD) fail here, and while Mutual Information (MI) is a good candidate, its computation via joint histograms is computationally expensive and difficult to integrate into DL frameworks.
Methods
The core innovation lies in the KLDivNet, a CNN designed to estimate a cross-modality similarity metric. Instead of directly computing Mutual Information (MI), the authors leverage the fact that MI can be expressed as a KL-divergence between the joint distribution and the product of marginal distributions. They then employ the Donsker-Varadhan variational representation of KL-divergence, which provides a lower bound that can be maximized. This maximization problem is solved by training a CNN (KLDivNet) to estimate a function Φ.
The KLDivNet itself has a Y-shaped architecture. It takes a pair of images (a fixed image and a transformed moving image) as input. To compute the similarity, it estimates two terms: the expected value of Φ and the expected value of e raised to the power of Φ. The latter term is computed by shuffling the voxels of the fixed image to create an independent sample, thus avoiding direct computation of joint distributions.
This KLDivNet is then integrated into a registration network, dubbed DivRegNet. DivRegNet uses a U-Net-like architecture (similar to VoxelMorph) to predict the deformation field. Crucially, DivRegNet is trained end-to-end in an unsupervised manner, using the KLDivNet's similarity score as the loss function. During training, both the registration network and the KLDivNet are optimized. For inference (testing), only the registration network is used.
Datasets
The authors evaluated their method on three datasets:
- AAL Brain: A synthetic dataset generated from the Anatomical Automatic Labeling (AAL) template, containing T1, T2, and PD MR images. For augmentation, 250 T2 images were created using simulated deformation fields.
- LiTS Liver: An open-source dataset primarily for liver lesion segmentation, consisting of 131 CT images.
- Hospital Liver: A dataset of CT and MR scans from different patients, with 14 CT and 19 MR images.
Key Insights and Ideas
- KL-divergence as a bridge to unsupervised multi-modality registration: By reformulating MI as a KL-divergence and using a variational lower bound, the authors create a pathway to an intensity-based similarity metric that is suitable for multi-modality registration and amenable to neural network estimation.
- Unsupervised learning without ground truth: The KLDivNet's ability to estimate similarity without explicit voxel-wise correspondences is key to enabling unsupervised training of the entire registration pipeline.
- The limitations of LNCC for multi-modality: The experiments clearly demonstrate that Local Normalized Cross-Correlation (LNCC), a common metric for intra-modality registration, performs poorly when applied to multi-modality tasks, highlighting the need for more robust metrics.
- The impact of MI bin settings: For MI-based registration, the number of histogram bins used for estimation significantly impacts performance, making it less robust and requiring careful tuning, unlike the proposed KLDivNet.
Key Results
- Superior performance in multi-modality registration: DivRegNet consistently outperformed other methods, including FFD SEMI and RegNet with LNCC and MI losses, across various metrics (Dice, ASD, HD) on the AAL Brain dataset for T1-T2 and PD-T2 registration.
- Competitive performance in single-modality registration: For T2-T2 registration, DivRegNet achieved state-of-the-art results, outperforming other methods.
- Effective similarity metric: The KLDivNet's similarity metric was shown to be effective for both intra- and inter-modality registration, producing perfect global optima in illustrative plots, unlike LNCC which suffered from local optima.
- Efficiency: While not explicitly quantified in terms of speedup, the paper implies that the CNN-based estimation is more efficient than traditional methods requiring complex iterative optimization or histogram computations.
Limitations and Future Work
- MI bin sensitivity: The paper notes that the performance of MI-based methods is sensitive to the number of histogram bins, which requires manual tuning. While their KLDivNet avoids this, it's a limitation of the comparative MI methods.
- FFD SEMI's HD advantage: For the AAL Brain dataset, FFD SEMI achieved the best Hausdorff Distance (HD), suggesting that its deformation fields might be more realistic, even if overall accuracy was lower. This hints at potential areas for improvement in the smoothness or realism of DivRegNet's deformations.
- Future work: The authors suggest that their framework has great potential for multi-modality registration, implying further applications and refinements are possible. The release of code on GitHub also facilitates future research building upon their work.
Summary updated