Abstract
This work introduces KidneyRegNet, a novel deep registration pipeline designed for the real-time alignment of 3D CT volumes and 2D ultrasound (U/S) sequences during free breathing. The system utilizes a dual-network approach: a feature network (ULBNet) with handcrafted texture layers to bridge the semantic gap between modalities, and a hierarchical encoder-decoder registration network. By employing a "one-cycle" unsupervised transfer learning strategy, the model adapts to patient-specific breathing patterns in minutes. Experimental results on a large dataset of 132 U/S sequences and 274 CT images demonstrate a mean contour distance (MCD) of 0.94 mm for CT-to-US registration, significantly outperforming existing state-of-the-art methods.
Introduction
In image-guided interventions, aligning a high-resolution pre-operative 3D CT scan with live 2D ultrasound is critical for accurate targeting. However, 3DCT-2DUS registration faces significant challenges: the dimensionality gap between 3D and 2D data, the modality gap (differing appearances of CT and U/S), and the continuous deformation caused by respiratory motion. Traditional methods often fail to provide the speed or accuracy required for clinical environments. KidneyRegNet addresses these hurdles through a specialized architecture and a two-step training paradigm.
Methods
1. KidneyRegNet Architecture
The pipeline consists of two primary components designed to handle the complexities of multi-modal registration:
- Feature Network (ULBNet): This 5-level U-Net extracts shared feature maps from both CT and U/S inputs. It incorporates Local Binary Convolution (LBC) layers to focus on anatomical textures, effectively normalizing the disparate visual styles of the two modalities into a common mathematical representation.
- Registration Network: Utilizing an encoder-decoder structure, this network performs hierarchical regression. Rather than predicting a single global transformation, it estimates parameters across multiple resolutions. This prevents the model from settling into local optima and ensures stability during the registration of moving organs.
2. Training and Loss Functions
The model is trained using a Feature-Image-Motion (FIM) loss function. This composite loss includes Dice coefficients for shape alignment, Modality Independent Neighbourhood Descriptors (MIND) for local detail, and smoothing constraints to ensure the predicted motion remains physiologically plausible.
To overcome the scarcity of paired CT-US data, the authors employed a two-step training strategy:
- Pre-training: The model is trained on synthetic data generated by modeling kidney motion with Gaussian distributions.
- One-Cycle Transfer Learning: Upon clinical application, the model is refined using the first breathing cycle of the specific patient. This "tune-up" reduces on-site preparation time from 46 minutes to approximately 2–3 minutes.
Key Results
KidneyRegNet was evaluated against several state-of-the-art (SOTA) models, including VoxelMorph and C2FViT. The proposed method achieved a Mean Contour Distance (MCD) of 0.94 mm for CT-to-US registration and 1.15 mm for CT-to-reference-CT registration.
As shown in the results table, the two-step learning approach (incorporating patient-specific transfer learning) significantly reduces error compared to one-step learning. KidneyRegNet consistently outperformed other models, particularly in scenarios involving large respiratory transformations, where it maintained sub-millimeter precision.
Conclusion
KidneyRegNet provides a robust solution for real-time 3DCT-2DUS registration. By combining hierarchical regression with a rapid unsupervised transfer learning strategy, the method achieves the sub-millimeter accuracy necessary for high-precision medical applications. Future work will focus on improving robustness against poor initial segmentations and incorporating non-rigid deformation modeling to further enhance registration fidelity during complex breathing cycles.
Summary updated