Deep Q-Network-Driven Catheter Segmentation in 3D US by Hybrid Constrained Semi-Supervised Learning and Dual-UNet

Hongxu Yang, Caifeng Shan, Alexander F. Kolen, Peter H. N. de With

Abstract

Catheter segmentation in 3D ultrasound is important for computer-assisted cardiac intervention. However, a large amount of labeled images are required to train a successful deep convolutional neural network (CNN) to segment the catheter, which is expensive and time-consuming. In this paper, we propose a novel catheter segmentation approach, which requests fewer annotations than the supervised learning method, but nevertheless achieves better performance. Our scheme considers a deep Q learning as the pre-localization step, which avoids voxel-level annotation and which can efficiently localize the target catheter. With the detected catheter, patch-based Dual-UNet is applied to segment the catheter in 3D volumetric data. To train the Dual-UNet with limited labeled images and leverage information of unlabeled images, we propose a novel semi-supervised scheme, which exploits unlabeled images based on hybrid constraints from predictions. Experiments show the proposed scheme achieves a higher performance than state-of-the-art semi-supervised methods, while it demonstrates that our method is able to learn from large-scale unlabeled images.

Introduction

In cardiac interventions such as radiofrequency (RF) ablation, clinicians must track catheters in 3D ultrasound (US) in real-time. Automated segmentation is challenging due to three primary factors: the extreme labor required for voxel-wise 3D labeling, the severe class imbalance (where the catheter occupies a tiny fraction of the volume), and the computational cost of processing high-resolution 3D data. This paper introduces a "coarse-to-fine" framework that combines reinforcement learning for localization with a semi-supervised Dual-UNet for segmentation, significantly reducing annotation requirements while maintaining high accuracy.

Methodology

The proposed framework consists of two distinct stages designed to optimize both accuracy and efficiency:

1. Coarse Localization via Deep Q-Network (DQN)

Instead of processing the entire volume, a DQN agent navigates the 3D space to identify the catheter's center. This stage requires only center-point coordinates for training, which are much easier to obtain than full voxel masks. This pre-localization step effectively filters out the background, solving the class imbalance problem.

2. Fine Segmentation via Dual-UNet and Hybrid SSL

A patch is extracted around the localized center and passed to a Dual-UNet—two independent UNet architectures. To leverage unlabeled data, the authors propose a Hybrid Semi-Supervised Learning (SSL) scheme using three constraints:

  • Intra-network Uncertainty: Uses Monte Carlo Dropout for internal consistency.
  • Inter-network Consistency: Ensures consensus between the two independent networks to avoid the biases of Mean-Teacher models.
  • Contextual Constraint: Employs an adversarial discriminator to align the distribution of predictions on unlabeled data with labeled data.
The two-stage framework: DQN for localization followed by Dual-UNet segmentation with hybrid constraints.

The model was trained and validated on an ex-vivo porcine heart dataset (88 volumes), using subsets of 10%, 20%, and 30% labeled data to evaluate semi-supervised performance.

Results and Discussion

The proposed method consistently outperformed state-of-the-art semi-supervised techniques and even surpassed fully supervised baselines.

Quantitative results showing the proposed method's performance across different labeling ratios compared to ablation studies and full supervision.

Key findings include:

  • Superior Accuracy: With only 30% labeled data (18 images), the method achieved a Dice Score (DSC) of 65.0%, outperforming a standard UNet trained on 100% labeled data (63.0%).
  • Computational Efficiency: The total inference time is approximately 1.2 seconds per volume (0.7s for localization and 0.5s for segmentation), making it suitable for near-real-time clinical use.
  • Robustness of Hybrid Constraints: The results confirm that combining intra-network, inter-network, and contextual constraints yields the best performance across all metrics (DSC, Jaccard, and 95HD).

Conclusion

By integrating reinforcement learning for localization and a novel hybrid-constrained Dual-UNet for segmentation, this framework addresses the core challenges of catheter tracking in 3D ultrasound. The approach is not only more data-efficient but also faster and more accurate than traditional supervised methods. Future work will focus on validating the system in noisier in-vivo environments.

Summary updated

Abstract

Catheter segmentation in 3D ultrasound is important for computer-assisted cardiac intervention. However, a large amount of labeled images are required to train a successful deep convolutional neural network (CNN) to segment the catheter, which is expensive and time-consuming. In this paper, we propose a novel catheter segmentation approach, which requests fewer annotations than the supervised learning method, but nevertheless achieves better performance. Our scheme considers a deep Q learning as the pre-localization step, which avoids voxel-level annotation and which can efficiently localize the target catheter. With the detected catheter, patch-based Dual-UNet is applied to segment the catheter in 3D volumetric data. To train the Dual-UNet with limited labeled images and leverage information of unlabeled images, we propose a novel semi-supervised scheme, which exploits unlabeled images based on hybrid constraints from predictions. Experiments show the proposed scheme achieves a higher performance than state-of-the-art semi-supervised methods, while it demonstrates that our method is able to learn from large-scale unlabeled images.
BibTeX
@inproceedings{yang2020deep,
  author = {Hongxu Yang and Caifeng Shan and Alexander F. Kolen and Peter H. N. de With},
  title = {{Deep Q-Network-Driven Catheter Segmentation in 3D US by Hybrid Constrained Semi-Supervised Learning and Dual-UNet}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  doi = {10.1007/978-3-030-59710-8\_63},
}
Figure