Deep Q-Network-Driven Catheter Segmentation in 3D US by Hybrid Constrained Semi-supervised Learning and Dual-UNet

Hongxu Yang, Caifeng Shan, Alexander F. Kolen, Peter H. N. de With

TL;DR

This paper introduces a two-stage deep learning framework for segmenting catheters in 3D ultrasound images. By combining a Deep Q-Network (DQN) for coarse localization with a "Dual-UNet" trained via semi-supervised learning, the authors achieve state-of-the-art results while significantly reducing the need for expensive, manual voxel-level annotations.

What Problem Does This Paper Address?

In cardiac interventions like RF-ablation, clinicians need to track catheters in 3D ultrasound (US) in real-time. However, automated segmentation faces three major hurdles:

  1. Annotation Cost: Voxel-wise labeling of 3D volumes is incredibly labor-intensive for medical experts.
  2. Class Imbalance: The catheter occupies a tiny fraction of the total 3D volume, making it a "needle in a haystack" problem for standard neural networks.
  3. Computational Efficiency: Exhaustively scanning a 3D volume with a patch-based segmenter is too slow for clinical use.

The authors aim to create a system that learns effectively from limited labeled data (semi-supervised learning) and processes volumes quickly using a "coarse-to-fine" strategy.

Methods

The proposed framework consists of two distinct stages:

1. Coarse Localization via Reinforcement Learning (DQN)

Instead of segmenting the whole volume, the authors use a Deep Q-Network (DQN) agent. The agent "navigates" through the 3D volume, taking steps to find the catheter's center. This mimics how a human might search for an object. Because the DQN only requires a center-point coordinate for training rather than a full voxel mask, the annotation effort is drastically reduced.

2. Fine Segmentation via Dual-UNet and Hybrid SSL

Once the DQN identifies a region of interest, a patch is extracted for segmentation. The authors use a Dual-UNet architecture—two independent UNets—trained with a novel Semi-Supervised Learning (SSL) scheme. To leverage unlabeled images, they apply three "Hybrid Constraints":

  • Intra-network Uncertainty: Uses Monte Carlo Dropout to ensure each network is internally consistent and confident.
  • Inter-network Uncertainty: Forces the two independent networks to reach a consensus, which helps prevent the model from developing the biases common in "Mean-Teacher" architectures.
  • Contextual Constraint: Uses an adversarial classifier (a discriminator) to ensure that the predictions on unlabeled data "look" like the predictions on labeled data at a high-level contextual level.

Datasets

The researchers evaluated their method on an ex-vivo porcine heart dataset:

  • Data: 88 3D US volumes collected from 8 porcine hearts placed in water tanks.
  • Ground Truth: All volumes were manually annotated at the voxel level for evaluation, and catheter centers were marked for DQN training.
  • Experimental Setup: 60 volumes were used for training (with subsets of 10%, 20%, and 30% labeled to test SSL performance) and 28 volumes were kept for testing.

Key Insights and Ideas

  • RL as a Filter: Using Reinforcement Learning for pre-localization is a powerful way to solve class imbalance. By narrowing the search space, the segmentation network can focus on high-resolution details rather than being overwhelmed by "empty" background voxels.
  • Independence Matters: Most SSL methods use a "Mean-Teacher" approach where the teacher is a moving average of the student. The authors argue this creates too much correlation between the two. By using two independent networks (Dual-UNet) and an inter-network constraint, they achieve more robust learning.
  • Efficiency Gains: The "coarse-to-fine" approach isn't just more accurate; it's faster. By avoiding an exhaustive search of the 3D volume, the system becomes viable for near-real-time applications.

Key Results

  • Superior Performance with Less Data: When trained on only 18 labeled images (30% of the training set), the proposed method achieved a Dice Score of 65.0%. This actually outperformed a standard UNet trained on the full set of 60 labeled images (63.0%).
  • Outperforming SOTA: The method consistently beat other state-of-the-art semi-supervised methods (like the $\Pi$-model or UA-MT) across all metrics (Dice, Jaccard, and 95% Hausdorff Distance).
  • Speed: The total processing time was approximately 1.2 seconds per volume (0.7s for localization + 0.5s for segmentation). This is nearly 10x faster than exhaustive patch-based methods, which typically take over 10 seconds.

Limitations and Future Work

  • Environment Complexity: The dataset was ex-vivo (tissues in water tanks). Real-world in-vivo human cardiac ultrasound is significantly noisier, contains more artifacts, and includes motion from breathing and heartbeats.
  • Model Complexity: While the Dual-UNet is effective, the authors suggest that investigating even more complex backbone architectures or refined supervised loss functions could further push the boundaries of accuracy.
  • Generalization: It remains to be seen how well the DQN agent generalizes to different types of catheters or different ultrasound probes without retraining.

Summary updated

Abstract

Catheter segmentation in 3D ultrasound is important for computer-assisted cardiac intervention. However, a large amount of labeled images are required to train a successful deep convolutional neural network (CNN) to segment the catheter, which is expensive and time-consuming. In this paper, we propose a novel catheter segmentation approach, which requests fewer annotations than the supervised learning method, but nevertheless achieves better performance. Our scheme considers a deep Q learning as the pre-localization step, which avoids voxel-level annotation and which can efficiently localize the target catheter. With the detected catheter, patch-based Dual-UNet is applied to segment the catheter in 3D volumetric data. To train the Dual-UNet with limited labeled images and leverage information of unlabeled images, we propose a novel semi-supervised scheme, which exploits unlabeled images based on hybrid constraints from predictions. Experiments show the proposed scheme achieves a higher performance than state-of-the-art semi-supervised methods, while it demonstrates that our method is able to learn from large-scale unlabeled images.
BibTeX
@inproceedings{yang2020deep,
  author = {Hongxu Yang and Caifeng Shan and Alexander F. Kolen and Peter H. N. de With},
  title = {{Deep Q-Network-Driven Catheter Segmentation in 3D US by Hybrid Constrained Semi-supervised Learning and Dual-UNet}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  doi = {10.1007/978-3-030-59710-8\_63},
}

Loading related content...

Figure