TL;DR
This paper introduces VibNet, a novel deep learning framework that uses external needle vibrations to significantly improve the detection of needles in ultrasound (US) images, especially when they are nearly invisible. By analyzing the frequency domain characteristics of these vibrations, VibNet achieves more robust and accurate needle detection compared to existing methods, even in challenging scenarios.
What Problem Does This Paper Address?
Ultrasound-guided interventions, like biopsies and regional anesthesia, are common due to their real-time nature and lack of radiation. However, accurately detecting and tracking the needle in US images is a major challenge. This is due to inherent US artifacts like speckle noise, needle-like artifacts from tissue interfaces, and reduced visibility, particularly in deeper tissues. While some specialized needles and advanced US techniques exist, they have limitations like cost, complexity, or restricted applicability. Existing deep learning methods often rely heavily on image intensity, which fails when the needle's visibility is compromised. This paper addresses the critical need for a robust and accurate needle detection method that works even when the needle is barely visible, improving safety and efficiency in US-guided procedures.
Methods
VibNet is an end-to-end deep learning framework designed to leverage the subtle, periodic vibrations externally applied to the needle shaft. The core idea is that these vibrations create distinct motion patterns in the US image sequence that are more robustly captured in the frequency domain than in the image intensity domain.
The framework consists of three main modules:
- Temporal Feature Extraction: A pre-trained CNN encoder (originally for video magnification) is used to extract motion representations from sequential US images. This module captures the subtle temporal changes caused by the vibration.
- Frequency Feature Extraction & Aggregation: The temporal features are then transformed into the frequency domain using a Discrete Fourier Transform (DFT)-inspired approach, approximated by 1D convolutions. This step aggregates frequency features, making them less susceptible to image intensity variations.
- Needle Prediction: To tackle the challenging problem of needle tip localization and class imbalance, a Deep Hough Transform (DHT) is applied to the frequency features. This module predicts the needle shaft and tip in Hough space, where a point in image space corresponds to a curve in Hough space. This allows for more robust detection of the needle's geometry.
The method uses a Gaussian-blurred ground truth and a Focal Loss function to handle imperfect annotations and class imbalance, respectively.
Datasets
The study utilized ex vivo porcine and bovine tissue samples for data acquisition. A total of 417 ten-second US videos (approximately 125K frames) were recorded. The dataset was divided into training, validation, and test sets. The test set was further categorized into "normal" cases with good needle visibility and "challenging" cases with less visibility. Data was also categorized by insertion angle (steep vs. low-angle).
Key Insights and Ideas
- Frequency Domain for Robustness: The most significant insight is that analyzing the frequency domain of the subtle vibrations is far more robust for needle detection than relying solely on image intensity, especially when visibility is low.
- Vibration as a Signal: External, periodic vibration acts as a powerful, albeit subtle, signal that can be reliably detected and exploited for localization.
- Hough Transform for Needle Geometry: Applying the Hough Transform in the latent feature space (Deep Hough Transform) effectively addresses the class imbalance issue and helps in precisely localizing the needle tip and shaft by transforming the detection problem into a geometric one.
- End-to-End Learning: The framework is end-to-end, integrating feature extraction, frequency analysis, and prediction into a single, trainable network.
Key Results
- Superior Performance in Challenging Cases: On porcine data, VibNet achieved tip errors of 1.61 ± 1.56 mm and needle direction errors of 1.64 ± 1.86° in challenging cases (nearly invisible needles), significantly outperforming UNet (8.15±9.98 mm tip error, 9.29±15.30° direction error) and WNet (6.63±7.58 mm tip error, 8.54±17.92° direction error).
- Low TER: VibNet achieved a Threshold Exceedance Rate (TER) of 0.07% on challenging porcine cases, compared to 27.94% for UNet and 33.23% for WNet.
- Generalization: VibNet showed good generalization capabilities, maintaining comparable performance when tested on unseen tissue types (e.g., porcine-trained model on bovine data).
- Inference Speed: The total inference time for VibNet was approximately 84.2 ms, allowing for an inference frequency of around 12 Hz, which is sufficient for many low-dynamic interventions.
Limitations and Future Work
- No Insertion Motion: The current study's data was recorded without simulating the needle insertion motion. Future work should investigate performance with dynamic insertion.
- Needle Properties: The study used a rigid needle. The performance might be affected by thinner or more flexible needles that could exhibit different vibration responses.
- Clinical Factors: Further investigation is needed to address practical factors in real patients, such as varying insertion depths, needle bending, different needle types, patient motion, and probe-induced tissue deformation.
- Out-of-Plane Detection: The current approach focuses on in-plane 2D US images. Extending it to out-of-plane detection (where the needle appears as a point) is a future direction.
- Vibration Pattern Optimization: Exploring different vibration frequencies and amplitudes for various clinical applications is also suggested.
Summary updated