A Vascular Feature Detection and Matching Method Based on Dual-Branch Fusion and Structure Enhancement

Kaiyang Xu, Haibin Wu, Yuji Iwahori, Xiaoyu Yu, Zeyu Hu, Aili WangSensors 2024 · DOI

TL;DR

This paper introduces a novel method for robustly detecting and matching vascular features in challenging laparoscopic 3D reconstruction scenarios. By combining a dual-branch vascular structure enhancement technique with a specialized feature detection and matching pipeline, the method significantly improves accuracy and repeatability in complex internal cavity environments.

What Problem Does This Paper Address?

Accurate 3D reconstruction of internal body cavities during minimally invasive surgery (MIS) is crucial for augmented reality guidance and improved surgical outcomes. A key challenge in this process is robustly tracking features within the laparoscopic view, which is often plagued by poor lighting, specular reflections, tissue deformation, and a lack of distinct landmarks. Traditional feature detection and matching methods struggle in these conditions, leading to inaccurate 3D reconstructions. Specifically, identifying and tracking vascular structures, which are prominent but complex features, is difficult due to variations in vessel size, intensity, and the presence of noise and artifacts. This paper aims to address these limitations by developing a specialized method for vascular feature detection and matching tailored for MIS environments.

Methods

The proposed method comprises several key stages:

  • Preprocessing Framework: A universal preprocessing pipeline is designed to handle the unique challenges of MIS images. This includes:
  • Adaptive Thresholding: To generate binary masks that preserve relevant information and exclude noisy boundary regions.
  • Green Channel Extraction: Leveraging the higher contrast of green channels for blood vessels.
  • Highlight Repair: Using the Fast Marching Method (FMM) to restore information in overexposed or specularly reflected areas.
  • Dual-Branch Vascular Structure Enhancement: This is the core of the enhancement process, designed to overcome limitations of existing methods:
  • Frangi Measure: A classic method for enhancing tubular structures, adapted to address structural measurement differences between venous and microvessels.
  • Multiscale Fractional Anisotropic Tensor (MFAT): A more advanced technique that provides a more uniform response to vascular structures.
  • Gaussian Weighted Fusion: The outputs from the Frangi and MFAT branches are fused using a Gaussian weighting function, with weights adaptively determined by the Structural Similarity Index (SSIM) to optimize for the specific internal cavity environment. This fusion aims to compensate for individual method weaknesses and ensure continuity and integrity of the vascular structure.
  • Single-Pixel Vascular Structure: To achieve precise localization, the enhanced vascular structures are thinned to single-pixel width using a specialized algorithm.
  • Vascular Feature Detection:
    • Dual-Circle Detection (CDVB): A method inspired by the FAST algorithm, adapted to detect vascular branch points by identifying intersections of three or more vascular segments.
  • Non-Maximum Suppression (NMS): Applied to reduce redundancy in detected feature points and ensure a more even distribution.
  • Feature Matching:
    • Neighborhood Block Matching: Based on the PTAM algorithm, this stage uses the Zero Sum of Squared Differences (ZSSD) to match feature points by comparing local image patches between consecutive frames.

Datasets

The paper primarily uses the Vivo data set for evaluation. This dataset includes abdominal wall and small flat tissue surfaces captured during laparoscopic colon surgery. The authors also mention using the Hamlyn data set (specifically Rectificed01) for illustrative purposes.

Key Insights and Ideas

  • The "One-Size-Fits-All" Problem: The paper highlights that standard computer vision techniques struggle with the highly variable and challenging conditions found in MIS. This necessitates specialized approaches.
  • Complementary Enhancement: Combining established methods like Frangi filtering with more advanced techniques like MFAT, and then intelligently fusing their outputs, proves more effective than using either alone. The adaptive Gaussian weighting based on SSIM is a clever way to optimize this fusion.
  • Single-Pixel Precision: Thinning vascular structures to a single pixel is crucial for accurate localization of branch points, which is vital for subsequent 3D reconstruction.
  • Robust Feature Detection in Noise: The dual-circle detection method, combined with NMS, offers a robust way to find branch points even in noisy and artifact-prone images.
  • SSIM as an Adaptive Optimizer: Using SSIM not just for evaluation but also to adaptively tune the fusion weights is a smart way to improve generalization across different MIS scenarios.

Key Results

The proposed method demonstrates significant improvements over existing algorithms:

  • Accuracy and Repeatability: The method achieved an average accuracy and repeatability score of 0.7149 and 0.5612 respectively on the Vivo dataset.
  • Quantitative Comparison: Compared to RBCT, the method showed a 3.23% improvement in matching accuracy, a 13.08% increase in branch point detections, and a 4.91% improvement in repeatability.
  • Robustness: The method proved robust across five different data sets, with only one (clip5) showing similar results to RBCT due to poor image quality.
  • Speed: While the method is computationally more intensive than some simpler algorithms (e.g., RBCT time of 0.95931s vs. proposed method's 1.9515s), the authors suggest optimizations and the trade-off is justified by the improved accuracy and robustness.

Limitations and Future Work

  • Computational Cost: The paper acknowledges that the parallel processing for vascular enhancement, while effective, impacts the overall running speed. Further optimization is suggested.
  • Future Work: The authors plan to explore incorporating this method into visual odometry systems to derive 3D information from internal cavity features. They also aim to integrate this into augmented reality displays for real-time surgical guidance, providing depth feedback and expanding the surgeon's field of view.

Summary updated

BibTeX
@article{xu2024vascular,
  author = {Kaiyang Xu and Haibin Wu and Yuji Iwahori and Xiaoyu Yu and Zeyu Hu and Aili Wang},
  title = {{A Vascular Feature Detection and Matching Method Based on Dual-Branch Fusion and Structure Enhancement}},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24061880},
}
Figure