Robust Cost Volume Generation Method for Dense Stereo Matching in Endoscopic Scenarios

Yucheng Jiang, Zehua Dong, Songping MaiSensors 2023 · DOI

TL;DR

This paper introduces a novel, non-deep learning method for generating robust cost volumes for dense stereo matching, specifically tailored for challenging endoscopic environments. By combining radiometric invariant HOG descriptors with a cross-scale propagation technique, the method achieves accuracy comparable to deep learning approaches while being significantly faster and less computationally demanding.

What Problem Does This Paper Address?

Dense stereo matching is crucial for 3D reconstruction and measurement in various applications, including medical endoscopies. However, endoscopic scenarios present unique challenges, primarily radiometric distortion caused by inconsistent lighting conditions. Traditional stereo matching algorithms often struggle with these distortions, leading to inaccurate disparity maps. While deep learning methods have shown high accuracy, they are often data-dependent, computationally expensive, and may not generalize well to real-world, unseen scenarios, limiting their use in embedded systems. This paper aims to bridge this gap by developing a traditional, non-deep learning method that offers robust performance in the face of radiometric distortion and is computationally efficient.

Methods

The proposed method focuses on generating a robust cost volume through a multi-step process:

  • Radiometric Invariant Cost Metrics: Instead of relying on simple intensity differences, the initial cost volume is constructed using two novel radiometric invariant metrics derived from Histogram of Oriented Gradients (HOG):
  • HOG Angle: This metric uses the angle of gradient vectors, which is invariant to linear radiometric distortions.
  • Relative HOG Amplitude: This metric uses the normalized amplitude of gradient vectors, further compensating for amplitude variations and providing complementary information to the HOG angle. These two metrics are fused to create a more robust initial cost volume.

  • Guided Image Filtering: To suppress noise in the initial cost volume, a guided image filter is employed. This filter is chosen for its edge-preserving capabilities and computational efficiency, and importantly, it operates under the same linear radiometric assumption as the HOG metrics, ensuring its effectiveness.

  • Cross-Scale Cost Volume Propagation: To improve matching reliability in texture-less regions, a cross-scale propagation framework is introduced. This involves down-sampling images vertically (to preserve horizontal accuracy) and propagating cost volumes from coarser scales to finer scales. Crucially, different propagation weights are used for textured and texture-less regions to maintain disparity detail.
  • Cost Volume Optimization: The generated cost volume is then optimized using established traditional methods: AD-Census for local optimization and Local-Expansion for global optimization. This demonstrates the compatibility and effectiveness of the proposed cost volume with existing optimization techniques.

Datasets

The paper evaluates its method on several datasets:

  • Middlebury Version 3 Benchmark: A standard benchmark for evaluating stereo matching algorithms, used for quantitative comparison of accuracy and reliability.
  • Surgical Endoscopic Dataset (SERV-CT): A dataset specifically designed for evaluating 3D reconstruction accuracy in real endoscopic scenarios.
  • Custom Binocular Endoscope Data: Real image pairs of standard objects captured by the authors' own binocular endoscope are used for qualitative and quantitative evaluation.
  • 3D Printed Objects: Spheres and a cylinder with known parameters are used to test the system's reconstruction accuracy in a controlled environment.

Key Insights and Ideas

  • HOG as a Radiometric Invariant: The paper successfully demonstrates that HOG angle and relative HOG amplitude descriptors can be effectively used as radiometric invariant cost metrics, outperforming traditional metrics like Census and NCC in challenging scenarios.
  • Fusion for Robustness: Fusing HOG angle and relative HOG amplitude significantly improves matching reliability, especially in areas where one metric might be less effective.
  • Guided Filter Synergy: The guided image filter's compatibility with the HOG metric's underlying assumptions makes it a powerful tool for noise reduction in the cost volume.
  • Cross-Scale Propagation for Texture-less Areas: The cross-scale propagation, with adaptive weights, effectively addresses the challenge of matching in homogeneous regions, a common weakness in traditional stereo methods.
  • Non-Deep Learning Competitiveness: The research shows that a well-designed traditional approach can achieve performance comparable to deep learning methods in specific challenging domains like endoscopy, while offering significant advantages in speed and computational cost.

Key Results

  • Accuracy: The proposed method, when combined with Local-Expansion, achieves accuracy comparable to deep learning algorithms on the Middlebury dataset and demonstrates a reconstruction accuracy of around 5 mm on the SERV-CT dataset, exceeding some deep learning methods.
  • Robustness: The HOG-based cost volume shows improved robustness in radiometric distorted and texture-less regions compared to traditional metrics.
  • Speed: The cost volume generation process is significantly faster than MC-CNN, being approximately 65 times faster on average. This makes it suitable for real-time applications.
  • Performance on Endoscopic Scenarios: The method performs well on endoscopic datasets, where large homogeneous regions (a limitation for the HOG metric) are less common.

Limitations and Future Work

  • Large Homogeneous Regions: The paper acknowledges that the HOG metric struggles with very large homogeneous areas, which can lead to performance drops. While this is less of an issue in endoscopic scenarios, it's a consideration for other applications.
  • Optimization Complexity: The overall efficiency is still limited by the complexity of the optimization methods used after cost volume generation.
  • Future Work: The authors suggest exploring acceleration of the optimization methods as a future research direction.

Summary updated

BibTeX
@article{jiang2023robust,
  author = {Yucheng Jiang and Zehua Dong and Songping Mai},
  title = {{Robust Cost Volume Generation Method for Dense Stereo Matching in Endoscopic Scenarios}},
  journal = {Sensors},
  year = {2023},
  doi = {10.3390/s23073427},
}
Figure