Bayesian Dense Inverse Searching Algorithm for Real-Time Stereo Matching in Minimally Invasive Surgery

Jingwei Song, Qiuchen Zhu, Jianyu Lin, Maani Ghaffari

Abstract

Real-time 3D tissue reconstruction is a cornerstone of Computer Assisted Surgery (CAS), yet it remains computationally demanding and sensitive to the unique optical properties of surgical sites. We introduce Bayesian Dense Inverse Searching (BDIS), a novel CPU-based algorithm designed for robust stereo matching. By embedding the fast Dense Inverse Searching (DIS) method within a Bayesian framework, BDIS effectively manages specular highlights and textureless surfaces. Our approach achieves 10–25 frames per second on a single standard CPU core, demonstrating superior accuracy and robustness compared to existing methods like ELAS and SGBM in challenging surgical environments.

Introduction

In the domain of Computer Assisted Surgery, providing robots and surgeons with a real-time 3D map of the operative field is essential for safety and precision. This depth information is typically derived via stereo matching. However, surgical environments are notoriously difficult for conventional computer vision algorithms due to three primary factors:

  1. Non-Lambertian Reflectance: Wet, mucosal tissues create intense specular highlights (glare) that violate the brightness constancy assumption used by most matching algorithms.
  2. Textureless Surfaces: Large areas of smooth tissue lack the distinctive features required for reliable point-to-point matching.
  3. Computational Constraints: While deep learning models offer high accuracy, their reliance on high-end GPUs is often incompatible with the low-latency, small-footprint requirements of clinical surgical systems.

To address these issues, we propose BDIS, which leverages probabilistic inference to handle uncertainty in matching, ensuring real-time performance on standard hardware.

Methodology

BDIS builds upon the Lucas-Kanade (LK) optical flow and Dense Inverse Searching (DIS) principles, extending them into a probabilistic domain.

Bayesian Patch-Wise Posterior

Instead of a deterministic "best-match" search, BDIS calculates the probability of a match being correct. Using a Conditional Random Field (CRF) model, the algorithm computes a posterior probability based on photometric consistency. This allows the system to weigh matches by their reliability, effectively ignoring regions dominated by noise or glare.

Illustration of the probabilistic matching approach for a tissue patch, represented as the posterior probability.

Spatial Gaussian Mixture Model (GMM)

To further refine patch matching, BDIS applies a spatial Gaussian mask within each image patch. This gives higher weight to central pixels and reduces the influence of edge pixels, which are more susceptible to occlusion and noise.

Coarse-to-Fine Refinement

The algorithm employs a hierarchical approach, starting at low resolutions to establish a global matching context and progressively refining the disparity map at higher resolutions. At each level, the Bayesian weights filter out dubious matches, preventing errors from propagating through the scales.

The BDIS pipeline utilizing a coarse-to-fine refinement strategy for dense inverse searching between stereo images.

Experimental Setup

The performance of BDIS was evaluated using two distinct datasets:

  • In-Vivo Surgical Video: Stereo footage from real surgeries (resolutions of 640x480 and 288x360) provided by the Hamlyn Centre.
  • Synthetic Virtual Phantom: A custom dataset generated in Unity3D simulating a colonoscope. This allowed for controlled testing of "diffuse" vs. "non-Lambertian" lighting conditions against perfect ground-truth depth data.

Results

BDIS demonstrates a significant advancement in both speed and robustness:

  • Real-Time Performance: The algorithm achieves 10 Hz at 640×480 and 25 Hz at 360×288 using only a single core of an Intel i5-9400 CPU, making it twice as fast as the industry-standard ELAS.
  • Enhanced Accuracy: In synthetic tests featuring heavy glare, BDIS outperformed ELAS by 17.68%.
  • Robustness to Artifacts: Qualitatively, BDIS produces significantly cleaner depth maps with fewer outliers (3D spikes or holes) in textureless and high-glare regions compared to standard DIS and SGBM methods.
  • Generalization: BDIS outperformed several state-of-the-art deep learning models (e.g., PSMNet) which struggled to generalize to the specific textures of surgical datasets without extensive retraining.

Conclusion

BDIS offers a robust, CPU-efficient solution for real-time stereo matching in minimally invasive surgery. By treating uncertainty as a core feature through a Bayesian framework, it overcomes the traditional pitfalls of glare and textureless surfaces. While the algorithm currently faces challenges at image boundaries and in extremely dark regions, its ability to provide confidence-weighted depth maps makes it a promising candidate for integration into future autonomous surgical SLAM pipelines.

Summary updated

Abstract

This paper reports a CPU-level real-time stereo matching method for surgical images (10 Hz on 640 * 480 image with a single core of i5-9400). The proposed method is built on the fast ''dense inverse searching'' algorithm, which estimates the disparity of the stereo images. The overlapping image patches (arbitrary squared image segment) from the images at different scales are aligned based on the photometric consistency presumption. We propose a Bayesian framework to evaluate the probability of the optimized patch disparity at different scales. Moreover, we introduce a spatial Gaussian mixed probability distribution to address the pixel-wise probability within the patch. In-vivo and synthetic experiments show that our method can handle ambiguities resulted from the textureless surfaces and the photometric inconsistency caused by the Lambertian reflectance. Our Bayesian method correctly balances the probability of the patch for stereo images at different scales. Experiments indicate that the estimated depth has higher accuracy and fewer outliers than the baseline methods in the surgical scenario.
BibTeX
@inproceedings{song2022bayesian,
  author = {Jingwei Song and Qiuchen Zhu and Jianyu Lin and Maani Ghaffari},
  title = {{Bayesian Dense Inverse Searching Algorithm for Real-Time Stereo Matching in Minimally Invasive Surgery}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2022},
  doi = {10.1007/978-3-031-16449-1\_32},
}
Figure