Abstract
Real-time 3D tissue reconstruction is a cornerstone of Computer Assisted Surgery (CAS), yet it remains computationally demanding and sensitive to the unique optical properties of surgical sites. We introduce Bayesian Dense Inverse Searching (BDIS), a novel CPU-based algorithm designed for robust stereo matching. By embedding the fast Dense Inverse Searching (DIS) method within a Bayesian framework, BDIS effectively manages specular highlights and textureless surfaces. Our approach achieves 10–25 frames per second on a single standard CPU core, demonstrating superior accuracy and robustness compared to existing methods like ELAS and SGBM in challenging surgical environments.
Introduction
In the domain of Computer Assisted Surgery, providing robots and surgeons with a real-time 3D map of the operative field is essential for safety and precision. This depth information is typically derived via stereo matching. However, surgical environments are notoriously difficult for conventional computer vision algorithms due to three primary factors:
- Non-Lambertian Reflectance: Wet, mucosal tissues create intense specular highlights (glare) that violate the brightness constancy assumption used by most matching algorithms.
- Textureless Surfaces: Large areas of smooth tissue lack the distinctive features required for reliable point-to-point matching.
- Computational Constraints: While deep learning models offer high accuracy, their reliance on high-end GPUs is often incompatible with the low-latency, small-footprint requirements of clinical surgical systems.
To address these issues, we propose BDIS, which leverages probabilistic inference to handle uncertainty in matching, ensuring real-time performance on standard hardware.
Methodology
BDIS builds upon the Lucas-Kanade (LK) optical flow and Dense Inverse Searching (DIS) principles, extending them into a probabilistic domain.
Bayesian Patch-Wise Posterior
Instead of a deterministic "best-match" search, BDIS calculates the probability of a match being correct. Using a Conditional Random Field (CRF) model, the algorithm computes a posterior probability based on photometric consistency. This allows the system to weigh matches by their reliability, effectively ignoring regions dominated by noise or glare.
Spatial Gaussian Mixture Model (GMM)
To further refine patch matching, BDIS applies a spatial Gaussian mask within each image patch. This gives higher weight to central pixels and reduces the influence of edge pixels, which are more susceptible to occlusion and noise.
Coarse-to-Fine Refinement
The algorithm employs a hierarchical approach, starting at low resolutions to establish a global matching context and progressively refining the disparity map at higher resolutions. At each level, the Bayesian weights filter out dubious matches, preventing errors from propagating through the scales.
Experimental Setup
The performance of BDIS was evaluated using two distinct datasets:
- In-Vivo Surgical Video: Stereo footage from real surgeries (resolutions of 640x480 and 288x360) provided by the Hamlyn Centre.
- Synthetic Virtual Phantom: A custom dataset generated in Unity3D simulating a colonoscope. This allowed for controlled testing of "diffuse" vs. "non-Lambertian" lighting conditions against perfect ground-truth depth data.
Results
BDIS demonstrates a significant advancement in both speed and robustness:
- Real-Time Performance: The algorithm achieves 10 Hz at 640×480 and 25 Hz at 360×288 using only a single core of an Intel i5-9400 CPU, making it twice as fast as the industry-standard ELAS.
- Enhanced Accuracy: In synthetic tests featuring heavy glare, BDIS outperformed ELAS by 17.68%.
- Robustness to Artifacts: Qualitatively, BDIS produces significantly cleaner depth maps with fewer outliers (3D spikes or holes) in textureless and high-glare regions compared to standard DIS and SGBM methods.
- Generalization: BDIS outperformed several state-of-the-art deep learning models (e.g., PSMNet) which struggled to generalize to the specific textures of surgical datasets without extensive retraining.
Conclusion
BDIS offers a robust, CPU-efficient solution for real-time stereo matching in minimally invasive surgery. By treating uncertainty as a core feature through a Bayesian framework, it overcomes the traditional pitfalls of glare and textureless surfaces. While the algorithm currently faces challenges at image boundaries and in extremely dark regions, its ability to provide confidence-weighted depth maps makes it a promising candidate for integration into future autonomous surgical SLAM pipelines.
Summary updated