Abstract
Stereo vision is a cornerstone of autonomous navigation, yet the high computational demands of state-of-the-art deep learning models often preclude their use on resource-constrained embedded systems. This paper presents Non-Local Anchoring (NLA), a lightweight, hardware-agnostic framework designed to refine noisy depth maps. By utilizing a Random Forest classifier to automatically identify reliable pixels and a non-local weighted median filter for reconstruction, NLA significantly improves stereo accuracy without the power overhead of convolutional neural networks (CNNs). Evaluations on the Middlebury and KITTI datasets demonstrate that NLA effectively bridges the gap between traditional block-matching algorithms and complex deep-learning refinement strategies.
Introduction
Calculating depth through stereo vision—comparing two offset images—is essential for robotics and drones. While modern CNNs achieve high accuracy, they are often impractical for mobile processors due to their massive power consumption. Furthermore, many commercial-off-the-shelf (COTS) stereo cameras provide "black box" disparity maps, hiding the internal raw matching scores (cost volumes) required by standard refinement techniques. There is a critical need for a refinement method that is fast, automated, and compatible with any stereo camera output.
The Non-Local Anchoring (NLA) Framework
The NLA pipeline operates entirely in the disparity domain, requiring only the initial noisy depth map and the original reference image. It consists of two primary stages:
- Threshold-free Reliable Point (RP) Selection: Instead of relying on hand-tuned sensitivity thresholds, NLA uses a Random Forest classifier. This model analyzes local and global features of a confidence map to automatically categorize pixels as either Reliable Points (RP) or Unreliable Points (UP).
- Non-Local Inference and Refinement: For every unreliable pixel, the algorithm searches in up to 16 directions to find the nearest reliable anchors. The final depth value is calculated using a weighted median of these anchors, where weights are determined by spatial distance and color similarity to ensure depth edges align with object boundaries.
Experimental Evaluation
The framework was evaluated on the Middlebury v3 (indoor) and KITTI 2015 (outdoor) benchmarks. The Random Forest classifier was trained on small subsets of these datasets to ensure generalization across diverse environments.
Key Results
- Accuracy Improvement: On the Middlebury dataset, NLA reduced the "bad pixel" rate (errors > 1px) of a standard Block-Matching algorithm from 35.13% to 22.90%.
- Outdoor Robustness: In the challenging KITTI dataset, NLA successfully reconstructed depth in areas affected by complex lighting and motion, outperforming traditional filters like the Bilateral or Weighted Median filters.
- Embedded Performance: Tested on an NVIDIA Jetson TX2, the core NLA algorithm executed in 1.82 seconds on the CPU. While not yet real-time for high-speed flight, it is significantly more efficient than running full-scale deep-learning stereo networks on the same hardware.
Discussion and Future Work
NLA's primary advantage is its "agnostic" nature; it can be integrated as a post-processing step for any stereo system. By replacing heuristic thresholds with machine learning, the system becomes self-tuning, adapting to the specific noise characteristics of different sensors automatically.
However, the performance remains tied to the quality of the initial confidence measure. Future research will focus on optimizing the local aggregation steps—potentially through GPU acceleration—to achieve real-time frame rates suitable for high-speed autonomous robotics.
Summary updated