Abstract
This technical report focuses on the evaluation and performance of depth estimation methods based on LiDAR data and stereo imagery (front left and front right). Utilizing datasets provided by Ford, the paper details optimizations for depth estimation performance and provides a rationale for favoring pure mathematical approaches over machine learning in specific cross-domain scenarios. The structure of this report covers performance evaluation of depth maps, methodologies for stereo-based estimation, and LiDAR-based depth completion techniques.
Introduction
Accurate depth estimation is a foundational requirement for autonomous driving and robotics. Currently, the field relies heavily on two modalities: LiDAR, which provides highly accurate but sparse 3D point clouds, and stereo cameras, which provide dense visual data but require complex matching algorithms to infer depth.
Recently, deep learning has dominated stereo depth estimation. However, a significant challenge remains: neural networks trained on standard benchmarks (like KITTI) often overfit to specific data distributions. When applied to new environments, such as the Ford dataset used here, these models frequently fail to produce accurate disparity maps. This paper explores how to optimize traditional, purely mathematical computer vision techniques to achieve robust depth estimation without relying on machine learning.
Methodology
Stereo Depth Estimation
The stereo pipeline utilizes a non-deep-learning approach centered on Semi-Global Block Matching (SGBM). Unlike basic Block Matching, SGBM calculates similarities across traversal windows to create a global energy function, yielding significantly higher precision. To further refine the output, a Weighted Least Squares (WLS) filter is applied to smooth low-gradient areas while preserving sharp edges. Additionally, padding optimization is used to resolve artifacts at the image edges caused by the matching window size.
LiDAR Depth Estimation
LiDAR depth maps are generated by projecting 3D point clouds onto a 2D image plane using coordinate transformations (global to body to camera frame). Because this projection results in sparse data, a neighborhood-weighted averaging technique is employed for depth completion. A critical optimization introduced here is "foreground preservation": sparse, short-distance pixels are dilated before completion to prevent background depths from "swallowing" thin foreground objects like light poles and fences.
Performance Evaluation
Quantitative and Qualitative Results
The traditional SGBM + WLS approach was evaluated against LiDAR-derived ground truth. Across three sample images, the mathematical approach achieved average per-pixel errors of 5.45m, 12.11m, and 4.93m. The higher error in the second scenario was attributed to complex environmental factors, such as high sunlight and intricate tree shadows, which challenge stereo matching algorithms.
Comparison with Machine Learning Models
When tested on the Ford dataset, pretrained neural networks (including HRSDE and Lac-GwcNet) trained on KITTI showed poor generalization. For instance, the HRSDE model yielded errors ranging from 13.9m to 24.0m on the same images where the mathematical approach performed significantly better. This highlights the robustness of traditional algorithms in novel environments.
Conclusion
While traditional stereo depth estimation remains sensitive to lighting and complex textures, it demonstrates superior generalization compared to overfitted neural networks. By integrating optimizations like WLS filtering and foreground-preserving LiDAR densification, robust depth maps can be generated for autonomous systems. Future work will focus on enhancing stereo matching robustness to extreme lighting conditions, potentially bridging the gap between mathematical generalization and the contextual strengths of machine learning.
Summary updated