Computer Stereo Vision for Autonomous Driving

Rui Fan, Li Wang, Mohammud Junaid Bocus, Ioannis PitasarXiv 2020 · arXiv

Abstract

Autonomous vehicles require precise 3D environmental perception to navigate safely. While LiDAR provides high-fidelity depth data, computer stereo vision offers a cost-effective, high-resolution alternative by mimicking human binocular vision. This paper explores the technical landscape of stereo vision for autonomous driving, focusing on the critical engineering trade-offs between real-time processing speed and depth-sensing accuracy.

Introduction: The Role of Perception

For an autonomous vehicle (AV), the perception system is the foundation upon which localization, planning, and control are built. Stereo vision serves as the "3D eyes" of the vehicle, transforming 2D image data into spatial information.

Figure 1: The autonomous car system architecture, highlighting the role of perception within the software stack.

The primary challenge in deploying stereo vision is the "Speed-Accuracy Conflict." High-accuracy algorithms often demand computational resources that exceed the capabilities of a car's embedded hardware, while faster algorithms may produce noisy depth maps that compromise safety.

Technical Pipeline

The stereo vision process involves several sophisticated stages that bridge classical geometry and modern machine learning.

1. Geometric Foundations and Rectification

The process begins with the pinhole camera model and lens distortion correction. A critical step is Stereo Rectification, which aligns the left and right camera images. By ensuring that corresponding points lie on the same horizontal scanline, the search space for matching pixels is reduced from 2D to 1D, significantly lowering computational overhead.

2. Disparity Estimation and Scene Reconstruction

Disparity estimation calculates the horizontal shift between images to derive depth.

  • Classical Approaches: Methods like Semi-Global Matching (SGM) are industry standards because they balance speed and quality through efficient optimization.
  • Learning-Based Approaches: Modern CNNs (e.g., PSMNet, DispNet) regress depth directly from frames, often outperforming classical methods in accuracy but at a higher computational cost.
Figure 3: Comparison between a real scene and its 3D reconstruction using stereo camera inputs.

3. Heterogeneous Computing

To achieve real-time performance (30+ FPS), algorithms must leverage parallel computing. This involves optimizing code for multi-core CPUs (OpenMP) and massively parallel GPUs (CUDA). Efficient memory management and alignment with GPU "warps" (32-thread blocks) are essential for maximizing throughput on automotive-grade hardware.

Benchmarking and Datasets

The development of these systems relies on high-quality datasets:

  • KITTI Stereo (2012 & 2015): The primary benchmark for real-world driving scenarios.
  • Scene Flow Datasets: Synthetic datasets (e.g., FlyingThings3D) that provide dense ground-truth depth maps for training deep learning models.

Key Insights and Future Directions

A significant trend is the emergence of Hybrid SGM-Nets, where CNNs are used to predict penalty parameters for classical SGM algorithms. This approach combines the geometric robustness of traditional computer vision with the feature-learning capabilities of AI.

The "deployment gap" remains the biggest hurdle. High-accuracy deep learning models are often too resource-intensive for embedded systems. Future research is shifting toward model compression and hardware-algorithm co-design, aiming to deliver high-fidelity 3D vision on low-power, automotive-grade silicon.

Summary updated

Abstract

As an important component of autonomous systems, autonomous car perception has had a big leap with recent advances in parallel computing architectures. With the use of tiny but full-feature embedded supercomputers, computer stereo vision has been prevalently applied in autonomous cars for depth perception. The two key aspects of computer stereo vision are speed and accuracy. They are both desirable but conflicting properties, as the algorithms with better disparity accuracy usually have higher computational complexity. Therefore, the main aim of developing a computer stereo vision algorithm for resource-limited hardware is to improve the trade-off between speed and accuracy. In this chapter, we introduce both the hardware and software aspects of computer stereo vision for autonomous car systems. Then, we discuss four autonomous car perception tasks, including 1) visual feature detection, description and matching, 2) 3D information acquisition, 3) object detection/recognition and 4) semantic image segmentation. The principles of computer stereo vision and parallel computing on multi-threading CPU and GPU architectures are then detailed.
BibTeX
@misc{fan2020computer,
  author = {Rui Fan and Li Wang and Mohammud Junaid Bocus and Ioannis Pitas},
  title = {{Computer Stereo Vision for Autonomous Driving}},
  year = {2020},
  eprint = {2012.03194},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2012.03194},
}
Figure