Abstract
Convolutional Neural Networks combined with a state-of-the-art stereo-matching method are used to find and estimate the 3D position of vehicles in pairs of stereo images. Pixel positions of vehicles are first estimated separately in pairs of stereo images using a Convolutional Neural Network for regression. These coordinates are then combined with a state-of-the-art stereo-matching method to determine the depth, and thus the 3D location, of the vehicles. We show in this paper that cars can be detected with a combined accuracy of approximately 90% with a tolerated radius error of 5%, and a Mean Absolute Error of 5.25m on depth estimation for cars up to 50m away.
Introduction
Accurately locating surrounding vehicles in 3D space is a fundamental requirement for safe autonomous navigation. Current approaches typically rely on either direct perception or stereo matching. Direct perception models use CNNs to regress 3D coordinates $(x, z)$ directly from images, but they are often brittle; minor pixel-level errors in the network's internal representation can translate into significant depth $(z)$ inaccuracies. Conversely, classical stereo matching provides geometric precision through triangulation but can be computationally intensive and struggles with object identification within complex depth maps.
This paper proposes a hybrid pipeline that leverages the strengths of both approaches: using a CNN for robust 2D object localization and stereo matching for high-precision depth estimation.
Methodology
The proposed system operates in two distinct stages:
- 2D Regression via CNN: A modified AlexNet architecture is employed for regression rather than classification. The network predicts the $(x, y)$ pixel coordinates for up to three vehicles per image segment. Training utilizes an L2-norm Euclidean Loss function to minimize the distance between predicted and ground-truth coordinates.
- Stereo Matching for Depth Estimation: The predicted 2D coordinates from the stereo image pairs are used to identify corresponding image patches. By calculating the disparity—the horizontal shift between these patches—the system determines the precise depth $(z)$ of the vehicle.
To optimize training and maintain resolution, high-resolution images are segmented into three sub-images, effectively tripling the training data and ensuring compatibility with the network's input constraints.
Experimental Setup
The research utilizes the KITTI Dataset, comprising approximately 6,000 stereo image pairs from urban environments. The study focuses on "car" and "van" labels. To enhance model robustness, the dataset was augmented through image mirroring and the aforementioned segmentation, resulting in over 48,000 training samples. Transfer learning was also utilized, fine-tuning a model pre-trained on ImageNet to accelerate convergence.
Results and Discussion
The hybrid approach demonstrates significant improvements in depth estimation reliability. Within a tolerated radius error of 5% of the image size, the system achieves a combined detection accuracy of approximately 90%.
Quantitatively, the system achieved a Mean Absolute Error (MAE) of 3.97m for vehicles within 25m and 5.25m for vehicles up to 50m away. This outperforms the "DeepDriving" baseline, which reported an MAE of 5.83m at the 50m range. These results validate the hypothesis that offloading depth calculation to geometric processes avoids the sensitivity issues inherent in pure regression models.
Conclusion and Future Work
While effective, the current implementation faces challenges with distant vehicles and asymmetric prediction accuracy due to the lack of ground-truth labels for right-side stereo images in the KITTI dataset. Future research will explore more efficient architectures like SqueezeNet to reduce computational overhead and multi-scale detection strategies to improve accuracy for distant objects.
Summary updated