Abstract
DALDet is an efficient, depth-aware 2D object detector designed specifically for the rigors of autonomous driving. By integrating depth cues directly into the convolutional architecture and employing a novel depth-guided loss function, it achieves state-of-the-art accuracy on the KITTI benchmark—ranking 1st among 2D detectors—while maintaining a high-speed inference rate of 66.7 FPS.
Introduction
In autonomous driving, object detection is typically split into two camps: 3D detection and 2D detection. 3D detectors provide high accuracy and spatial context but are computationally expensive and rely on costly hardware. 2D detectors are much faster and easier to deploy on edge devices, but they lack the "spatial common sense" provided by depth, often struggling with small or distant objects and failing to provide critical distance information for obstacle avoidance.
DALDet bridges this gap by building a 2D detector that "thinks" in 3D by leveraging depth maps, without sacrificing the speed and efficiency that makes 2D detection attractive.
Methodology
DALDet is a one-stage detection framework consisting of a backbone, a neck (PAFPN), and a detection head. Its performance stems from three main innovations:
- Depth-Aware Convolution & Pooling: Standard convolutions treat all pixels in a window equally. DALDet introduces a depth similarity term; pixels with similar depth to the center pixel are given higher weights, allowing the network to learn spatially and geometrically consistent features.
- Depth-Guided Loss Function: A new loss function uses the depth map to calculate the difference between predicted and ground-truth boxes. By focusing on non-overlapping regions in depth space, it forces the model to improve localization accuracy, especially where standard IoU is insufficient.
- Efficient Architecture: The model utilizes specialized blocks like the Efficient Down-sampling Block (EDB) and Lightweight Residual Blocks (LRB) to keep the parameter count low and inference speed high.
Notably, while the model outputs 2D bounding boxes, it maps these boxes back to the input depth map to provide a distance estimate ($d$) for every detected object, as shown in the pipeline above.
Experimental Results
The model was trained and evaluated on the KITTI Dataset, focusing on Car, Pedestrian, and Cyclist categories across three difficulty levels. Stereo-generated depth maps (via IGEV-Stereo) provided the depth input during training and inference.
Performance Benchmarks
At the time of submission, DALDet ranked 1st on both the KITTI Car and Cyclist 2D detection leaderboards among all 2D detectors. It achieves 66.7 FPS on an NVIDIA RTX 3090, making it viable for real-time applications.
On the KITTI validation set, DALDet outperformed YOLOv7-x by 0.24% on Moderate Car AP, 4.01% on Moderate Pedestrian AP, and a significant 21.62% on Moderate Cyclist AP.
Discussion and Conclusion
DALDet demonstrates that depth can be used as a feature to reformulate how convolutions operate, rather than just being an additional input channel. This "geometric awareness" helps the model distinguish between objects that might overlap in a 2D image but are far apart in 3D space, leading to massive performance gains for small, distant objects.
While DALDet rivals the performance of leading 3D detectors like DSGN++ while being significantly faster, it remains dependent on the quality of input depth maps. Future work will explore integrating depth estimation and object detection into a single, end-to-end trainable multi-task network to further optimize efficiency.
Summary updated