Understanding Depth Map Progressively: Adaptive Distance Interval Separation for Monocular 3d Object Detection

Xianhui Cheng, Shoumeng Qiu, Zhikang Zou, Jian Pu, Xiangyang Xue

TL;DR

The Adaptive Distance Interval Separation Network (ADISN) improves monocular 3D object detection by partitioning estimated depth maps into distance-based subgraphs. This technique creates sharp, learned edges that allow standard CNNs to extract precise localization features, significantly outperforming existing depth-assisted methods on the KITTI benchmark.

What Problem Does This Paper Address?

Monocular 3D object detection—identifying the position, size, and orientation of objects from a single 2D image—is a "holy grail" for low-cost autonomous systems. Unlike expensive LiDAR sensors that provide direct 3D points, a single camera lacks depth information.

Current researchers try to fill this gap using estimated depth maps. However, they usually treat these maps either as regular images (which ignores their physical meaning) or as "pseudo-LiDAR" point clouds (which are highly sensitive to estimation errors). There is a missing middle ground: a way to use the strengths of Convolutional Neural Networks (CNNs) to process depth data in a way that respects its physical, distance-based nature.

Methods

The authors introduce ADISN, which features three novel components:

  1. Adaptive Distance Interval Separation (ADIS): Instead of feeding a raw depth map into a CNN, ADISN learns to slice the depth map into several "sub-depth maps" (SDs) based on distance intervals. For example, one slice might contain everything 5–10 meters away, and another 10–20 meters. This creates sharp artificial edges where objects are "cut" by the interval boundaries. Since CNNs are naturally great at edge detection, these boundaries provide strong cues for object localization.
  2. Uncertainty Module: Monocular depth estimation is notoriously noisy, especially for distant objects or the sky. ADISN learns an uncertainty map that assigns a confidence score to every pixel. This allows the model to "distrust" unreliable depth regions and rely more on the original RGB image in those areas.
  3. Feature Decoupling: The network uses separate "heads" for different tasks. An appearance branch focuses on category and visual features (fusing RGB and depth), while a localization branch uses the ADIS features and uncertainty map to pinpoint the object's 3D coordinates $(x, y, z)$.
The ADISN framework uses separate branches for appearance and localization, incorporating adaptive depth separation and uncertainty maps.

Datasets

The model was evaluated on the KITTI monocular 3D object detection benchmark, a standard in the field. It contains:

  • 7,481 training samples and 7,518 testing samples.
  • Evaluations across three categories: Car, Pedestrian, and Cyclist.
  • Three difficulty levels: Easy, Moderate, and Hard, based on object size, occlusion, and truncation.

Key Insights and Ideas

  • Manufacturing Edges: The most innovative "aha moment" is realizing that depth maps are often too "smooth" for CNNs to extract precise location features. By thresholding them into intervals, the authors effectively "manufacture" high-contrast textures that represent physical distance, making the depth data much more "digestible" for standard vision architectures.
  • Bridging the Gap: ADISN treats depth maps as a hybrid form—somewhere between a 2D image and a 3D point cloud. This allows it to use the efficient processing of 2D CNNs while maintaining the physical intuition of 3D space.
  • Learning to Distrust: By jointly learning uncertainty, the model acknowledges that its auxiliary depth input isn't perfect. This robustness is key to handling real-world scenes where depth estimation often fails at the horizon.
The uncertainty map (bottom) shows higher confidence (red) for nearby, reliable areas and lower confidence (blue) for distant or sky regions.

Key Results

ADISN achieved state-of-the-art performance on the KITTI test set for the "Car" category:

  • Ranked 1st among monocular methods in the Easy, Moderate, and Hard categories for 3D Average Precision (AP 3D).
  • Significant Gains: It outperformed the baseline (M3D-RPN) by 3.11% on Easy, 2.43% on Moderate, and 3.00% on Hard tasks.
  • Qualitative Success: Visualizations show that ADISN is particularly effective at detecting distant or partially occluded vehicles that baseline models often miss.
Qualitative results show ADISN (red boxes) accurately detecting distant and occluded cars compared to the ground truth (green).

Limitations and Future Work

  • Computational Trade-off: The number of distance intervals ($n_d$) is a hyperparameter. While more intervals generally improve accuracy, they also increase computational cost. The authors found that performance plateaus around 16 intervals and settled on 8 for their final model.
  • Dependency on Depth Estimators: While the uncertainty module helps, the system still relies on an external depth estimator (like DORN or PSMNet). If the initial depth map is catastrophically wrong, the model's performance will suffer.
  • Future Directions: The authors suggest exploring ways to infer 3D space that are even more independent of RGB inputs, potentially refining how knowledge is distilled from LiDAR-based "teacher" models into monocular "student" models.

Summary updated

Abstract

Monocular 3D object detection aims to locate objects in different scenes with just a single image. Due to the absence of depth information, several monocular 3D detection techniques have emerged that rely on auxiliary depth maps from the depth estimation task. There are multiple approaches to understanding the representation of depth maps, including treating them as pseudo-LiDAR point clouds, leveraging implicit end-to-end learning of depth information, or considering them as an image input. However, these methods have certain drawbacks, such as their reliance on the accuracy of estimated depth maps and suboptimal utilization of depth maps due to their image-based nature. While LiDAR-based methods and convolutional neural networks (CNNs) can be utilized for pseudo point clouds and depth maps, respectively, it is always an alternative. In this paper, we propose a framework named the Adaptive Distance Interval Separation Network (ADISN) that adopts a novel perspective on understanding depth maps, as a form that lies between LiDAR and images. We utilize an adaptive separation approach that partitions the depth map into various subgraphs based on distance and treats each of these subgraphs as an individual image for feature extraction. After adaptive separations, each subgraph solely contains pixels within a learned interval range. If there is a truncated object within this range, an evident curved edge will appear, which we can leverage for texture extraction using CNNs to obtain rich depth information in pixels. Meanwhile, to mitigate the inaccuracy of depth estimation, we designed an uncertainty module. To take advantage of both images and depth maps, we use different branches to learn localization detection tasks and appearance tasks separately. Our approach significantly enhances the baseline and outperforms depth-assisted techniques, as shown by our extensive experiments on the KITTI monocular 3D object detection benchmark.
BibTeX
@inproceedings{cheng2023understanding,
  author = {Xianhui Cheng and Shoumeng Qiu and Zhikang Zou and Jian Pu and Xiangyang Xue},
  title = {{Understanding Depth Map Progressively: Adaptive Distance Interval Separation for Monocular 3d Object Detection}},
  booktitle = {IJCNN},
  year = {2023},
  doi = {10.1109/ijcnn54540.2023.10191066},
}

Loading related content...

Figure