Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications

Feng Xue, Guirong Zhuo, Ziyuan Huang, Wufei Fu, Zhuoyue Wu, Marcelo H. Ang

TL;DR

This paper introduces DNet, a novel self-supervised monocular depth estimation pipeline that improves object-level depth accuracy and enables absolute depth estimation without relying on expensive external sensors like LiDAR. DNet achieves this through a new Dense Connected Prediction (DCP) layer for better object detail and a Dense Geometrical Constraint (DGC) module for robust scale recovery using ground plane information.

What Problem Does This Paper Address?

Estimating depth from a single camera image (monocular depth estimation) is crucial for applications like autonomous driving, but current self-supervised methods face two main challenges. First, they often produce imprecise object boundaries and "texture copying" artifacts, making it hard to discern fine details. Second, they typically estimate relative depth, meaning the scale of the scene is unknown. To get absolute depth, these methods usually require additional sensors (like LiDAR) or stereo cameras for training, which increases cost and system complexity. This paper aims to overcome these limitations by providing more accurate object-level depth and enabling absolute depth estimation using only monocular video data, making it more practical for real-world deployment.

Methods

DNet builds upon the Monodepth2 framework but introduces two key innovations:

  • Dense Connected Prediction (DCP) Layer: Traditional multi-scale depth prediction methods often upsample features from lower resolutions independently, leading to depth artifacts. The proposed DCP layer addresses this by densely connecting features across different scales hierarchically. It combines lower-resolution features (which provide more reliable object-level inference) with higher-resolution features (which capture local details). This fusion allows for more precise object boundary inference and smoother depth maps.
  • Dense Geometrical Constraint (DGC) Module: To recover the absolute scale factor without external sensors, DNet leverages the common presence of ground in autonomous driving scenes. The DGC module first calculates surface normals for each pixel. Then, it detects ground points by comparing their normals to an ideal ground normal. For each detected ground point, it estimates a potential camera height. Finally, it uses a statistical method (specifically, the median) to robustly estimate the overall camera height from these individual estimates. This estimated camera height is then used to compute the scale factor, allowing for absolute depth estimation.

Datasets

The experiments were conducted on the KITTI 2015 Eigen Split dataset. This dataset provides ground truth depth maps projected from LiDAR point clouds for evaluation. The paper notes that while ground truth for depth is available, there is no ground truth for scale factors, which is why methods like using the ratio between LiDAR-detected depth and estimated depth are typically used for scale evaluation.

Key Insights and Ideas

  • Hierarchical Feature Fusion is Key for Object Detail: The paper demonstrates that simply combining features from different resolutions isn't enough. Densely connecting and fusing these hierarchical features, as done in the DCP layer, is crucial for improving object boundary accuracy and reducing artifacts.
  • Ground Plane as a Free, Powerful Scale Cue: The DGC module highlights the significant potential of using the ground plane as a reliable and cost-effective source for absolute scale estimation in autonomous driving scenarios. This bypasses the need for expensive LiDAR or stereo setups for scale recovery.
  • Robustness through Statistical Aggregation: Using the median of estimated camera heights from multiple ground points makes the DGC module robust to outliers and variations in ground point detection.

Key Results

  • Improved Relative Depth: DNet with GT scale recovery achieved competitive results on standard depth estimation metrics, outperforming Monodepth2 by a small margin.
  • Competitive Absolute Depth with DGC: The DGC module for scale recovery performed comparably to using LiDAR ground truth. It showed significant improvements over previous methods that relied on additional sensors or pose information.
  • Object-Level Performance Boost: The DCP layer specifically improved object-level prediction performance, with gains in metrics like Absolute Relative error and RMSE.
  • DGC Robustness: The DGC module proved to be robust when ground points constitute more than 1.03% of the image pixels, performing comparably to GT scale recovery.

Limitations and Future Work

  • Ground Point Dependency: The DGC module's performance is inherently dependent on the presence of sufficient ground points in the image. In scenarios with very little visible ground, scale estimation might be less accurate. The paper notes that with extremely low ground point ratios, scale may be incorrectly estimated.
  • Future Work: The authors suggest that this work provides a foundation for further exploration of hierarchical features and scale recovery methods. Future research could focus on extending these ideas to more challenging scenarios or integrating them into more complex autonomous driving systems.

Summary updated

Abstract

In recent years, self-supervised methods for monocular depth estimation has rapidly become an significant branch of depth estimation task, especially for autonomous driving applications. Despite the high overall precision achieved, current methods still suffer from a) imprecise object-level depth inference and b) uncertain scale factor. The former problem would cause texture copy or provide inaccurate object boundary, and the latter would require current methods to have an additional sensor like LiDAR to provide depth ground-truth or stereo camera as additional training inputs, which makes them difficult to implement. In this work, we propose to address these two problems together by introducing DNet. Our contributions are twofold: a) a novel dense connected prediction (DCP) layer is proposed to provide better object-level depth estimation and b) specifically for autonomous driving scenarios, dense geometrical constrains (DGC) is introduced so that precise scale factor can be recovered without additional cost for autonomous vehicles. Extensive experiments have been conducted and, both DCP layer and DGC module are proved to be effectively solving the aforementioned problems respectively. Thanks to DCP layer, object boundary can now be better distinguished in the depth map and the depth is more continues on object level. It is also demonstrated that the performance of using DGC to perform scale recovery is comparable to that using ground-truth information, when the camera height is given and the ground point takes up more than 1.03% of the pixels. Code is available at https://github.com/TJ-IPLab/DNet.
BibTeX
@inproceedings{xue2020toward,
  author = {Feng Xue and Guirong Zhuo and Ziyuan Huang and Wufei Fu and Zhuoyue Wu and Marcelo H. Ang},
  title = {{Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=NF_vRXBDfmM},
}
Figure