Monocular Depth Estimation with Adaptive Geometric Attention

Taher Naderi, Amir Sadovnik, Jason P. Hayward, Hairong Qi

Abstract

Monocular depth estimation—predicting 3D distance from a single 2D image—is a notoriously difficult "ill-posed" problem. This paper introduces a lightweight Adaptive Geometric Attention (AGA) module that helps neural networks distinguish between misleading texture edges and true geometric boundaries. By adding only 0.03% more parameters to a standard model, the authors achieve state-of-the-art accuracy on major benchmarks, producing sharper and more physically consistent depth maps.

Introduction

When humans view a photograph, we intuitively use cues like perspective and object size to understand depth. Computers struggle because a single 2D image lacks the mathematical information required to uniquely determine the third dimension. Existing deep learning models often produce "blurry" or distorted depth maps due to two primary factors:

  1. Downsampling: Encoders shrink image resolution to capture global context, often sacrificing fine spatial details.
  2. Texture vs. Geometry: RGB images are saturated with "edges" caused by colors and textures (e.g., grout lines in a brick wall) that do not represent physical depth changes. Conversely, critical depth boundaries (e.g., the corner of a white cabinet against a white wall) may be nearly invisible in the RGB domain.

The authors address this by creating a mechanism that forces models to prioritize geometric edges over superficial color changes.

Methodology: Adaptive Geometric Attention (AGA)

The researchers proposed the AGA module, designed to be integrated into standard encoder-decoder architectures. Unlike standard attention mechanisms, AGA focuses on local correlations to identify structural boundaries.

Visualization of geometric attention maps capturing structural boundaries.

Key components of the module include:

  • Cross-Correlation Guidance: The module calculates similarity between encoder features (RGB) and decoder features (depth) to align representations.
  • Cosine Similarity Constraint: By using the absolute value of cosine similarity between local features, the model enforces a geometric constraint that aligns internal features with the physical scene structure.
  • Sensitivity Enhancement: An amplification function ($SA \exp(SA)$) is used to sharpen the attention signal, making the model more sensitive to subtle correlations.
  • Hybrid Loss Function: The model is optimized using a combination of Weighted Cross-Entropy (WCEL), Virtual Normal Loss (VNL) for surface geometry, and standard $L_2$ loss.

Experimental Results

The approach was evaluated on the NYUDV2 (indoor) and KITTI (outdoor) benchmarks. The model consistently outperformed or matched state-of-the-art methods, achieving a record low Relative Error of 0.097 on NYUDV2.

Comparison of input RGB, predicted depth map, and the resulting geometric attention maps.

Key Insights

  • Local is Better: While global attention (like Transformers) is popular, this paper argues that local attention is superior for depth estimation as it focuses on the immediate neighborhoods where geometric boundaries occur, while remaining computationally efficient.
  • Extreme Efficiency: The AGA module adds only 28,672 parameters to a base model of over 90 million—a negligible 0.03% increase for a significant performance gain.
  • Texture Filtering: Qualitative results show the module successfully "filters out" texture noise (like brick patterns) to find the underlying geometry, resulting in much sharper object boundaries.

Limitations and Future Work

Despite its efficiency, the authors noted that adding excessive multiplication operations can complicate gradient optimization. Performance is also sensitive to the hyperparameter balancing of the hybrid loss functions. Future research may explore more complex local operations to further refine the guidance provided by depth features during training.

Summary updated

Abstract

ingle image depth estimation is an ill-posed problem. That is, it is not mathematically possible to uniquely estimate the 3rd dimension (or depth) from a single 2D image. Hence, additional constraints need to be incorporated in order to regulate the solution space. In this paper, we explore the idea of constraining the model by taking advantage of the similarity between the RGB image and the corresponding depth map at the geometric edges of the 3D scene for more accurate depth estimation. We propose a general light-weight adaptive geometric attention module that uses the cross-correlation between the encoder and the decoder as a measure of this similarity. More precisely, we use the cosine similarity between the local embedded features in the encoder and the decoder at each spatial point. The proposed module along with the encoder-decoder network is trained in an end-to-end fashion and achieves superior and competitive performance in comparison with other state-of-the-art methods. In addition, adding our module to the base encoder-decoder model adds only an additional 0.03% (or 0.0003) parameters. Therefore, this module can be added to any base encoder-decoder network without changing its structure to address any task at hand.
BibTeX
@inproceedings{naderi2022monocular,
  author = {Taher Naderi and Amir Sadovnik and Jason P. Hayward and Hairong Qi},
  title = {{Monocular Depth Estimation with Adaptive Geometric Attention}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
  doi = {10.1109/wacv51458.2022.00069},
}
Figure