EdgeConv with Attention Module for Monocular Depth Estimation

Minhyeok Lee, Sangwon Hwang, Chaewon Park, Sangyoun Lee

Abstract

Monocular depth estimation is a critical task for autonomous systems, yet it remains inherently ill-posed due to the lack of explicit 3D geometric data in 2D images. This paper presents a novel framework that treats image patches as nodes in a graph, applying Edge Convolution (EdgeConv)—a technique typically used for 3D point clouds—to extract structural information. By incorporating Patch-Wise EdgeConv Modules (PEM) and an EdgeConv Attention Module (EAM) into a Feature Pyramid Network (FPN), the proposed model effectively captures local geometric transitions and global context. Experimental results on NYU Depth V2 and KITTI benchmarks demonstrate state-of-the-art performance, particularly in reducing Root Mean Square Error (RMSE).

Introduction

Predicting the distance of objects from a single 2D camera, known as monocular depth estimation, is essential for applications in robotics and autonomous driving. Despite its importance, the task is challenging because standard Convolutional Neural Networks (CNNs) primarily excel at identifying textures and semantic features but often fail to capture the underlying 3D structure of a scene. This is especially evident in scenarios with complex surfaces, thin objects, or extreme lighting conditions.

Traditional methods often rely on auxiliary data, such as surface normals, to improve geometric understanding. However, obtaining such data is difficult. This research proposes a method to extract the 3D structural "DNA" of a scene directly from the spatial relationships between image patches, bridging the gap between 2D computer vision and 3D geometric deep learning.

Methodology

The core of the proposed architecture is a Feature Pyramid Network (FPN) enhanced by two novel components: the Patch-Wise EdgeConv Module (PEM) and the EdgeConv Attention Module (EAM).

Figure 4: The proposed FPN-based architecture integrating PEM and EAM for depth prediction.

Patch-Wise EdgeConv Module (PEM)

EdgeConv was originally designed to process 3D point clouds by considering a node and its $k$-nearest neighbors in feature space. In this work, image patches are treated as nodes. The PEM calculates the relative difference between a patch and its neighbors rather than just processing absolute feature values. This focus on transitions allows the model to learn the "edges" of depth, which are critical for defining object boundaries and scene layout.

EdgeConv Attention Module (EAM)

The EAM aggregates features from PEMs across multiple scales. By applying a self-attention mechanism to these edge features, the module identifies which structural relationships are most significant for the final depth prediction. This combination of local structural detail and global context ensures a robust estimation across diverse environments.

Graph-Based Structural Learning

The fundamental innovation of this paper lies in the transition from pixel-based processing to graph-based patch relationships. By treating patches as nodes, the model can leverage geometric deep learning techniques to understand the scene's topology.

Figure 1: Conceptual overview of patch-wise graph nodes and the EdgeConv operation.

As shown in Figure 1, the model identifies relationships ($\xi_{ij}$) between a central patch ($p_i$) and its neighbors ($p_j$). The EdgeConv operation (shown in part b) processes the feature vector $e_i$ and the difference vector $e_i - e_j$. This approach is particularly effective at handling "optical illusions" that typically confuse CNNs, such as reflections on glass or thin structures like traffic sign poles, by inferring depth from the surrounding context and relative changes.

Experimental Results

The model was evaluated on the NYU Depth V2 (indoor) and KITTI (outdoor) datasets using various backbones, including MobileNetV2, DenseNet161, and EfficientNet-B6.

  • NYU Depth V2: The model achieved a state-of-the-art RMSE of 0.373, significantly outperforming previous benchmarks. Visualizations indicated sharper object contours and better handling of low-light indoor scenes.
  • KITTI: On the Eigen split (80m range), the EfficientNet-B6 variant achieved an RMSE of 2.642, the lowest reported among compared methods.

These results confirm that the integration of PEM and EAM consistently improves performance regardless of the underlying encoder architecture.

Conclusion and Future Work

This paper demonstrates that graph-based convolution can be successfully adapted for 2D monocular depth estimation. By focusing on the relative differences between image patches, the model captures structural information that standard CNNs often miss. While the $k$-nearest neighbor calculations introduce some computational complexity, the gains in accuracy and robustness are substantial.

Future research may focus on optimizing the real-time performance of these modules for deployment on mobile robotic platforms and exploring multi-task learning frameworks that combine depth estimation with semantic segmentation using shared graph-based features.

Summary updated

Abstract

Monocular depth estimation is an especially important task in robotics and autonomous driving, where 3D structural information is essential. However, extreme lighting conditions and complex surface objects make it difficult to predict depth in a single image. Therefore, to generate accurate depth maps, it is important for the model to learn structural information about the scene. We propose a novel Patch-Wise EdgeConv Module (PEM) and EdgeConv Attention Module (EAM) to solve the difficulty of monocular depth estimation. The proposed modules extract structural information by learning the relationship between image patches close to each other in space using edge convolution. Our method is evaluated on two popular datasets, the NYU Depth V2 and the KITTI Eigen split, achieving state-of-the-art performance. We prove that the proposed model predicts depth robustly in challenging scenes through various comparative experiments.
BibTeX
@inproceedings{lee2022edgeconv,
  author = {Minhyeok Lee and Sangwon Hwang and Chaewon Park and Sangyoun Lee},
  title = {{EdgeConv with Attention Module for Monocular Depth Estimation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
  url = {https://openreview.net/forum?id=1iqu4dVx8gu},
}
Figure