Abstract
Monocular depth estimation is a critical task for autonomous systems, yet it remains inherently ill-posed due to the lack of explicit 3D geometric data in 2D images. This paper presents a novel framework that treats image patches as nodes in a graph, applying Edge Convolution (EdgeConv)—a technique typically used for 3D point clouds—to extract structural information. By incorporating Patch-Wise EdgeConv Modules (PEM) and an EdgeConv Attention Module (EAM) into a Feature Pyramid Network (FPN), the proposed model effectively captures local geometric transitions and global context. Experimental results on NYU Depth V2 and KITTI benchmarks demonstrate state-of-the-art performance, particularly in reducing Root Mean Square Error (RMSE).
Introduction
Predicting the distance of objects from a single 2D camera, known as monocular depth estimation, is essential for applications in robotics and autonomous driving. Despite its importance, the task is challenging because standard Convolutional Neural Networks (CNNs) primarily excel at identifying textures and semantic features but often fail to capture the underlying 3D structure of a scene. This is especially evident in scenarios with complex surfaces, thin objects, or extreme lighting conditions.
Traditional methods often rely on auxiliary data, such as surface normals, to improve geometric understanding. However, obtaining such data is difficult. This research proposes a method to extract the 3D structural "DNA" of a scene directly from the spatial relationships between image patches, bridging the gap between 2D computer vision and 3D geometric deep learning.
Methodology
The core of the proposed architecture is a Feature Pyramid Network (FPN) enhanced by two novel components: the Patch-Wise EdgeConv Module (PEM) and the EdgeConv Attention Module (EAM).
Patch-Wise EdgeConv Module (PEM)
EdgeConv was originally designed to process 3D point clouds by considering a node and its $k$-nearest neighbors in feature space. In this work, image patches are treated as nodes. The PEM calculates the relative difference between a patch and its neighbors rather than just processing absolute feature values. This focus on transitions allows the model to learn the "edges" of depth, which are critical for defining object boundaries and scene layout.
EdgeConv Attention Module (EAM)
The EAM aggregates features from PEMs across multiple scales. By applying a self-attention mechanism to these edge features, the module identifies which structural relationships are most significant for the final depth prediction. This combination of local structural detail and global context ensures a robust estimation across diverse environments.
Graph-Based Structural Learning
The fundamental innovation of this paper lies in the transition from pixel-based processing to graph-based patch relationships. By treating patches as nodes, the model can leverage geometric deep learning techniques to understand the scene's topology.
As shown in Figure 1, the model identifies relationships ($\xi_{ij}$) between a central patch ($p_i$) and its neighbors ($p_j$). The EdgeConv operation (shown in part b) processes the feature vector $e_i$ and the difference vector $e_i - e_j$. This approach is particularly effective at handling "optical illusions" that typically confuse CNNs, such as reflections on glass or thin structures like traffic sign poles, by inferring depth from the surrounding context and relative changes.
Experimental Results
The model was evaluated on the NYU Depth V2 (indoor) and KITTI (outdoor) datasets using various backbones, including MobileNetV2, DenseNet161, and EfficientNet-B6.
- NYU Depth V2: The model achieved a state-of-the-art RMSE of 0.373, significantly outperforming previous benchmarks. Visualizations indicated sharper object contours and better handling of low-light indoor scenes.
- KITTI: On the Eigen split (80m range), the EfficientNet-B6 variant achieved an RMSE of 2.642, the lowest reported among compared methods.
These results confirm that the integration of PEM and EAM consistently improves performance regardless of the underlying encoder architecture.
Conclusion and Future Work
This paper demonstrates that graph-based convolution can be successfully adapted for 2D monocular depth estimation. By focusing on the relative differences between image patches, the model captures structural information that standard CNNs often miss. While the $k$-nearest neighbor calculations introduce some computational complexity, the gains in accuracy and robustness are substantial.
Future research may focus on optimizing the real-time performance of these modules for deployment on mobile robotic platforms and exploring multi-task learning frameworks that combine depth estimation with semantic segmentation using shared graph-based features.
Summary updated