Abstract
Monocular depth estimation is an especially important task in robotics and autonomous driving, where 3D structural information is essential. However, extreme lighting conditions and complex surface objects make it difficult to predict depth in a single image. Therefore, to generate accurate depth maps, it is important for the model to learn structural information about the scene. We propose a novel Patch-Wise EdgeConv Module (PEM) and EdgeConv Attention Module (EAM) to solve the difficulty of monocular depth estimation. The proposed modules extract structural information by learning the relationship between image patches close to each other in space using edge convolution. Our method is evaluated on two popular datasets, the NYU Depth V2 and the KITTI Eigen split, achieving state-of-the-art performance. We prove that the proposed model predicts depth robustly in challenging scenes through various comparative experiments.
Introduction
Monocular depth estimation (MDE)—the task of predicting distance from a single camera view—is a fundamental yet ill-posed challenge for autonomous systems. Standard Convolutional Neural Networks (CNNs) excel at pattern recognition but often struggle with the structural reasoning required for complex geometries or extreme lighting conditions. This paper presents a novel architectural shift by applying Edge Convolution (EdgeConv), a technique typically used for 3D point clouds, to 2D image features to better capture 3D structural information.
Methodology
The researchers integrated two novel modules into a Feature Pyramid Network (FPN) architecture to enhance depth prediction accuracy.
- Patch-Wise EdgeConv Module (PEM): Instead of pixel-level processing, the PEM divides feature maps into patches. It uses a k-Nearest Neighbor (k-NN) algorithm in the feature space to find related patches and applies EdgeConv to learn the "edges" or geometric relationships between them.
- EdgeConv Attention Module (EAM): Positioned at the network's bottleneck, the EAM applies a self-attention mechanism to the structural features from the PEM. This allows the model to combine local structural insights with global contextual information, increasing robustness to scene-wide lighting changes.
Graph-Based Structural Learning
The core innovation is treating 2D image patches as nodes in a graph. By using k-NN in the feature space, the model can relate patches that are physically distant but structurally similar, overcoming the local-only receptive field of standard convolutions.
As illustrated in the figure above, the model computes edge features $\xi_{ij}$ between a central patch $p_i$ and its neighbors $p_j$. This approach enables the network to reason about the relative geometry of the scene, making it significantly more robust to common failure points like reflections on glass or deep shadows.
Experimental Results
The model was rigorously tested on the NYU Depth V2 and KITTI datasets, achieving state-of-the-art (SOTA) results across several metrics, most notably in Root Mean Square Error (RMSE):
- NYU Depth V2: Achieved an RMSE of 0.373, outperforming previous leading methods.
- KITTI: On the 0-80m range, the model reached an RMSE of 2.642.
- Backbone Versatility: The PEM and EAM modules provided a performance boost regardless of the underlying backbone (e.g., MobileNetV2, DenseNet161, EfficientNet-B6), demonstrating their general utility.
Conclusion
By introducing graph-based reasoning into monocular depth estimation, this work successfully enhances the extraction of 3D structural information from 2D images. While the computational overhead of graph-based methods remains a challenge for real-time robotics, the significant gains in accuracy and robustness to challenging scene conditions mark a major step forward for the field.
Summary updated