Abstract
Monocular depth estimation is a challenging task due to its inherently ill-posed nature. While modern convolutional neural networks (CNNs) effectively capture global scene structures, they often struggle with local pixel relationships, leading to blurry object boundaries. This paper introduces the Patch-Wise Attention (PWA) network, which applies dedicated attention modules to localized image patches. By focusing on regional geometric nuances, PWA achieves sharper boundaries and state-of-the-art performance on the KITTI and NYU Depth V2 benchmarks.
Introduction
Predicting 3D distance from a single 2D image is mathematically ill-posed because a single 2D projection can correspond to infinite 3D configurations. Consequently, AI models must rely heavily on contextual cues and learned patterns.
Current CNN-based methods are proficient at understanding global scene layout but frequently fail to model local pixel relationships accurately. Intuitively, neighboring pixels on the same object should share similar depth values, while pixels at an edge should exhibit sharp transitions. Global attention mechanisms often overlook these critical local details, resulting in "depth bleeding" between foreground and background objects.
Proposed Method: Patch-Wise Attention
The authors propose the Patch-Wise Attention (PWA) module, a plug-and-play component designed for standard encoder-decoder architectures. The PWA module processes feature maps in four distinct stages:
- Patch Extraction: Decoder feature maps are divided into localized patches (e.g., $s_p = 64$).
- Patch-Wise Channel Attention: Each patch utilizes a dedicated Multi-Layer Perceptron (MLP) to determine the importance of specific feature channels relative to its local context.
- Patch-Wise Spatial Attention: A convolutional layer learns pixel-wise relevance within each patch, enhancing the model's ability to resolve local shapes and edges.
- Re-assembly: Refined patches are re-stitched and integrated with global features via skip connections.
By treating each patch as a semi-independent sub-problem, the network avoids the distractions of global context when refining local geometric details like table corners or vehicle silhouettes.
Experimental Results
The PWA network was evaluated on two industry-standard datasets: KITTI (outdoor driving) and NYU Depth V2 (complex indoor scenes).
Quantitative Performance
At the time of its release, the PWA model ranked #1 on the official KITTI online leaderboard, achieving a Scale-Invariant Logarithmic error (SILog) of 11.45. On the NYU Depth V2 dataset, the model reduced Absolute Relative Error (AbsRel) and Root Mean Square Error (RMSE) by approximately 4% compared to previous state-of-the-art methods using identical backbones.
Qualitative Analysis
Visual comparisons demonstrate that PWA produces significantly "cleaner" depth maps. While baseline models often generate blurry "blobs" for distant or thin objects, PWA maintains distinct silhouettes for cars, poles, and furniture.
Key Insights
- Locality is Key: Depth is a highly local property. A pixel's depth is most strongly correlated with its immediate neighbors, making local attention more effective for boundary preservation than global attention.
- Position-Specific Learning: By assigning unique attention parameters to different patches, the model learns that patches in different image regions (e.g., road vs. sky) require different feature processing.
- Noise Robustness: The PWA module acts as an intelligent filter, leveraging local context to smooth out noise in LiDAR training data while preserving sharp geometric transitions.
Limitations and Future Work
The primary hyperparameter introduced is the patch size ($s_p$). The authors identified $s_p = 64$ as optimal; smaller patches lose necessary context, while larger patches suffer from the same blurring issues as global attention. Future research may explore more sophisticated local relationship layers to handle challenging scenarios like transparent surfaces or high reflectivity.
Summary updated