Abstract
Monocular Depth Estimation (MDE) typically requires large, accurately labeled datasets, which are often difficult to obtain. In real-world scenarios, depth labels are frequently "defective," containing missing or incorrect values for regions like the sky or reflective surfaces. We present ANUW, a weakly-supervised framework designed to overcome these data limitations. By integrating a nested U-net architecture (ANU) with dual attention mechanisms and an iterative soft-labeling process, the model learns to self-correct training errors. Experimental results on the Mai, KITTI, and DIODE datasets demonstrate that ANUW achieves state-of-the-art performance, producing more realistic and detailed depth maps even when trained on imperfect data.
Introduction
Predicting 3D depth from a single 2D RGB image is an inherently underdetermined problem. While deep learning has significantly advanced MDE, most models rely on high-quality ground truth data. However, sensors often produce "defective" labels—for instance, failing to capture depth for the sky, resulting in zero-values that mislead standard supervised training. This paper addresses the challenge of training robust MDE models on datasets with partially incorrect or incomplete ground truth.
Methodology
The proposed ANUW (Attention Nested U-net with Weakly-supervised learning) framework introduces three core innovations to handle defective data and improve estimation accuracy.
1. ANU Architecture
The Attention Nested U-net (ANU) builds upon the U-net++ structure. It utilizes dense residual connections and inter-layer skipping to preserve high-resolution spatial details that are typically lost during downsampling operations. This architecture ensures that fine-grained features are effectively fused into the final depth prediction.
2. Dual Attention Modules
To enhance the model's ability to capture contextual dependencies, two attention modules are integrated:
- Adaptive Weight Channel Attention (AWCA): Learns the relative importance of different feature channels for depth estimation and weights them accordingly.
- Patch Second Non-Local (PSNL): Implements spatial attention using second-order statistics. This allows the model to understand long-range relationships between distant image regions without the computational burden of standard non-local blocks.
3. Iterative Soft Label Generation
This weakly-supervised component allows the model to "self-correct." During training, the system identifies regions with missing or invalid labels (such as the sky). It then uses its own current predictions to fill these gaps, generating "soft labels" that serve as the ground truth for subsequent training iterations.
Datasets and Evaluation
The model was evaluated on three distinct datasets:
- Mai Depth Dataset: A "defective" dataset from the Mobile AI MDE challenge, characterized by incorrect labels in sky regions.
- KITTI: A standard benchmark for outdoor autonomous driving scenes.
- DIODE: A dense indoor and outdoor dataset used to verify generalization capabilities.
Key Results and Insights
The ANUW framework consistently outperformed established baselines, including standard U-net, FCRN, and Monodepth2.
- Robustness to Defective Data: On the Mai dataset, ANUW achieved a Si-RMSE of 0.582, a significant improvement over the U-net++ baseline (0.612).
- Superior Generalization: Despite training on "messy" data, ANUW showed excellent performance on clean benchmarks. On KITTI, it achieved an SSIM of 0.875, surpassing other tested methods.
- Qualitative Quality: Visual inspections reveal that ANUW produces smoother, more realistic depth maps, particularly in background and sky regions where other models often generate noisy artifacts.
Conclusion
The ANUW framework demonstrates that models can effectively "teach themselves" to ignore and correct faulty training data. By combining advanced architectural design with iterative self-supervision, it achieves high accuracy and strong generalization. Future work will explore deep imbalanced regression to improve performance on the "long-tail" distribution of depth values, specifically for very near or very far objects.
Summary updated