A Weakly-Supervised Depth Estimation Network Using Attention Mechanism

Fang Gao, Jiabao Wang, Jun Yu, Yaoxiong Wang, Feng ShuangarXiv 2021 · arXiv

Abstract

Monocular Depth Estimation (MDE) typically requires large, accurately labeled datasets, which are often difficult to obtain. In real-world scenarios, depth labels are frequently "defective," containing missing or incorrect values for regions like the sky or reflective surfaces. We present ANUW, a weakly-supervised framework designed to overcome these data limitations. By integrating a nested U-net architecture (ANU) with dual attention mechanisms and an iterative soft-labeling process, the model learns to self-correct training errors. Experimental results on the Mai, KITTI, and DIODE datasets demonstrate that ANUW achieves state-of-the-art performance, producing more realistic and detailed depth maps even when trained on imperfect data.

Introduction

Predicting 3D depth from a single 2D RGB image is an inherently underdetermined problem. While deep learning has significantly advanced MDE, most models rely on high-quality ground truth data. However, sensors often produce "defective" labels—for instance, failing to capture depth for the sky, resulting in zero-values that mislead standard supervised training. This paper addresses the challenge of training robust MDE models on datasets with partially incorrect or incomplete ground truth.

Methodology

The proposed ANUW (Attention Nested U-net with Weakly-supervised learning) framework introduces three core innovations to handle defective data and improve estimation accuracy.

1. ANU Architecture

The Attention Nested U-net (ANU) builds upon the U-net++ structure. It utilizes dense residual connections and inter-layer skipping to preserve high-resolution spatial details that are typically lost during downsampling operations. This architecture ensures that fine-grained features are effectively fused into the final depth prediction.

The ANU architecture (a) utilizes dense connections and multi-scale feature fusion, supported by VGGblocks (b), to maintain high-resolution depth maps.

2. Dual Attention Modules

To enhance the model's ability to capture contextual dependencies, two attention modules are integrated:

  • Adaptive Weight Channel Attention (AWCA): Learns the relative importance of different feature channels for depth estimation and weights them accordingly.
  • Patch Second Non-Local (PSNL): Implements spatial attention using second-order statistics. This allows the model to understand long-range relationships between distant image regions without the computational burden of standard non-local blocks.

3. Iterative Soft Label Generation

This weakly-supervised component allows the model to "self-correct." During training, the system identifies regions with missing or invalid labels (such as the sky). It then uses its own current predictions to fill these gaps, generating "soft labels" that serve as the ground truth for subsequent training iterations.

The ANUW framework employs an iterative loop where the model's own predictions are used to generate soft labels, refining the training process for regions with missing ground truth.

Datasets and Evaluation

The model was evaluated on three distinct datasets:

  • Mai Depth Dataset: A "defective" dataset from the Mobile AI MDE challenge, characterized by incorrect labels in sky regions.
  • KITTI: A standard benchmark for outdoor autonomous driving scenes.
  • DIODE: A dense indoor and outdoor dataset used to verify generalization capabilities.

Key Results and Insights

The ANUW framework consistently outperformed established baselines, including standard U-net, FCRN, and Monodepth2.

  • Robustness to Defective Data: On the Mai dataset, ANUW achieved a Si-RMSE of 0.582, a significant improvement over the U-net++ baseline (0.612).
  • Superior Generalization: Despite training on "messy" data, ANUW showed excellent performance on clean benchmarks. On KITTI, it achieved an SSIM of 0.875, surpassing other tested methods.
  • Qualitative Quality: Visual inspections reveal that ANUW produces smoother, more realistic depth maps, particularly in background and sky regions where other models often generate noisy artifacts.

Conclusion

The ANUW framework demonstrates that models can effectively "teach themselves" to ignore and correct faulty training data. By combining advanced architectural design with iterative self-supervision, it achieves high accuracy and strong generalization. Future work will explore deep imbalanced regression to improve performance on the "long-tail" distribution of depth values, specifically for very near or very far objects.

Summary updated

BibTeX
@misc{gao2021weakly,
  author = {Fang Gao and Jiabao Wang and Jun Yu and Yaoxiong Wang and Feng Shuang},
  title = {{A Weakly-Supervised Depth Estimation Network Using Attention Mechanism}},
  year = {2021},
  eprint = {2107.04819},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2107.04819},
}
Figure