Unsupervised High-Resolution Depth Learning From Videos With Dual Networks

Junsheng Zhou, Yuwang Wang, Kaihuai Qin, Wenjun Zeng

Introduction

Unsupervised depth learning has emerged as a powerful alternative to supervised methods, primarily by leveraging "view synthesis" as a supervisory signal. In this framework, a model predicts depth and camera motion to reconstruct one video frame from another, using the reconstruction error as a training loss. However, standard approaches often down-sample images to manage computational costs, leading to three critical failures: the inability to distinguish depth for distant objects, the loss of slim structures like poles, and blurry object boundaries that pose safety risks in autonomous driving.

This paper introduces a dual-network architecture designed to estimate high-resolution depth maps without ground-truth labels. By decoupling global semantic extraction from fine-detail refinement, the proposed method achieves state-of-the-art accuracy on benchmarks like KITTI while maintaining high computational efficiency.

Dual-Network Architecture

The core innovation is a Dual Network architecture that processes visual information at two distinct scales to balance semantic understanding with spatial precision.

The proposed framework: (a) shows the standard unsupervised depth/pose pipeline, while (b) details the dual-network architecture consisting of LR-Net, HR-Net, and the SA-Attention module.
  • LR-Net (Low-Resolution Network): A deep encoder-decoder that processes $128 \times 416$ images to capture "global semantics," providing the overall structural context of the scene.
  • HR-Net (High-Resolution Network): A lightweight, shallow network processing $384 \times 1248$ images. It integrates deep features from the LR-Net with high-resolution inputs to sharpen boundaries and recover fine-grained details.
  • SA-Attention (Self-assembled Attention): This module facilitates information sharing between the networks by calculating pixel similarity based on features, color, and spatial distance. This is particularly effective for "low-texture" regions like flat roads where standard matching signals are weak.
  • Feature Reconstruction Loss: To enhance supervision, the authors warp deep decoder features rather than relying solely on pixel-matching, providing a more robust signal during training.

Key Insights and Results

The separation of concerns—using deep layers for meaning and shallow layers for edges—allows the model to handle high-resolution data with fewer GFLOPs than a standard deep network. Quantitatively, the dual-network (using a ResNet50 backbone) requires only 25.80 GFLOPs, compared to 33.11 GFLOPs for a standard ResNet18 processing the same high-resolution input.

Performance on KITTI and Generalization

On the KITTI benchmark, the model achieved an Absolute Relative Error (Abs Rel) of 0.121, outperforming previous unsupervised methods. The high-resolution input specifically improved accuracy for objects further than 20 meters away by allowing the model to perceive sub-pixel disparities that are lost at lower resolutions.

Qualitative comparison on the KITTI dataset. The proposed method ("Ours") demonstrates significantly sharper boundaries and better reconstruction of slim structures compared to previous methods like Zhou et al. and GeoNet.

Qualitative results highlight the model's ability to reconstruct slim objects like traffic poles, which are typically invisible to unsupervised models. Furthermore, the model demonstrated strong generalization capabilities when tested on the Make3D dataset without any additional fine-tuning.

Conclusion and Future Work

While the dual-network approach significantly advances unsupervised depth estimation, limitations remain. Like most Structure-from-Motion (SfM) based methods, it assumes a static world; dynamic objects like moving cars can still cause depth artifacts. Additionally, reflective and transparent surfaces violate the constant appearance assumption, remaining a challenge for the field. Future work will likely focus on explicitly modeling dynamic objects to enhance robustness for real-world autonomous navigation.

Summary updated

Abstract

Unsupervised depth learning takes the appearance difference between a target view and a view synthesized from its adjacent frame as supervisory signal. Since the supervisory signal only comes from images themselves, the resolution of training data significantly impacts the performance. High-resolution images contain more fine-grained details and provide more accurate supervisory signal. However, due to the limitation of memory and computation power, the original images are typically down-sampled during training, which suffers heavy loss of details and disparity accuracy. In order to fully explore the information contained in high-resolution data, we propose a simple yet effective dual networks architecture, which can directly take high-resolution images as input and generate high-resolution and high-accuracy depth map efficiently. We also propose a Self-assembled Attention (SA-Attention) module to handle low-texture region. The evaluation on the benchmark KITTI and Make3D datasets demonstrates that our method achieves state-of-the-art results in the monocular depth estimation task.
BibTeX
@inproceedings{zhou2019unsupervised,
  author = {Junsheng Zhou and Yuwang Wang and Kaihuai Qin and Wenjun Zeng},
  title = {{Unsupervised High-Resolution Depth Learning From Videos With Dual Networks}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00697},
}
Figure