Abstract
Monocular depth estimation plays a critical role in various computer vision and robotics applications such as localization, mapping, and 3D object detection. Recently, learning-based algorithms achieve huge success in depth estimation by training models with a large amount of data in a supervised manner. However, it is challenging to acquire dense ground truth depth labels for supervised training, and the unsupervised depth estimation using monocular sequences emerges as a promising alternative. Unfortunately, most studies on unsupervised depth estimation explore loss functions or occlusion masks, and there is little change in model architecture in that ConvNet-based encoder-decoder structure becomes a de-facto standard for depth estimation. In this paper, we employ a convolution-free Swin Transformer as an image feature extractor so that the network can capture both local geometric features and global semantic features for depth estimation. Also, we propose a Densely Cascaded Multi-scale Network (DCMNet) that connects every feature map directly with another from different scales via a top-down cascade pathway. This densely cascaded connectivity reinforces the interconnection between decoding layers and produces high-quality multi-scale depth outputs. The experiments on two different datasets, KITTI and Make3D, demonstrate that our proposed method outperforms existing state-of-the-art unsupervised algorithms.
Introduction
Estimating 3D depth from a single 2D image—monocular depth estimation—is a fundamental challenge in computer vision. It is essential for tasks ranging from autonomous driving to augmented reality. However, this task is inherently "ill-posed," as a single 2D projection can correspond to an infinite number of 3D configurations. While supervised learning has made significant strides, the requirement for dense, high-quality ground truth depth labels (often requiring expensive LiDAR setups) remains a major bottleneck.
Unsupervised depth estimation offers a compelling alternative. By leveraging monocular video sequences, models can learn to predict depth by minimizing the "photometric reprojection loss" between consecutive frames. Despite the success of this paradigm, architectural innovation has largely stagnated, with most models relying on standard Convolutional Neural Network (CNN) encoder-decoder structures. SwinDepth breaks this mold by introducing a Transformer-based architecture designed to capture both local geometry and global semantics.
The SwinDepth Architecture
The core innovation of SwinDepth lies in its departure from traditional CNN backbones in favor of two primary components: a Swin Transformer encoder and a Densely Cascaded Multi-scale Network (DCMNet) decoder.
Swin Transformer Encoder
Traditional CNNs are limited by their local receptive fields, making it difficult for them to model long-range dependencies across an image. In contrast, the Swin Transformer employs a hierarchical structure with "shifted windows" to calculate self-attention. This allows the network to capture global context—such as the relationship between a distant horizon and the immediate foreground—while maintaining manageable computational costs. By using a convolution-free encoder, SwinDepth can extract richer semantic features that are often missed by standard convolutional layers.
Densely Cascaded Multi-scale Network (DCMNet)
A common issue in depth estimation decoders is the "attenuation" of high-level semantic information as features are upsampled. To combat this, the authors propose DCMNet. Unlike standard skip connections that simply link corresponding encoder and decoder layers, DCMNet utilizes top-down cascade addition and dense concatenation. This ensures that every feature map is directly connected to others across different scales, reinforcing the interconnection between decoding layers and producing high-fidelity multi-scale depth outputs.
Global Contextual Prior
To further enhance the model's understanding of complex scenes, a Pyramid Pooling Module (PPM) is integrated into the first stage of the decoder. This module provides a global contextual prior, helping the model resolve ambiguities in large, textureless regions or complex environmental layouts.
Experimental Evaluation
The researchers evaluated SwinDepth on two prominent benchmarks: KITTI and Make3D.
KITTI Performance
On the KITTI 2015 dataset, SwinDepth established a new state-of-the-art for unsupervised monocular depth estimation. It achieved an Absolute Relative (Abs Rel) error of 0.106, significantly outperforming established models like Monodepth2 (0.115) and the much larger PackNet-SfM (0.111).
Beyond numerical metrics, SwinDepth demonstrated superior qualitative results. The model produces remarkably sharp boundaries for challenging, thin objects such as poles, traffic lights, and tree branches—areas where CNN-based models typically struggle with "bleeding" or blurred edges.
Generalization and Efficiency
To test the robustness of the model, it was evaluated on the Make3D dataset in a "zero-shot" manner (without any training on Make3D data). SwinDepth consistently outperformed previous unsupervised methods, proving that its learned features generalize well across different camera parameters and environments.
Furthermore, SwinDepth achieves this performance with high efficiency. While models like PackNet-SfM utilize approximately 120 million parameters, SwinDepth achieves better accuracy with only ~25 million parameters, representing a significant reduction in model size without sacrificing power.
Conclusion and Future Directions
SwinDepth demonstrates that moving beyond the "de-facto" CNN standard can yield substantial gains in unsupervised depth estimation. By combining the global awareness of the Swin Transformer with the dense connectivity of DCMNet, the model achieves a new level of precision and detail.
While the model is more efficient than standard Vision Transformers, it still presents a higher computational load than lightweight models like ResNet-18. Future work may focus on optimizing the architecture for real-time performance on edge devices. Additionally, addressing the "static scene" assumption by explicitly modeling moving objects could further refine depth accuracy in dynamic urban environments.
Summary updated