Abstract
Monocular Depth Estimation (MDE) is a critical task for robotics and augmented reality, yet it remains constrained by the high cost of acquiring dense depth labels. This paper presents a data-efficient training framework that leverages auxiliary tasks—such as semantic segmentation and dense classification—from unrelated datasets to improve MDE performance. By utilizing a shared decoder on a frozen DINOv2 backbone, the proposed method achieves an average 11% improvement in depth quality and demonstrates that auxiliary tasks can reduce the requirement for depth-labeled data by up to 99%.
Introduction
Predicting 3D depth from a single 2D image is a fundamental challenge in computer vision. While foundation models like DINOv2 provide robust feature representations, fine-tuning them for specific tasks like MDE is computationally expensive and often hindered by the scarcity of high-quality depth labels. Traditional Multi-Task Learning (MTL) approaches frequently encounter "task interference," where optimizing for one objective degrades another. This research explores how to effectively harness abundant, non-depth datasets to enhance depth prediction without full model fine-tuning.
Proposed Methodology
The authors introduce a lightweight pipeline designed for maximum data and computational efficiency. The architecture consists of three main components:
- Frozen Foundation Backbone: A pre-trained DINOv2 ViT-Giant serves as the feature extractor. By keeping this backbone frozen, the training focus shifts entirely to the task-specific components, drastically reducing resource requirements.
- Shared Dense Prediction Transformer (DPT) Decoder: Unlike standard MTL architectures that use separate decoders for different tasks, this method employs a shared decoder. This forces the model to learn a unified feature representation that is beneficial for both depth estimation and auxiliary tasks.
- Alternating Training Scheme: The model alternates between MDE and auxiliary task updates. A task-focusing parameter, $\alpha$, regulates the gradient flow, typically prioritizing depth estimation ($\alpha = 0.9$).
A key innovation is the use of Multi-Label Dense Classification (MLDC) as an auxiliary task. Rather than requiring pixel-perfect segmentation masks, MLDC identifies the presence of various classes within a scene. This global semantic understanding provides powerful spatial cues that significantly aid depth perception.
Key Insights and Results
The research yields several surprising findings regarding the relationship between semantic understanding and depth:
- Classification vs. Segmentation: MLDC often outperforms full semantic segmentation as an auxiliary task. This suggests that for depth estimation, understanding the scene's composition (e.g., "there is a bed in the foreground") is more valuable than precise boundary localization, which can often be noisy.
- Auxiliary Tasks as Data Augmentation: Auxiliary datasets act as a sophisticated form of augmentation. Learning to identify objects in diverse environments helps the shared decoder internalize spatial relationships that generalize back to the depth task.
- Exceptional Data Efficiency: Perhaps the most significant result is the reduction in required depth data. By incorporating auxiliary tasks, the model can match or exceed baseline performance using as little as 1% to 20% of the original depth-labeled training set.
Quantitatively, the method achieved an average 11% improvement over the DINOv2 baseline. Significant gains were observed in challenging scenarios, such as a 22.2% improvement on the DIODE Outdoor dataset and a 13.9% gain on NYUv2. Even when applied to state-of-the-art models like "Depth Anything," the approach yielded a 2.5% average performance boost.
Limitations and Future Outlook
The primary constraint identified is the domain gap. While the "MIX6" auxiliary dataset (a union of indoor and general-purpose datasets) significantly improved performance on indoor tasks, it offered no benefit for the KITTI autonomous driving dataset. This indicates that the effectiveness of auxiliary tasks is contingent on the relevance of the auxiliary data to the target domain.
Future research will focus on automating the selection of auxiliary tasks and exploring more diverse datasets to bridge the gap in specialized domains like autonomous driving.
Summary updated