Abstract
Monocular depth estimation (MDE) is a critical computer vision task that traditionally requires extensive, high-quality labeled datasets. We present an efficient training framework that leverages auxiliary tasks—specifically multi-label dense classification—alongside a frozen vision foundation model to enhance MDE performance. By employing an alternating training scheme with a task-focusing parameter, our approach achieves an average 11% improvement in depth quality. Notably, the method maintains baseline performance even with a 99% reduction in target training data, offering a robust solution for data-constrained scenarios.
Introduction
Monocular Depth Estimation (MDE)—the process of predicting 3D depth from a single 2D image—is notoriously data-hungry. Training robust models typically requires vast amounts of high-quality, densely labeled depth data, often necessitating expensive hardware like LiDAR or structured light sensors. While Multi-Task Learning (MTL) is a common strategy to leverage related vision tasks (such as semantic segmentation), it frequently suffers from "task interference," where conflicting gradients from different tasks degrade overall performance.
This paper introduces a resource-efficient alternating training scheme that utilizes a frozen pre-trained foundation model (DINOv2 ViT-Giant) to boost MDE performance without the computational burden of full-backbone fine-tuning or the pitfalls of standard MTL interference.
Methodology
The proposed architecture avoids fine-tuning the massive backbone by jointly training a shared Dense Prediction Transformer (DPT) decoder. The training process alternates between gradient steps for the primary MDE task and an auxiliary task. To mitigate task interference, the authors introduce a task-focusing parameter ($\alpha$). By setting $\alpha$ high (e.g., 0.9), the learning rate prioritizes the MDE task, treating the auxiliary signal as a regularizing support rather than an equal priority.
Three auxiliary tasks were evaluated: semantic segmentation, image reconstruction, and a novel Multi-Label Dense Classification (MLDC). MLDC collapses semantic masks into simple multi-label classification vectors, focusing on object presence rather than precise spatial boundaries. The authors also introduced MIX6, a diverse training pool combining six auxiliary datasets (ADE20K, COCO-Stuff, etc.), which proved to be the most effective auxiliary source.
Experimental Results
The proposed method achieved an average ~11% improvement in Absolute Relative Error (AbsRel) across various indoor and outdoor datasets compared to the DINOv2 baseline. Specific highlights include a 13.9% improvement on NYUv2 and a 22.2% gain on DIODE Outdoor. The framework also proved backbone-agnostic, providing a 2.5% gain when applied to the state-of-the-art Depth Anything model.
One of the most significant findings is the extreme data efficiency of this approach. The auxiliary training acts as a powerful regularization strategy, allowing the model to achieve or exceed baseline performance using as little as 1% to 20% of the target depth data.
Key Insights and Discussion
- Task Selection Over Volume: Improvements in MDE are driven more by the nature of the auxiliary task than by simply increasing data volume.
- MLDC Superiority: Surprisingly, MLDC often outperformed standard semantic segmentation. This suggests that identifying object presence provides sufficient semantic cues for depth, while low-quality segmentation boundaries might introduce counterproductive noise.
- Domain Sensitivity: The method showed a slight regression on the KITTI driving dataset (-0.6%). This failure is attributed to a domain mismatch between the MIX6 auxiliary data and the specific characteristics of sparse LiDAR ground truth in driving scenes.
Conclusion
By leveraging frozen foundation models and a novel task-focusing training scheme, this research demonstrates that auxiliary tasks can significantly enhance MDE performance while drastically reducing the need for expensive labeled depth data. Future work will focus on dynamic task balancing and bridging the domain gap for specialized environments like autonomous driving.
Summary updated