Abstract
Self-supervised monocular methods can efficiently learn depth information of weakly textured surfaces or reflective objects. However, the depth accuracy is limited due to the inherent ambiguity in monocular geometric modeling. In contrast, multi-frame depth estimation methods improve depth accuracy thanks to the success of Multi-View Stereo (MVS), which directly makes use of geometric constraints. Unfortunately, MVS often suffers from texture-less regions, non-Lambertian surfaces, and moving objects, especially in real-world video sequences without known camera motion and depth supervision. Therefore, we propose MOVEDepth, which exploits the MOnocular cues and VElocity guidance to improve multi-frame Depth learning. Unlike existing methods that enforce consistency between MVS depth and monocular depth, MOVEDepth boosts multi-frame depth learning by directly addressing the inherent problems of MVS. The key of our approach is to utilize monocular depth as a geometric priority to construct MVS cost volume, and adjust depth candidates of cost volume under the guidance of predicted camera velocity. We further fuse monocular depth and MVS depth by learning uncertainty in the cost volume, which results in a robust depth estimation against ambiguity in multi-view geometry. Extensive experiments show MOVEDepth achieves state-of-the-art performance: Compared with Monodepth2 and PackNet, our method relatively improves the depth accuracy by 20% and 19.8% on the KITTI benchmark. MOVEDepth also generalizes to the more challenging DDAD benchmark, relatively outperforming ManyDepth by 7.2%.
Introduction
Depth estimation is a critical component for autonomous systems, enabling navigation and obstacle avoidance. While LiDAR sensors provide high-precision data, they are expensive and difficult to scale. Consequently, researchers have turned to self-supervised depth learning from standard video sequences. Current approaches generally fall into two categories: monocular methods, which are robust to textureless surfaces but lack absolute geometric accuracy, and Multi-View Stereo (MVS) methods, which leverage parallax for precision but fail when camera motion is minimal or surfaces are non-Lambertian.
MOVEDepth addresses this dichotomy by bridging the gap between monocular robustness and multi-frame geometric precision. Rather than simply enforcing consistency between these two paradigms, MOVEDepth uses monocular cues to resolve the inherent geometric failures of MVS, particularly in challenging real-world scenarios.
Methodology
The MOVEDepth framework integrates three core innovations: Monocular Depth Priority, Velocity-Guided Sampling, and Uncertainty-Based Fusing.
Monocular Depth Priority
Standard MVS methods search for depth candidates across a broad range, which often leads to matching ambiguities. MOVEDepth utilizes a monocular network to predict an initial depth prior, centering the MVS search range around this prediction. This targeted approach narrows the search space significantly, allowing the model to achieve high accuracy with only 16 depth candidates, compared to the 64 or more typically required by traditional MVS methods.
Velocity-Guided Sampling
The effectiveness of multi-view geometry is directly tied to camera motion. In scenarios with low camera velocity (e.g., a vehicle stuck in traffic), the baseline is insufficient for reliable triangulation. MOVEDepth adaptively adjusts the depth search range based on predicted camera velocity. At high velocities, the system trusts geometric constraints more; at low velocities, it constrains the search range to stay close to the reliable monocular estimate, avoiding depth "hallucinations."
Uncertainty-Based Fusing
To handle scenarios where geometry is inherently ambiguous, such as on reflective car surfaces or blank walls, MOVEDepth learns an uncertainty map based on the entropy of the MVS cost volume. This map guides the fusion of monocular and MVS depth maps, allowing the system to lean on monocular predictions when geometric cues are unreliable.
Experiments and Results
The framework was evaluated on the KITTI and DDAD benchmarks. On KITTI, MOVEDepth achieved a 20% improvement in accuracy over Monodepth2 and a 19.8% improvement over PackNet. On the more challenging DDAD dataset, it outperformed the previous state-of-the-art, ManyDepth, by 7.2%.
Beyond accuracy, the model maintains high efficiency, running at 32 FPS on an NVIDIA RTX 3090, making it a viable solution for real-time robotics and autonomous driving applications.
Conclusion
MOVEDepth demonstrates that monocular cues and velocity guidance can significantly enhance multi-frame depth learning. By treating monocular depth as a geometric priority rather than just a consistency target, the model overcomes traditional MVS pitfalls. Future work will focus on improving performance in highly dynamic environments where independent object motion further complicates geometric modeling.
Summary updated