Introduction
In the field of robotic vision, depth perception is a fundamental capability. However, most existing models are trained on static, "non-interactive" datasets such as KITTI or NYUv2. These datasets consist of pre-recorded video frames from fixed camera paths, leading to systematic biases where models learn features specific to those trajectories rather than general geometric principles.
To address these limitations, we introduce DEUX (DEpth Uncertainty-guided eXploration), a novel active exploration strategy. DEUX allows robots to autonomously collect their own training data by prioritizing areas that the current model finds most challenging, effectively "visiting the places the model finds most confusing."
The Problem with Passive Data
Conventional depth perception training relies on user-driven exploration for generic downstream tasks. This approach suffers from data inefficiency; in a real-world deployment, a robot should not merely "watch" data but should actively seek out the most informative data points to refine its performance. While standard exploration strategies like "coverage" or "curiosity" exist, they are rarely optimized for the specific task of depth completion—the process of converting a sparse point cloud and an RGB image into a dense depth map.
Methodology: Task-Informed Exploration
DEUX utilizes a task-informed motion planning approach guided by the model's own uncertainty. The framework operates without ground-truth depth, using photometric reprojection error as a proxy for uncertainty. If the predicted depth is accurate, the system can mathematically "warp" one video frame to match the next; a high reconstruction error indicates poor model understanding.
Key Components of DEUX:
- Uncertainty Mapping: The reconstruction error serves as a signal to identify "frontiers" or boundaries between explored and unexplored space.
- Motion Planning: Using an A* planner, the robot selects target locations that maximize expected photometric error, specifically targeting "hard-to-predict" regions.
- Seed Model Initialization: The process begins with a brief period of random exploration to train an initial "seed" model, which then provides the uncertainty estimates necessary to guide the sophisticated DEUX exploration phase.
Experimental Setup and Results
Research was conducted in the Habitat-Sim simulator using the Matterport3D (MP3D) and Habitat-Matterport 3D (HM3D) datasets. DEUX was compared against eight navigation paradigms, including heuristic methods (Random, Frontier) and Reinforcement Learning-based methods (Curiosity, Coverage).
Key Findings:
- Performance Gains: Training on DEUX-collected data improved depth completion by an average of 18.5% across four state-of-the-art architectures (KBNet, VOICED, FusionNet, and ScaffNet).
- Superiority over RL: DEUX outperformed the best RL-based method, Smooth Coverage, by approximately 7% in Mean Absolute Error (MAE).
- Robust Generalization: In zero-shot tests on the HM3D dataset, models trained via DEUX outperformed random exploration by 24% and standard frontier exploration by 17%.
- Qualitative Quality: Visual analysis confirms that DEUX-trained models produce sharper object boundaries and more accurate global geometry compared to those trained on standard paths.
Conclusion and Future Work
DEUX demonstrates that task-informed data collection is significantly more effective than task-agnostic exploration. By actively seeking out "failure modes," the resulting training set is richer in edge cases, leading to more robust models.
Future work will focus on bridging the sim-to-real gap, as real-world sensors introduce noise (e.g., LiDAR reflections) that may affect the photometric error signal. Additionally, the authors suggest adapting this uncertainty-guided approach to other robotic vision tasks, such as semantic segmentation or object detection, by utilizing task-specific uncertainty metrics.
Summary updated