DEUX: Active Exploration for Learning Unsupervised Depth Perception

Marvin Chancán, Alex Wong, Ian AbrahamarXiv 2023 · DOI · arXiv

Introduction

In the field of robotic vision, depth perception is a fundamental capability. However, most existing models are trained on static, "non-interactive" datasets such as KITTI or NYUv2. These datasets consist of pre-recorded video frames from fixed camera paths, leading to systematic biases where models learn features specific to those trajectories rather than general geometric principles.

The DEUX framework enables active exploration to collect task-informed data, improving depth perception training.

To address these limitations, we introduce DEUX (DEpth Uncertainty-guided eXploration), a novel active exploration strategy. DEUX allows robots to autonomously collect their own training data by prioritizing areas that the current model finds most challenging, effectively "visiting the places the model finds most confusing."

The Problem with Passive Data

Conventional depth perception training relies on user-driven exploration for generic downstream tasks. This approach suffers from data inefficiency; in a real-world deployment, a robot should not merely "watch" data but should actively seek out the most informative data points to refine its performance. While standard exploration strategies like "coverage" or "curiosity" exist, they are rarely optimized for the specific task of depth completion—the process of converting a sparse point cloud and an RGB image into a dense depth map.

Traditional approaches rely on passive, video-based datasets that are often biased and task-agnostic.

Methodology: Task-Informed Exploration

DEUX utilizes a task-informed motion planning approach guided by the model's own uncertainty. The framework operates without ground-truth depth, using photometric reprojection error as a proxy for uncertainty. If the predicted depth is accurate, the system can mathematically "warp" one video frame to match the next; a high reconstruction error indicates poor model understanding.

Key Components of DEUX:

  1. Uncertainty Mapping: The reconstruction error serves as a signal to identify "frontiers" or boundaries between explored and unexplored space.
  2. Motion Planning: Using an A* planner, the robot selects target locations that maximize expected photometric error, specifically targeting "hard-to-predict" regions.
  3. Seed Model Initialization: The process begins with a brief period of random exploration to train an initial "seed" model, which then provides the uncertainty estimates necessary to guide the sophisticated DEUX exploration phase.

Experimental Setup and Results

Research was conducted in the Habitat-Sim simulator using the Matterport3D (MP3D) and Habitat-Matterport 3D (HM3D) datasets. DEUX was compared against eight navigation paradigms, including heuristic methods (Random, Frontier) and Reinforcement Learning-based methods (Curiosity, Coverage).

Key Findings:

  • Performance Gains: Training on DEUX-collected data improved depth completion by an average of 18.5% across four state-of-the-art architectures (KBNet, VOICED, FusionNet, and ScaffNet).
  • Superiority over RL: DEUX outperformed the best RL-based method, Smooth Coverage, by approximately 7% in Mean Absolute Error (MAE).
  • Robust Generalization: In zero-shot tests on the HM3D dataset, models trained via DEUX outperformed random exploration by 24% and standard frontier exploration by 17%.
  • Qualitative Quality: Visual analysis confirms that DEUX-trained models produce sharper object boundaries and more accurate global geometry compared to those trained on standard paths.

Conclusion and Future Work

DEUX demonstrates that task-informed data collection is significantly more effective than task-agnostic exploration. By actively seeking out "failure modes," the resulting training set is richer in edge cases, leading to more robust models.

Future work will focus on bridging the sim-to-real gap, as real-world sensors introduce noise (e.g., LiDAR reflections) that may affect the photometric error signal. Additionally, the authors suggest adapting this uncertainty-guided approach to other robotic vision tasks, such as semantic segmentation or object detection, by utilizing task-specific uncertainty metrics.

Summary updated

Abstract

Depth perception models are typically trained on non-interactive datasets with predefined camera trajectories. However, this often introduces systematic biases into the learning process correlated to specific camera paths chosen during data acquisition. In this paper, we investigate the role of how data is collected for learning depth completion, from a robot navigation perspective, by leveraging 3D interactive environments. First, we evaluate four depth completion models trained on data collected using conventional navigation techniques. Our key insight is that existing exploration paradigms do not necessarily provide task-specific data points to achieve competent unsupervised depth completion learning. We then find that data collected with respect to photometric reconstruction has a direct positive influence on model performance. As a result, we develop an active, task-informed, depth uncertainty-based motion planning approach for learning depth completion, which we call DEpth Uncertainty-guided eXploration (DEUX). Training with data collected by our approach improves depth completion by an average greater than 18% across four depth completion models compared to existing exploration methods on the MP3D test set. We show that our approach further improves zero-shot generalization, while offering new insights into integrating robot learning-based depth estimation.
BibTeX
@misc{chancan2023deux,
  author = {Marvin Chancán and Alex Wong and Ian Abraham},
  title = {{DEUX: Active Exploration for Learning Unsupervised Depth Perception}},
  year = {2023},
  eprint = {2310.06164},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.06164},
}
Figure