Navigating to Objects in Unseen Environments by Distance Prediction

Minzhao Zhu, Binglei Zhao, Tao Kong

Abstract

Object Goal Navigation (ObjectNav) involves guiding an agent to a specific object category in an unfamiliar environment without a pre-built map. This paper introduces a method that predicts the distance to the target object using visible semantic cues. By estimating path lengths from the exploration frontier to the target based on a bird's-eye view semantic map, the agent can select optimal mid-term goals. Our approach outperforms state-of-the-art baselines in success rate and efficiency while requiring significantly less training data. Real-world experiments further demonstrate the robustness and generalizability of the learned semantic priors.

Introduction

Imagine being placed in a new home and asked to "find a chair." Humans rely on common sense: chairs are typically found near tables or desks. In robotics, this is the Object Goal Navigation (ObjectNav) problem. Traditional solutions often rely on Reinforcement Learning (RL), which requires millions of interactions, or complex scene completion models that attempt to reconstruct the entire unseen environment.

This paper addresses these challenges by asking a simpler, more learnable question: instead of guessing exactly where an object is, can we estimate how far away it is based on visible objects? By learning spatial relationships (e.g., "chairs are usually near tables"), the agent can navigate more efficiently in unseen spaces.

Methodology

The navigation system is divided into three modular components: semantic mapping, distance prediction, and a local policy.

The system architecture consists of three main modules: Semantic Mapping, Target Distance Prediction, and Local Policy.

Semantic Mapping

As the agent explores, it uses an RGB-D camera and a pre-trained Mask-RCNN to detect objects. These detections are projected into a 2D bird’s-eye view (BEV) semantic map, which tracks both explored areas and the locations of identified objects.

Target Distance Prediction

The core contribution is a fully convolutional neural network that predicts the shortest path distance from the exploration frontier to the target object.

The model converts a semantic map into a distance map, categorizing distances into discrete bins for robust prediction.

Crucially, the authors frame this as a classification problem. The network predicts which of five discrete distance bins (e.g., <1m, 1-2m, 2-4m, 4-8m, or >8m) the target falls into. This discretization provides a more robust signal than continuous regression, especially in uncertain, unexplored regions.

Local Policy and Path Planning

Based on the predicted distance map, the agent selects a mid-term goal. The authors tested strategies such as "Closest-First" (heading to the frontier cell with the lowest predicted distance) and "Door-Exploring-First" (prioritizing doors leading to promising areas). A standard path planner then routes the agent to the selected goal.

Results and Discussion

The model was trained and evaluated in the Habitat simulator using the Matterport3D dataset and validated through real-world deployment on a mobile robot.

Performance and Efficiency

The proposed method outperformed the state-of-the-art RL baseline (SemExp) by +2.6% in Success Rate and +3.5% in SPL (Success weighted by Path Length). Most notably, it achieved these results with only 1.2 million training samples—an 88% reduction in training data compared to the 10 million samples required by SemExp.

Real-World Generalization

In 150 physical robot trials, the agent achieved an 84.7% success rate. This demonstrates that the semantic distance priors learned in simulation (e.g., the proximity of beds to nightstands or chairs to tables) are highly effective in real-world settings.

Key Insights

  • Distance is easier to learn than location: Guessing exact coordinates behind a wall is difficult, but learning that "the distance between a bed and a chair is usually small" is a reliable prior.
  • Classification beats regression: Discretizing distances into bins yields a much more robust signal for the navigation policy.
  • Supervised learning improves efficiency: Offline training on collected semantic map samples bypasses the need for interactive learning via sparse RL rewards.

Conclusion

The proposed distance-prediction approach offers a sample-efficient and robust solution for ObjectNav. While currently limited by the accuracy of underlying perception models (e.g., missing small objects or sensor noise), the system shows great promise. Future work may involve incorporating room-type classification as an auxiliary task or combining these supervised distance maps with more advanced RL policies.

Summary updated

Abstract

Object Goal Navigation (ObjectNav) task is to navigate an agent to an object category in unseen environments without a pre-built map. In this paper, we solve this task by predicting the distance to the target using semantically-related objects as cues. Based on the estimated distance to the target object, our method directly choose optimal mid-term goals that are more likely to have a shorter path to the target. Specifically, based on the learned knowledge, our model takes a bird's-eye view semantic map as input, and estimates the path length from the frontier map cells to the target object. With the estimated distance map, the agent could simultaneously explore the environment and navigate to the target objects based on a simple human-designed strategy. Empirical results in visually realistic simulation environments show that the proposed method outperforms a wide range of baselines on success rate and efficiency. Real-robot experiment also demonstrates that our method generalizes well to the real world. Video at https://www.youtube.com/watch?v=R79pWVGFKS4
BibTeX
@inproceedings{zhu2022navigating,
  author = {Minzhao Zhu and Binglei Zhao and Tao Kong},
  title = {{Navigating to Objects in Unseen Environments by Distance Prediction}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=lHTO6hyz3v},
}
Figure