TL;DR
This paper tackles the Object Goal Navigation (ObjectNav) problem by teaching an agent to predict the distance to unseen target objects using semantic cues (e.g., knowing that chairs are usually near tables). Instead of relying on sample-inefficient reinforcement learning or overly complex scene completion models, the authors use a supervised learning approach to estimate target distances at the exploration frontier, achieving higher success rates with significantly less training data.
What Problem Does This Paper Address?
Imagine being dropped into a brand-new house and asked to "find a chair." You don't have a map, but you have common sense: you'll probably look for a dining table or a desk, knowing chairs are usually nearby. This is the essence of Object Goal Navigation (ObjectNav).
Historically, researchers have tackled this using Reinforcement Learning (RL), which requires the agent to stumble around millions of times to learn a good search policy, making it highly sample-inefficient. Other recent approaches try to use "Semantic Scene Completion"—guessing the exact layout of the unseen rooms. However, predicting the exact location and pose of an unseen object is incredibly difficult. This paper addresses the gap by asking a simpler, more learnable question: instead of guessing exactly where the object is, can we just guess how far away it is based on the objects we can currently see?
Methods
The authors break the navigation system into three distinct modules:
- Semantic Mapping (Standard): As the agent moves, it uses an RGB-D camera and a pre-trained Mask-RCNN to detect objects and project them into a 2D bird’s-eye view semantic map.
- Target Distance Prediction (Novel): This is the core contribution. A fully convolutional neural network looks at the current semantic map and predicts the shortest path distance from the exploration frontier (the boundary between the explored and unexplored areas) to the target object. Crucially, the authors formulate this as a classification problem rather than a regression problem. The network predicts which of 5 discrete distance bins (e.g., <1m, 1-2m, 2-4m, etc.) the target falls into.
- Local Policy (Heuristic): Based on the predicted distance map, the agent selects a mid-term goal. The authors test a few simple strategies, such as a "Closest-First" strategy (head to the frontier cell with the lowest predicted distance) and a "Door-Exploring-First" strategy (prioritize doors if they lead to areas with low predicted distances). A standard path planner (Fast Marching Method) then routes the agent to that goal.
Datasets
- Simulation: The model is trained and evaluated in the Habitat simulator using the Matterport3D (MP3D) dataset. The training set includes 54 scenes, and the test set includes 10 scenes, searching for 6 object categories (chair, couch, plant, bed, toilet, TV).
- Real-World: The authors also deployed their system on a physical mobile robot equipped with a Kinect V2 camera and a 2D LiDAR, testing it in a real 120m² house.
Key Insights and Ideas
- Distance is easier to learn than location: Guessing the exact coordinates of a chair behind a wall is nearly impossible. But learning the spatial relationship that "the distance between a bed and a chair is usually small" is a highly reliable and learnable prior.
- Classification beats regression for distance: The authors found that predicting a precise continuous distance value is too difficult and noisy. Discretizing the distances into bins (e.g., <1m, 1-2m) yields a much more robust signal for the navigation policy.
- Supervised learning drastically improves sample efficiency: Because the distance prediction model can be trained offline using collected semantic map samples (with known ground-truth distances), it completely bypasses the need for the agent to interactively learn via sparse RL rewards.
Key Results
- Higher Efficiency: The proposed method outperforms the state-of-the-art RL baseline (SemExp) by +2.6% in Success Rate and +3.5% in SPL (Success weighted by inverse Path Length, a measure of path efficiency).
- Massive Data Reduction: The model achieved these state-of-the-art results using only 1.2 million training samples, compared to the 10 million samples required by the RL-based SemExp baseline (an 88% reduction in training data).
- Real-World Transfer: The system generalized beautifully to the real world. In 150 physical robot episodes, the agent achieved an 84.7% success rate, proving that the semantic distance priors learned in simulation hold up in actual physical environments.
Limitations and Future Work
- Bottlenecked by Perception: The system's performance is heavily tied to the accuracy of the underlying semantic segmentation model. For example, the agent almost entirely failed to find "TVs" in simulation because the 3D reconstruction quality of TVs in the MP3D dataset is notoriously poor. Real-world failures were also primarily caused by the camera missing objects or projecting them incorrectly due to sensor noise.
- Future Work: The authors suggest that distance predictions could be made even more accurate by introducing auxiliary tasks, such as predicting the type of room the agent is looking at. They also note that combining their supervised distance map with a more powerful RL policy (rather than simple heuristics) could yield even better performance.
Summary updated