Image-Goal Navigation in Complex Environments via Modular Learning

Qiaoyun Wu, Jun Wang, Jing Liang, Xiaoxi Gong, Dinesh Manocha

TL;DR

Researchers have developed a modular, hierarchical framework for "image-goal navigation"—finding a location based only on a picture—that significantly outperforms end-to-end learning models. By decoupling mapping, long-term planning, and local obstacle avoidance, the system achieves a 17% higher success rate and a 23% reduction in collisions, demonstrating robust "sim-to-real" transfer in crowded, real-world environments.

What Problem Does This Paper Address?

In the world of robotics, Point-Goal Navigation is relatively straightforward: you give a robot coordinates, and it goes there. Image-Goal Navigation is much harder. The robot is shown a picture of a destination and must find it in an unfamiliar environment without a pre-existing map or GPS.

Most current solutions use "end-to-end" deep reinforcement learning (RL), where a single neural network takes in pixels and spits out motor commands. While impressive in simulators, these models often fail in the real world because they overfit to their training data. They struggle with long distances, get confused by "visual noise," and often crash into obstacles because they haven't learned a robust sense of geometry or physics.

Methods

The authors propose a modular architecture that breaks the navigation task into four specialized components:

  1. Automatic Mapping: Instead of relying on pure memory, the robot builds an explicit 2D obstacle map in real-time. In simulation, it uses Neural SLAM; in the real world, it uses Gmapping (a laser-based geometric approach).
  2. Long-term Goal Prediction: This is the "brain" of the operation. An RL-based policy looks at the current map and the goal image to predict a "waypoint" on the map that is likely to lead toward the target.
  3. Reactive Motion Planning: Once a waypoint is set, this module handles the "how." It translates the waypoint into collision-free steering commands. The authors used the Fast Marching Method in simulation and CrowdMove (a more robust algorithm) for real-world crowded scenarios.
  4. Navigation Ending Prediction: A Siamese neural network compares the robot's current view with the goal image. When they are "close enough" (within 1 meter), it issues a stop command.

By separating these tasks, the robot doesn't have to learn "how to see," "how to plan," and "how to not hit a chair" all at once.

Datasets

  • Simulation: The model was trained and tested using the Habitat simulator on the Gibson dataset, which contains high-quality 3D scans of real indoor environments (86 different scenes).
  • Real-World: Evaluation was conducted using a TurtleBot2 mobile robot equipped with an RPLIDAR A1 laser and four monocular cameras, tested across three different physical indoor layouts with varying levels of "clutter" and human activity.

Key Insights and Ideas

  • Decoupling is Key: The most significant "aha moment" is that global planning (where to go) and local control (how to move safely) should be handled by different systems. This allows the robot to use precise geometric mapping for obstacles while using deep learning for the more abstract task of visual recognition.
  • Continuous Reward Design: In RL, "sparse rewards" (only rewarding the robot when it reaches the goal) make training slow and difficult. The authors introduced a reward function based on "explored area increments," encouraging the robot to actively seek out new parts of the map until it finds the goal.
  • Sim-to-Real Bridge: Because the mapping and motion planning modules can be swapped for more "real-world ready" versions (like Gmapping and CrowdMove) without retraining the high-level goal predictor, the system generalizes to physical hardware much better than end-to-end models.

Key Results

The modular approach, dubbed Ours, was compared against several state-of-the-art (SOTA) baselines, including Neural Topological SLAM (NTS).

  • Higher Success: In the Habitat simulator, the proposed method achieved an overall success rate (SR) of 57%, outperforming the SOTA end-to-end model (TD-RL-ITR) which sat at 44%.
  • Efficiency: The Success weighted by Path Length (SPL)—a measure of how direct the robot's path was—showed significant improvements, especially in "Hard" (long-range) tasks.
  • Real-World Superiority: In physical tests, the modular system maintained a success rate of 32% to 58% across different rooms, while end-to-end models plummeted to as low as 6% success in complex scenarios.
  • Safety: The collision rate in the real world was reduced by 23% compared to the best-performing baseline.

Limitations and Future Work

While the system is robust, it isn't perfect:

  • Mapping Dependency: The system's performance is heavily tied to the quality of the SLAM (Simultaneous Localization and Mapping). If the laser or cameras produce a messy map, the goal predictor will fail.
  • "Freezing" Behavior: In extremely noisy or crowded real-world environments, the reactive planner sometimes "freezes" the robot to avoid any risk of collision, leading to a failed mission due to time-out.
  • Semantic Blindness: Currently, the robot treats all obstacles the same. The authors suggest that future versions should incorporate semantic properties—understanding that a "door" is something you can go through, while a "wall" is not—to improve high-level planning.

Summary updated

Abstract

We present a novel approach for image-goal navigation, where an agent navigates with a goal image rather than accurate target information, which is more challenging. Our goal is to decouple the learning of navigation goal planning, collision avoidance, and navigation ending prediction, which enables more concentrated learning of each part. This is realized by four different modules. The first module maintains an obstacle map during robot navigation. The second predicts a long-term goal on the real-time map periodically, which can thus convert an image-goal navigation task to several point-goal navigation tasks. To achieve these point-goal navigation tasks, the third module plans collision-free command sets for navigating to these long-term goals. The final module stops the robot properly near the goal image. The four modules are designed or maintained separately, which helps cut down the search time during navigation and improves the generalization to previously unseen real scenes. We evaluate the method in both a simulator and in the real world with a mobile robot. The results in real complex environments show that our method attains at least a $17\%$ increase in navigation success rate and a $23\%$ decrease in navigation collision rate over some state-of-the-art models.
BibTeX
@article{wu2022image,
  author = {Qiaoyun Wu and Jun Wang and Jing Liang and Xiaoxi Gong and Dinesh Manocha},
  title = {{Image-Goal Navigation in Complex Environments via Modular Learning}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2022},
  doi = {10.1109/lra.2022.3178810},
}

Loading related content...

Figure