From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN)

Xin Ye, Yezhou YangarXiv 2020 · arXiv

In the evolving landscape of robotics, the field is undergoing a significant shift from classical, modular pipelines to end-to-end, data-driven "Embodied AI." This transition is the focus of the survey paper, "From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN)," by Xin Ye and Yezhou Yang. The authors categorize current VIN research, identify the critical challenge of generalization, and discuss the ongoing friction between simulated training and real-world deployment.

The Shift in Navigation Paradigms

Traditional navigation systems have long relied on a "divide and conquer" strategy, utilizing separate modules for mapping, localization, path planning, and motor control. While robust in controlled settings, these systems are inherently brittle. A minor error in the mapping stage can propagate through the pipeline, leading to total system failure. Furthermore, these classical methods require extensive manual engineering for every new environment.

Overview of the Visual Indoor Navigation (VIN) framework, where an agent maps visual observations to actions based on a user-specified goal.

Learning-based VIN addresses these limitations by training an agent to map first-person visual observations (such as RGB images) directly to actions. The complexity of this task is immense: the agent must perceive its surroundings, reason about a goal that may not yet be visible, and execute a sequence of actions that generalizes to environments it has never encountered before.

Methodological Approaches: SL vs. RL

The survey organizes learning-based VIN into two primary methodological camps:

  1. Supervised Learning (SL): In this approach, the agent is trained to imitate "expert" trajectories, typically the shortest path to a goal. While effective for learning feature representations, SL can be fragile; small prediction errors accumulate over time, eventually leading the agent off-course without a mechanism for recovery.
  2. Reinforcement Learning (RL): Formulated as a Partially Observable Markov Decision Process (POMDP), RL allows an agent to learn through trial and error to maximize a reward. To combat RL's inherent sample inefficiency, researchers often employ Reward Shaping, providing the agent with incremental rewards for reducing the distance to a goal or successfully identifying a target object.

The "Holy Grail" of Generalization

A central theme of the survey is that a model is only as good as its ability to perform in unseen environments. Generalization remains the "Holy Grail" of the field. The authors note that using semantic masks and depth maps instead of raw RGB images significantly improves an agent's ability to generalize to new layouts.

The training and testing pipeline for VIN, highlighting the role of representation learning and the challenge of generalization to unseen environments.

Recent state-of-the-art methods are increasingly incorporating "knowledge-driven" navigation. By using Graph Convolutional Networks (GCNs) to model common-sense relationships (e.g., a bed is likely located in a bedroom), agents can navigate more efficiently by leveraging semantic expectations of a home's layout.

Goal Modalities and Simulation

The way a goal is described fundamentally changes the navigation task. The survey identifies three main modalities:

  • Label-specified goals: (e.g., "Go to the kitchen") requiring semantic understanding.
  • Image-specified goals: requiring visual matching between the current view and a target image.
  • Language-specified goals: (e.g., "Turn left at the sofa") requiring complex cross-modal grounding.

To train these models, researchers rely on high-fidelity 3D simulators like AI2-THOR, Habitat, and House3D. While these platforms have enabled rapid progress, a significant "Sim-to-Real" gap remains. Simulators often lack the physical realism of the real world, such as motor noise or the fact that a physical robot cannot "slide" along a wall.

Future Directions

The authors conclude by identifying several lingering issues that must be addressed for VIN to reach maturity. These include moving beyond static environments to handle dynamic obstacles (people and pets), improving evaluation fairness between SL and RL methods, and developing agents capable of continuous learning and adaptation once deployed in a new real-world home.

Summary updated

Abstract

Visual Indoor Navigation (VIN) task has drawn increasing attention from the data-driven machine learning communities especially with the recently reported success from learning-based methods. Due to the innate complexity of this task, researchers have tried approaching the problem from a variety of different angles, the full scope of which has not yet been captured within an overarching report. This survey first summarizes the representative work of learning-based approaches for the VIN task and then identifies and discusses lingering issues impeding the VIN performance, as well as motivates future research in these key areas worth exploring for the community.
BibTeX
@misc{ye2020from,
  author = {Xin Ye and Yezhou Yang},
  title = {{From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN)}},
  year = {2020},
  eprint = {2002.11310},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2002.11310},
}
Figure