Active Neural Localization

Devendra Singh Chaplot, Emilio Parisotto, Ruslan Salakhutdinov

TL;DR

The "Active Neural Localizer" (ANL) is a fully differentiable neural network that teaches autonomous agents not just to estimate where they are, but to move strategically to find out faster. By combining the mathematical rigor of Bayesian filtering with the flexibility of reinforcement learning, the model achieves higher accuracy and is orders of magnitude faster than traditional hand-tuned methods.

What Problem Does This Paper Address?

Localization—knowing where you are on a map—is a cornerstone of robotics. Most traditional approaches are passive: the agent moves (perhaps randomly or for another task) and updates its position estimate based on what it happens to see. This is often inefficient; if you’re lost in a building, you don't just walk in circles—you look for a room number or a unique landmark.

Existing active localization methods attempt to choose actions that reduce uncertainty, but they usually rely on hand-tuned features and are computationally "heavy," often requiring complex simulations of future states to decide the next move. This paper tackles the challenge of creating an end-to-end system that learns a movement policy directly from raw pixels to localize accurately and efficiently.

Methods

The authors propose a hybrid architecture that mirrors a classical Bayesian Filter but implements every component as a neural network layer:

  1. Perceptual Model: A Convolutional Neural Network (CNN) that takes the agent's current view (RGB pixels) and produces a "Likelihood Map"—a grid showing how likely the current observation is at every possible location on the map.
  2. Belief Map: A structured tensor representing the agent's probability distribution over all possible positions and orientations.
  3. Belief Propagation: Instead of a black-box RNN, the model uses a formal transition function ($f_T$) to shift the belief map based on actions (e.g., if the agent moves forward, the probability mass moves forward on the grid).
  4. Policy Model: A reinforcement learning component (trained via A3C) that looks at the current belief and the map to decide the best action to reduce uncertainty.
Figure 1: The architecture of the proposed model. The perceptual model computes the likelihood of the current observation in all possible locations. The belief of agent’s location is propagated through time by taking element-wise dot-product with the likelihood. The policy model learns a policy to localize accurately while minimizing the number of steps required for localization. See text for more details.

The entire pipeline is differentiable, meaning the "vision" part and the "moving" part are trained together to optimize the final goal: accurate localization in the fewest steps possible.

Datasets

The researchers evaluated ANL across three distinct simulation environments:

  • Maze 2D: Procedurally generated grid worlds used to test the policy logic in ideal conditions.
  • Maze3D (Doom): 3D environments built in the Doom engine. These introduce raw-pixel observations and varying textures/landmarks.
  • Unreal3D: A photo-realistic office environment built in the Unreal Engine to test generalization to high-fidelity visuals.

The authors contributed these scenarios as new benchmarks for the active localization community.

Key Insights and Ideas

  • Inductive Bias Matters: By forcing the neural network to follow the structure of a Bayesian filter (using Likelihood and Belief maps), the model learns much faster and generalizes better than a generic "black-box" LSTM.
  • Strategic Curiosity: The agent learns to seek out "unambiguous" locations. In a repetitive hallway, the agent might learn to turn into a specific room or find a unique corner to "break" its uncertainty.
  • Zero-Shot Domain Adaptation: One of the most striking findings was that a model trained on random, ugly textures in Doom could generalize to a photo-realistic Unreal Engine office without any extra training. This suggests the model learns the geometry of localization rather than just memorizing textures.
Figure 2: The map design, agent’s observation and the corresponding likelihood maps in different domains. In 2D domains, agent’s observation is the pixels in front of the agent until the first obstacle. In the 3D domain, the agent’s observation is the image showing the first-person view of the world as seen by the agent.

Key Results

The Active Neural Localizer consistently outperformed both passive baselines and traditional active methods:

  • Accuracy: In 2D mazes, ANL achieved ~93% accuracy, nearly double the ~48% of passive Markov Localization.
  • Efficiency: ANL was significantly more efficient than "Active Markov Localization" (AML). While a "Slow" version of AML could match ANL's accuracy, it took hours to evaluate 1,000 episodes. ANL performed the same task in seconds—a speedup of several orders of magnitude.
  • Generalization: In 3D environments, ANL maintained high accuracy (up to 92% in domain adaptation tasks) even when faced with unseen map designs and textures.
Table 1: Results on the 2D Environments. ‘Time’ refers to the number of seconds required to evaluate 1000 episodes with the corresponding method and ‘Acc’ stands for accuracy over 1000 episodes.

Limitations and Future Work

While powerful, the model has its limits:

  • Dynamic Lighting: The current RGB-based perceptual model struggles when lighting conditions change drastically (e.g., turning the lights off in the Unreal office). The authors suggest that incorporating depth sensors, which are invariant to light, could solve this.
  • Mapping: The model assumes the agent already has a map ($M$). A natural next step is combining this with SLAM (Simultaneous Localization and Mapping) where the agent must build the map and localize within it simultaneously.
  • Real-World Noise: The experiments were conducted in simulation. Transitioning to physical robots will require handling sensor noise and motion inaccuracies not fully captured in these environments.

Summary updated

Abstract

Localization is the problem of estimating the location of an autonomous agent from an observation and a map of the environment. Traditional methods of localization, which filter the belief based on the observations, are sub-optimal in the number of steps required, as they do not decide the actions taken by the agent. We propose "Active Neural Localizer", a fully differentiable neural network that learns to localize accurately and efficiently. The proposed model incorporates ideas of traditional filtering-based localization methods, by using a structured belief of the state with multiplicative interactions to propagate belief, and combines it with a policy model to localize accurately while minimizing the number of steps required for localization. Active Neural Localizer is trained end-to-end with reinforcement learning. We use a variety of simulation environments for our experiments which include random 2D mazes, random mazes in the Doom game engine and a photo-realistic environment in the Unreal game engine. The results on the 2D environments show the effectiveness of the learned policy in an idealistic setting while results on the 3D environments demonstrate the model's capability of learning the policy and perceptual model jointly from raw-pixel based RGB observations. We also show that a model trained on random textures in the Doom environment generalizes well to a photo-realistic office space environment in the Unreal engine.
BibTeX
@inproceedings{chaplot2018active,
  author = {Devendra Singh Chaplot and Emilio Parisotto and Ruslan Salakhutdinov},
  title = {{Active Neural Localization}},
  booktitle = {International Conference on Learning Representations},
  year = {2018},
  url = {https://openreview.net/forum?id=ry6-G_66b},
}

Loading related content...

Figure