Abstract
This work focuses on object goal visual navigation, aiming at finding the location of an object from a given class, where in each step the agent is provided with an egocentric RGB image of the scene. We propose to learn the agent's policy using a reinforcement learning algorithm. Our key contribution is a novel attention probability model for visual navigation tasks. This attention encodes semantic information about observed objects, as well as spatial information about their place. This combination of the "what" and the "where" allows the agent to navigate toward the sought-after object effectively. The attention model is shown to improve the agent's policy and to achieve state-of-the-art results on commonly-used datasets.
Introduction
Object Goal Visual Navigation requires an AI agent to find a specific object (e.g., "find the toaster") in an unfamiliar environment using only egocentric RGB images. This task is challenging because it requires both semantic understanding (recognizing a toaster or related objects like a counter) and spatial reasoning (planning a path). Traditional reinforcement learning (RL) models often lose spatial resolution when processing visual features, making it difficult for the agent to pinpoint the exact location of relevant cues in its field of view.
Methodology
The proposed framework builds upon the Self-Adaptive Visual Navigation (SAVN) model, introducing a specialized Spatial Attention module. Unlike standard attention mechanisms that average convolutional features into a single vector, this module preserves a spatial grid ($n_v \times n_v$).
The core of the approach is a fused attention map derived from three components:
- Target Attention: Relates the goal (using GloVe embeddings) to image regions.
- Action Attention: Focuses on new areas based on the agent's previous movement.
- Memory Attention: Uses an LSTM hidden state to track explored regions.
These maps are combined into a single Fused Map using learned weights. This map is then applied element-wise to the visual features, ensuring the agent maintains a precise "sense of place" for important objects. The model was trained and evaluated in the AI2-THOR environment, encompassing 120 indoor scenes and 22 object classes.
Results and Discussion
The spatial attention model achieved state-of-the-art performance, reaching a Success Rate of 46.20% and a Success weighted by Path Length (SPL) of 17.88. This significantly outperforms the SAVN baseline (40.86% Success, 16.15 SPL). Notably, the model is more parameter-efficient, using only 7.5M parameters compared to 20.4M in some baseline architectures.
Explainability and Dynamic Weighting
A key advantage of this model is its interpretability. By visualizing the attention maps, researchers can observe how the agent prioritizes different visual cues over time.
As shown in the progression above, the agent's focus shifts dynamically. Early in the search, Target Attention helps identify landmarks. As the agent moves, Action and Memory Attention become more prominent to refine the navigation path and avoid redundant exploration.
Limitations
Despite its success, the agent occasionally suffers from semantic ambiguity (e.g., confusing a bathtub with a large sink) or fails when objects are heavily occluded.
Conclusion
By explicitly modeling the "what" and the "where" through a spatially-preserved attention mechanism, this work advances the capability of autonomous agents in complex 3D environments. Future work will explore the integration of depth data (RGB-D) to further improve distance estimation and obstacle avoidance.
Summary updated