Abstract
This work focuses on object goal visual navigation, aiming at finding the location of an object from a given class, where in each step the agent is provided with an egocentric RGB image of the scene. We propose to learn the agent's policy using a reinforcement learning algorithm. Our key contribution is a novel attention probability model for visual navigation tasks. This attention encodes semantic information about observed objects, as well as spatial information about their place. This combination of the "what" and the "where" allows the agent to navigate toward the sought-after object effectively. The attention model is shown to improve the agent's policy and to achieve state-of-the-art results on commonly-used datasets.
Introduction
Visual navigation—the ability of an autonomous agent to find an object (like a "toaster") in a previously unvisited room—is a fundamental challenge in robotics. To succeed, an agent must solve two problems simultaneously: semantic understanding (recognizing objects and their relationships, like knowing toasters are usually near refrigerators) and path planning (moving through 3D space efficiently).
Existing reinforcement learning (RL) methods often struggle to integrate these two aspects. Many current attention models "flatten" or average out spatial information to create a global feature vector, effectively losing the "where" component of the visual scene. This research bridges this gap by creating an attention model that preserves spatial dimensions, allowing the agent to focus on specific regions of its field of view that are most relevant to its current goal.
Methods: Spatial Attention Mechanism
Building upon the Self-Adaptive Visual Navigation (SAVN) framework, this work introduces the Attended Embedding unit. This unit consists of three specialized attention modules that preserve the spatial grid of convolutional features through pointwise multiplication:
- Target Attention: Uses a GloVe word embedding of the goal (e.g., "TV") to highlight regions in the current image likely to contain the target or helpful landmarks.
- Action Attention: Considers the agent's most recent move (e.g., "RotateRight") to focus attention on the new areas of the image revealed by that movement.
- Memory Attention: Leverages an LSTM hidden state to "remember" previously seen areas, helping the agent avoid redundant exploration.
These three attention maps are fused using learned weights ($\beta$) that dynamically adjust based on the agent's current situation. By refusing to average spatial features, the agent maintains a better "egocentric map," leading to more efficient navigation.
Experimental Setup
The model was evaluated using AI2-THOR, a photo-realistic 3D indoor simulator, across 120 scenes in four room categories: Kitchen, Living Room, Bedroom, and Bathroom. The agent was tasked with finding 22 different object classes (e.g., laptop, microwave, alarm clock). A standard 20/5/5 split was used for training, validation, and testing.
Key Insights and Results
The proposed model, using Synchronous Actor-Critic (A2C), outperformed previous state-of-the-art methods across major metrics.
- Success Rate: Achieved 46.20%, a significant improvement over the baseline SAVN's 40.86%.
- SPL (Success weighted by Path Length): Reached 17.88, indicating more direct routes to the goal.
- Long-Range Navigation: For tasks requiring paths longer than 5 steps, success rates rose to 32.63% (vs. 28.70% for the baseline).
The agent learns to use "semantic landmarks." For example, when searching for a toaster, the attention map often highlights the refrigerator first. Furthermore, the importance of different attention types shifts dynamically; the agent may rely on memory for exploration and switch to target attention once a landmark is spotted.
Conclusion and Future Work
This spatial attention mechanism transforms the RL policy from a "black box" into an interpretable system where the agent's focus can be verified visually. While the model achieves state-of-the-art results, it still faces challenges with visual similarity (e.g., mistaking a bathtub for a sink) and occlusions. Future work may involve incorporating depth data (RGBD) to improve distance estimation and obstacle avoidance.
Summary updated