Abstract
Visual navigation tasks require an embodied agent to find a given target based on visual observations. The effectiveness of the transformation from observation to visual representation determines the navigation policy's success. We propose a Spatial Sequential Transformer Network (SSTNet) for learning informative visual representations in deep reinforcement learning. SSTNet consists of a Spatial Attention probability Fused model (SAF) and a Sequential Transformer Network (STNet). SAF enforces cross-modal states into visual clues, encoding semantic and spatial information to exploit image inter-relations. STNet generates future observations and makes action inferences by decoding image intra-relations, helping the agent understand the causality between actions and environmental changes. Evaluated on the AI2THOR framework, our method achieves at least a 10% improvement in average success rate over state-of-the-art models.
Introduction
Visual navigation—answering the question "Where is the pillow?"—is a cornerstone of embodied AI. Traditional Reinforcement Learning (RL) approaches often struggle with spatial reasoning, long-term memory, and generalization to unseen environments. TransNav addresses these challenges by treating visual navigation as a sequence modeling task rather than a simple state-to-action mapping.
The core innovation, the Spatial Sequential Transformer Network (SSTNet), moves away from standard RL state representations toward a transformer-based approach that fuses object-level spatial attention with causal sequence modeling and common-sense scene graphs.
Methodology: The SSTNet Architecture
The SSTNet framework is built upon three primary modules designed to capture both local spatial details and global temporal context.
Spatial Attention probability Fused model (SAF)
Rather than processing raw pixels, the SAF module utilizes a Detection Transformer (DETR) to identify objects within the agent's view. It fuses semantic information (via GloVe word embeddings) with spatial data (bounding box coordinates) to create a "spatial-aware descriptor." This allows the agent to understand not just what objects are present, but their relative positions in the 3D environment.
Sequential Transformer Network (STNet)
The STNet acts as a causal, auto-regressive transformer. By processing sequences of past observations, actions, and rewards, it "imagines" future states. This predictive capability enables the agent to learn the causal relationships between its movements and the resulting changes in its visual field, effectively bypassing the long-term credit assignment issues prevalent in traditional RL.
Graph Association and Meta-Learning
To incorporate "common sense," the model builds a Scene Graph using relationships (e.g., "on," "left of") derived from the Visual Genome dataset. A Markov Logical Network (MLN) reasons over this graph to suggest optimal actions based on object proximity. Furthermore, a meta-learning strategy (similar to MAML) is employed to allow the agent to adapt its policy rapidly when placed in a novel room layout.
Experimental Results
The model was rigorously tested within the AI2THOR simulation environment, using Visual Genome for pre-training and GloVe for semantic embeddings.
Performance Metrics
- Success Rate (SR): SSTNet combined with meta-learning achieved an 80.20% success rate, outperforming baselines like SAVN and VTNet by significant margins.
- Efficiency (SPL): The Success weighted by Path Length (SPL) improved by 15% in unseen environments, indicating more direct and efficient navigation paths.
- Ablation Study: Removing the STNet module caused the success rate to plummet from 76.2% to 54.5%, confirming that causal reasoning and memory are critical to the framework's success.
Key Insights
- Navigation as Sequence Modeling: Treating navigation like a language task (predicting the "next state" like a "next word") provides a more stable training signal than traditional bootstrapping RL.
- Explainable Attention: Spatial attention maps reveal that the agent focuses on logically related objects; for instance, searching for a "pen" triggers attention toward desks and lamps.
- Spatial Context: The inclusion of 3D localization and target orientation proved more effective than simple 2D object co-occurrence, as knowing an object's relative direction is a stronger navigational cue.
Conclusion and Future Work
TransNav demonstrates that transformer-based sequence modeling, combined with spatial awareness and common-sense reasoning, significantly advances the state of visual navigation. While the current model relies on RGB data, future iterations could incorporate RGBD (depth) sensors for better obstacle avoidance. Additionally, optimizing the computational overhead of running multiple deep networks (DETR, Transformers, MLN) remains a priority for real-time robotic applications.
Reference: Zhou, K., Zhang, H., & Li, F. (2022). TransNav: spatial sequential transformer network for visual navigation. Journal of Computational Design and Engineering, 9(5), 1866–1878.
Summary updated