Abstract
Recently there has been a rising interest in training agents, embodied in virtual environments, to perform language-directed tasks by deep reinforcement learning. In this paper, we propose a simple but effective neural language grounding module for embodied agents that can be trained end to end from scratch taking raw pixels, unstructured linguistic commands, and sparse rewards as the inputs. We model the language grounding process as a language-guided transformation of visual features, where latent sentence embeddings are used as the transformation matrices. In several language-directed navigation tasks that feature challenging partial observability and require simple reasoning, our module significantly outperforms the state of the art. We also release XWorld3D, an easy-to-customize 3D environment that can potentially be modified to evaluate a variety of embodied agents.
Introduction
For an autonomous agent to follow a command like "Navigate to the object in front of the monster," it must perform language grounding: the ability to map abstract words ("monster," "in front of") to raw pixels in its camera feed. Current approaches often struggle with this, typically relying on simple concatenation or gating mechanisms that fail in cluttered or partially hidden environments. The core challenge lies in creating a perception system flexible enough to let the linguistic meaning fundamentally change how visual data is interpreted.
Guided Feature Transformation (GFT)
The proposed Guided Feature Transformation (GFT) module addresses this by using the command to generate a transformation matrix rather than just a scalar gate. The process involves:
- Visual Processing: A standard CNN extracts a "feature cube" from the input image.
- Language Processing: The command is encoded into a latent embedding.
- The Transformation: This embedding is reshaped into a matrix ($T$), which is then multiplied by the visual features.
Mathematically, GFT performs a "Rotate-Scale-Rotate" operation on the visual feature space. Unlike previous methods like FiLM that only scale features, GFT can rotate the axes of the visual representation based on the command, providing significantly higher representational power. The agent is trained end-to-end using Deep Reinforcement Learning (ParallelA2C), learning to associate sparse rewards with the correct grounding of visual and linguistic features.
Experimental Setup
The module was evaluated in two environments:
- XWorld2D: A grid-based environment modified for egocentric vision, introducing partial observability.
- XWorld3D: A new 3D environment featuring perspective distortion and realistic visual challenges.
Tasks included basic navigation (nav), finding objects near others (nav_nr), locating positions between objects (nav_bw), avoiding specific objects (nav_avoid), and relative directional navigation (nav_dir).
Results and Discussion
GFT significantly outperformed state-of-the-art baselines across nearly all tasks. Notably, in the complex 3D nav_dir tasks, GFT achieved a success rate over 15% higher than the next best method.
Key insights include:
- Expressiveness: Increasing the number of transformation steps (e.g., GFT-2) significantly improved performance on tasks requiring complex spatial reasoning.
- Semantic Consistency: Visualization of transformation matrices showed that semantically similar commands produced nearly identical matrices, indicating the module captures underlying meaning rather than just word patterns.
- Generalizability: The architecture proved robust across both 2D and 3D environments without requiring hyperparameter adjustments.
Conclusion
While GFT is more parameter-heavy than simpler gating methods, its superior performance in complex spatial reasoning tasks demonstrates the value of high-dimensional feature transformations. Future work will explore recurrent transformations to reduce parameter counts and the application of GFT to Visual Question Answering (VQA).
Summary updated