Emergent Systematic Generalization in a Situated Agent

Felix Hill, Andrew K. Lampinen, Rosalia Schneider, Stephen Clark, Matthew M. Botvinick, James L. McClelland, Adam Santoro

Introduction

Deep neural networks are frequently criticized for their perceived inability to generalize systematically to novel combinations of concepts. This paper challenges that notion, demonstrating that standard neural network agents can achieve robust systematic generalization without specialized architectures. The key, the authors argue, lies in training these agents within rich, interactive 3D environments. This embodied, multi-modal experience—reminiscent of a human child's development—naturally fosters the emergence of combinatorial reasoning.

The Problem: Systematicity in Neural Networks

In human cognition, "systematicity" refers to the ability to understand that if one knows the meaning of "John loves Mary," they inherently understand "Mary loves John." Whether artificial neural networks can exhibit this kind of combinatorial generalization remains a central debate in AI.

Many researchers have argued that standard architectures fail at systematicity, leading to proposals for explicit symbolic modules or heavy inductive biases. This study investigates an alternative hypothesis: perhaps the failure is not in the architecture, but in the training environment. By situating a standard Reinforcement Learning (RL) agent in a rich, first-person 3D world with language instructions, the authors explore whether systematicity can emerge naturally, contrasting this with the abstract, static datasets typically used in such evaluations.

Methods: A Standard Agent in a 3D World

To isolate the effect of the environment, the authors intentionally utilize a minimal, standard agent architecture.

The agent architecture uses standard CNN and LSTM modules to process visual and linguistic inputs.
  • Vision: A standard Convolutional Neural Network (CNN) processes raw pixel input from the agent's first-person perspective.
  • Language: A word-level LSTM processes textual instructions.
  • Action/Memory: Visual and language representations are concatenated and fed into a core LSTM. This core module outputs value predictions and actions (e.g., move, look, grab, spin) via the IMPALA actor-critic algorithm.

The agent is trained on specific combinations of verbs and nouns (e.g., "find a pencil," "lift a spaceship") and evaluated zero-shot on withheld combinations (e.g., "lift a pencil").

Experimental Setup and Datasets

Unlike traditional vision tasks using static datasets like ImageNet, this research focuses on situated agents in dynamic environments:

  • 3D Environments: Simulated rooms built in Unity and DeepMind Lab, featuring physics and 3D renderings of everyday objects.
  • 2D Environments: A custom 9x9 grid-world designed to mirror the logical structure of the 3D tasks, serving as a baseline.
  • Tasks: Procedurally generated instructions testing concepts such as "Lifting," "Putting," Color-Shape binding, and logical "Negation."

Key Insights: Why 3D Environments Work

The study identifies several factors that drive the emergence of systematicity:

  1. Diversity Breeds Systematicity: For abstract operators like negation ("find a not box"), the variety of training data is crucial. Increasing the number of training objects significantly improves the agent's ability to generalize to novel negated instructions.
  2. Perspective Matters: An egocentric (first-person) perspective forces visual invariances. Because the agent is always at the center of its own visual field, it can more easily factorize its experience into reusable components compared to a top-down "God's eye" view.
  3. Embodiment as Implicit Data Augmentation: Physical navigation provides a rich, temporal stream of multi-modal data. An agent sees an object from hundreds of angles and distances, building more robust and composable representations than those derived from static images.

Results: Achieving Zero-Shot Generalization

The results demonstrate that situated agents can achieve remarkable levels of systematicity, as shown by the high overlap between training and testing performance across various tasks.

Accuracy curves for training and testing on various tasks demonstrate strong zero-shot generalization.
  • Zero-shot Verb-Noun Binding: Agents achieved near-perfect accuracy when performing actions on novel objects they had only encountered in different contexts.
  • Complex Composition: In a "putting" task requiring navigation and object manipulation, agents achieved over 90% zero-shot accuracy on withheld objects.
  • 3D vs. 2D: The 3D agent significantly outperformed the 2D grid-world agent (63% vs. 40% accuracy) on the putting task, despite identical underlying logic.
  • Negation and Interaction: When trained with 100 objects, the agent reached 78% accuracy on novel negated instructions. Furthermore, the interactive agent hit 100% test accuracy on a color-shape task, whereas a static vision-language classifier reached only 80%.

Conclusion and Future Directions

This research provides strong empirical evidence that systematic generalization can emerge in standard neural networks when they are situated in rich, interactive environments. While the simulated 3D worlds used are simpler than the real world, the findings suggest that embodiment and environmental diversity are powerful drivers of combinatorial reasoning. Future work will explore whether even more realistic environments or real-world robotics can push neural networks toward the full, rapid combinatorial flexibility seen in humans.

Summary updated

BibTeX
@inproceedings{hill2020emergent,
  author = {Felix Hill and Andrew K. Lampinen and Rosalia Schneider and Stephen Clark and Matthew M. Botvinick and James L. McClelland and Adam Santoro},
  title = {{Emergent Systematic Generalization in a Situated Agent}},
  booktitle = {ICLR},
  year = {2020},
  url = {https://openreview.net/forum?id=_Z4fGrMm4m5},
}
Figure