Learning with Latent Language

Jacob Andreas, Dan Klein, Sergey Levine

Abstract

The named concepts and compositional operators present in natural language provide a rich source of information about the kinds of abstractions humans use to navigate the world. Can this linguistic background knowledge improve the generality and efficiency of learned classifiers and control policies? This paper aims to show that using the space of natural language strings as a parameter space is an effective way to capture natural task structure. In a pretraining phase, we learn a language interpretation model that transforms inputs (e.g. images) into outputs (e.g. labels) given natural language descriptions. To learn a new concept (e.g. a classifier), we search directly in the space of descriptions to minimize the interpreter's loss on training examples. Crucially, our models do not require language data to learn these concepts: language is used only in pretraining to impose structure on subsequent learning. Results on image classification, text editing, and reinforcement learning show that, in all settings, models with a linguistic parameterization outperform those without.

Introduction

Machine learning models typically struggle with few-shot learning—mastering a new concept from only a handful of examples. Standard approaches like meta-learning attempt to solve this by finding a "good initialization" in a high-dimensional, continuous weight space. However, these continuous representations often lack the structural priors that make human learning so efficient. Humans utilize compositional abstractions (e.g., "to the left of" or "red square") to navigate the world.

This paper introduces Learning with Latent Language ($L^3$), a framework that uses natural language strings as a parameter space for few-shot learning. By forcing models to search for a linguistic description of a task rather than tuning abstract numerical weights, the authors achieve better generalization, faster learning in reinforcement learning (RL), and provide a built-in "window" into the model's reasoning.

Methodology

The $L^3$ approach consists of three distinct phases:

Overview of the $L^3$ framework: (a) language learning, (b) concept learning, and (c) evaluation.
  1. Pretraining (Language Learning): The model learns a "language interpreter" ($f$). Given an input (like an image) and a natural language description (like "a red cross below a square"), the interpreter predicts the output (e.g., a binary label). This phase establishes the semantics of language—how words relate to visual features or actions.
  2. Concept Learning: When faced with a new task with only a few examples, the model does not update its neural weights. Instead, it searches for a natural language string ($w$) that best describes the pattern seen in the examples. To make this search efficient, a proposal model ($q$)—essentially a captioning engine—suggests candidate descriptions.
  3. Evaluation: The best-fitting description found in phase 2 is then used by the pretrained interpreter to solve new instances of that task.
The concept learning phase involves searching for a linguistic description by scoring candidates.

This "linguistic bottleneck" forces the model to represent the task using discrete, compositional concepts rather than arbitrary floating-point numbers, acting as a powerful regularizer against overfitting.

Experiments and Results

The authors evaluated $L^3$ across three diverse domains:

  • ShapeWorld: A synthetic image classification dataset. $L^3$ achieved 71% accuracy on validation tasks, compared to 62-66% for standard multitask and meta-learning baselines.
  • String Editing: A dataset involving "programming by demonstration." The model achieved 76% accuracy on test tasks, significantly higher than the 62% achieved by the best meta-learning baseline.
  • Treasure Hunting: A 2D RL environment. $L^3$ agents discovered high-reward policies significantly faster than "from scratch" or multitask agents. Once a correct description was sampled, the reward curve spiked almost instantly.

Discussion and Conclusion

The $L^3$ framework demonstrates that language can serve as an effective scaffold for learning. Key insights include:

  • Compositional Generalization: Because language is inherently compositional, $L^3$ is better at "zero-shot" generalization to novel combinations of concepts.
  • Interpretability: Because the "parameters" are literal English sentences, the model's "thought process" is transparent. Even errors often reveal that the model captured the correct high-level idea.
  • Structured Exploration: In RL, sampling meaningful behaviors via descriptions leads to much faster discovery of optimal policies compared to random exploration.

While discrete search remains computationally expensive and the system is capped by the interpreter's pretraining, $L^3$ moves us closer to AI that operates using the same abstract building blocks that humans use to understand the world.

Summary updated

Abstract

The named concepts and compositional operators present in natural language provide a rich source of information about the kinds of abstractions humans use to navigate the world. Can this linguistic background knowledge improve the generality and efficiency of learned classifiers and control policies? This paper aims to show that using the space of natural language strings as a parameter space is an effective way to capture natural task structure. In a pretraining phase, we learn a language interpretation model that transforms inputs (e.g. images) into outputs (e.g. labels) given natural language descriptions. To learn a new concept (e.g. a classifier), we search directly in the space of descriptions to minimize the interpreter's loss on training examples. Crucially, our models do not require language data to learn these concepts: language is used only in pretraining to impose structure on subsequent learning. Results on image classification, text editing, and reinforcement learning show that, in all settings, models with a linguistic parameterization outperform those without.
BibTeX
@inproceedings{andreas2018learning,
  author = {Jacob Andreas and Dan Klein and Sergey Levine},
  title = {{Learning with Latent Language}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
  doi = {10.18653/v1/n18-1197},
}
Figure