Learning with Latent Language

Jacob Andreas, Dan Klein, Sergey Levine

Standard machine learning models typically require thousands of examples to master a new concept. In contrast, humans can learn from just one or two examples because we possess rich background knowledge about the world's structure. While "few-shot learning" research attempts to bridge this gap using meta-learning, most existing methods represent learned priors as abstract vectors of real numbers. These vectors are often hard to interpret and fail to capture the high-level, compositional abstractions—like spatial relationships or logical rules—that make human reasoning efficient.

This paper introduces Learning with Latent Language ($L^3$), a framework that uses natural language as a "latent parameter space" for few-shot learning. By pretraining models to understand descriptions and then searching for the best description to explain a new task, the authors show that language provides a powerful, compositional prior that helps models learn faster and more interpretably than standard approaches.

The $L^3$ Framework

The $L^3$ approach consists of three distinct phases:

  1. Language Learning (Pretraining): The model learns a language interpretation model ($f$) that relates natural language descriptions to visual or procedural inputs. It learns the "semantics" of language—how words relate to features like color, shape, or position.
  2. Concept Learning: When faced with a new task, the model does not update its neural weights. Instead, it searches for a natural language string ($w$) that best describes the provided examples. To make this search efficient, a proposal model ($q$) suggests likely descriptions based on the data.
  3. Evaluation: The chosen description is plugged back into the interpreter ($f$) to make predictions on new, unseen data.
The $L^3$ framework involves (a) pretraining an interpreter, (b) searching for a descriptive string for a new task, and (c) using that string for evaluation.

By searching in "string space" rather than "weight space," the model is restricted to hypotheses expressible in language, which acts as a regularizer against overfitting to low-level statistical noise.

Experimental Results

The authors evaluated $L^3$ across three diverse domains: ShapeWorld (spatial relations), String Editing (transformation rules), and Treasure Hunting (navigation). In all three, $L^3$ outperformed standard multitask and meta-learning baselines:

  • Image Classification: $L^3$ achieved 70% accuracy on novel concepts, compared to 64% for the best meta-learning baseline.
  • String Editing: $L^3$ reached 80% accuracy in predicting the correct output string, significantly higher than the 66% achieved by standard meta-learning.
  • Policy Search: In RL, the $L^3$ agent discovered high-reward policies much faster than "from-scratch" RL. Once a description was sampled, the reward typically jumped almost immediately.

Key Insights and Interpretability

Forcing the model to "think" in language prevents it from picking up on statistical noise. Because the latent parameters are actual words, the model is inherently more interpretable; researchers can see exactly what the model believes the rule for a task to be.

During concept learning, the model evaluates candidate descriptions against the provided images to find the most likely explanation.

Interestingly, the model often makes correct predictions even when its inferred description doesn't perfectly match the ground truth. For example, it might describe a "circle" as a "shape" but still correctly identify the spatial relationship, showing that the learned semantics are flexible. Unlike rigid formal logic, natural language's "fuzzy" semantics allow the model to handle visual ambiguity better than a strict logical solver.

Limitations and Future Work

Searching through the space of all possible natural language strings remains computationally expensive. Furthermore, the model is only as good as its pretraining data; if the pretraining tasks don't cover the vocabulary needed for a new task, the model will fail to find a valid description. Future work will explore more sophisticated search algorithms and apply the method to broader domains like robotic manipulation.

Summary updated

BibTeX
@inproceedings{andreas2018learning,
  author = {Jacob Andreas and Dan Klein and Sergey Levine},
  title = {{Learning with Latent Language}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
  doi = {10.18653/v1/n18-1197},
}
Figure