Word learning as Bayesian inference

Joshua B. Tenenbaum, Fei XueScholarship 2000

Abstract

We apply a computational theory of concept learning based on Bayesian inference (Tenenbaum, 1999) to the problem of learning words from examples. The theory provides a framework for understanding how people can generalize meaningfully from just one or a few positive examples of a novel word, without assuming that words are mutually exclusive or map only onto basic-level categories.

Introduction: The Induction Problem

When a child hears a novel word like "dog" while pointing at a Golden Retriever, they face an "induction problem": the word could refer to that specific dog, the breed, the species, or all mammals. Traditional theories often rely on rigid constraints like the "taxonomic assumption" or "mutual exclusivity." However, these do not explain how learners flexibly navigate different levels of a hierarchy (e.g., learning "Dalmatian" vs. "Dog" vs. "Animal"). This paper proposes a Bayesian framework to explain "fast mapping"—the ability to learn a word's meaning from minimal data.

Theoretical Framework: The Size Principle

The authors treat word learning as Bayesian inference over a hypothesis space structured as a hierarchical taxonomic tree. The model's core innovation is the Size Principle in the likelihood function $P(X|h)$. It assumes that examples are sampled at random from the true category. Consequently, smaller, more specific categories provide a much higher likelihood for a set of examples than broad ones. Seeing three Dalmatians labeled "blick" is a "suspicious coincidence" if the word meant "all animals"; it is far more likely to mean "Dalmatians."

Experimental Methodology and Results

The model was tested against behavioral data from 25 adults and a group of 4-year-olds. Participants were shown one or three examples of a novel word and asked to generalize to other objects from a set containing subordinate, basic-level, and superordinate matches.

Generalization judgments across stimulus categories. Panel (a) shows the average across all categories, while (b-d) show results for Animals, Vehicles, and Vegetables. Note the shift from graded generalization with one example to specific, rule-like generalization with three examples.

The Bayesian model achieved a high quantitative fit ($R^2 = 0.93$) with adult data. A key finding was the transition from graded to rule-like generalization. With one example, humans show graded generalization based on similarity, often favoring the basic level (e.g., "dog"). With three examples, they jump to a "rule-like" generalization, picking the most specific category consistent with all examples. Preliminary data from children showed the same fundamental shift, suggesting the Bayesian engine is active early in life.

Discussion and Limitations

The model reconciles rationalist ideas (pre-existing taxonomic structures) with empiricist statistical learning. It shows that the "size principle" acts as a soft constraint that naturally penalizes overly broad hypotheses as more data is encountered.

However, the model assumes the learner already possesses a hierarchical "map" of the world and does not explain how these trees are initially constructed. Additionally, a manual "basic-level bias" was added to the prior to fully match human preference for categories like "dog" over "Dalmatian" when given only one example. Future work will explore if this bias can be derived from environmental structure and how the framework applies to other word types like verbs or adjectives.

Summary updated

Abstract

We apply a computational theory of concept learning based on Bayesian inference (Tenenbaum, 1999) to the problem of learning words from examples. The theory provides a framework for understanding how people can generalize meaningfully from just one or a few positive examples of a novel word, without assuming that words are mutually exclusive or map only onto basic-level categories. We also
BibTeX
@inproceedings{tenenbaum2000word,
  author = {Joshua B. Tenenbaum and Fei Xu},
  title = {{Word learning as Bayesian inference}},
  booktitle = {eScholarship},
  year = {2000},
}
Figure