A Bayesian Framework for Cross-Situational Word-Learning

Michael C. Frank, Noah D. Goodman, Joshua B. TenenbaumNeurIPS 2007 · OpenReview

For infants, early word learning presents a classic chicken-and-egg problem. To understand social context, you need to know some words, but to learn what words mean, you often need to rely on social context. When a mother says "Look at the dax!" while holding a rattle and looking at a ball, does "dax" mean the rattle, the ball, the act of looking, or the hand?

Existing theories usually fall into two camps: innate constraints (where children are born with rules like mutual exclusivity that limit the search space) and social bootstrapping (where children use social cues to narrow down referents). However, children also have to learn which social cues are informative—for instance, a mother's gaze is a much better cue than her elbow moving.

This 2008 paper by Frank, Goodman, and Tenenbaum presents a landmark approach to solving this dilemma. The authors propose a Bayesian generative model that treats word learning as an inference about a speaker's hidden intentions. By simultaneously tracking cross-situational statistics and social cues, the model not only learns a lexicon but also discovers which social cues are actually relevant for determining meaning.

Methods: A Generative Model of Intent

The authors frame word learning as a Bayesian generative model. Instead of directly mapping words to objects, they assume a speaker has a hidden intent ($I_s$)—a specific subset of objects in the environment they want to talk about. This intent then generates two observable outputs: utterances and social cues.

Graphical model of the Bayesian framework
  1. Utterances ($W_s$): Words are generated based on a lexicon ($L$). The model distinguishes between referential words (mapped to objects) and non-referential "filler" words (like "the" or "look").
  2. Social Cues ($S_s$): Cues like eye gaze or touching are generated from the intent via a "noisy-or" process. This allows the model to learn the relevance ($r$) and base rate ($b$) of different cues.

To find the best lexicon, the model performs Maximum A Posteriori (MAP) inference using a stochastic search (simulated tempering). It looks for the lexicon that provides the most parsimonious explanation for all observed situations.

Datasets

The model was tested on naturalistic data from the Rollins section of the CHILDES corpus. The dataset consisted of two video files featuring roughly 20 minutes of mother-infant play with a box of toys. The authors manually annotated the videos for visible objects and seven social features, including the infant's eyes, hands, mouth, and touching, as well as the mother's eyes, hands, and touching. They then compared the model's learned lexicon against a "gold-standard" human-coded lexicon.

Key Insights and Ideas

  • Intent as the Latent Variable: The core breakthrough of this model is treating the speaker's intention as the bridge. Words and social cues aren't directly associated with each other; rather, they are both independent consequences of what the speaker is thinking.
  • Emergent Constraints: Phenomena like Mutual Exclusivity (the tendency to assume a new word refers to a new object) aren't hard-coded rules. In this model, they emerge naturally because the Bayesian prior favors smaller, more efficient lexicons. Mapping a new word to an already-known object would require a more complex, less probable explanation of the data.
  • Learning to Learn: The model doesn't just use social cues; it discovers them. It correctly identified that the "infant’s gaze" and "caregiver’s hand" were the most relevant cues for reference in the corpus, effectively learning which parts of the social environment to attend to without prior hard-coding.

Key Results

The Bayesian model demonstrated superior performance in extracting a lexicon from the noisy, naturalistic corpus data.

Precision-recall comparison of models
  • Quantitative Superiority: The Bayesian model achieved an F-score of 0.54, significantly outperforming simple associative models (which scored between 0.12 and 0.21) and a previous state-of-the-art machine translation-based model (0.44).
  • The Social Boost: While social cues didn't drastically change performance on the full corpus, they allowed the model to succeed on much smaller datasets. Furthermore, the model could correctly infer intentions even when no referential words were used (e.g., understanding the target of "See that?" based on gaze alone).
  • Experimental Alignment: The model qualitatively matched child behavior in classic psychological experiments, successfully replicating phenomena like fast-mapping (learning from a single exposure) and social generalization.

Limitations and Future Work

While groundbreaking, the framework has a few notable limitations. First, it relies on simplified syntax, treating sentences as "bags of words" and only distinguishing between referential and non-referential words without accounting for word order or complex grammar. Second, the MAP inference is performed over the entire corpus at once (batch processing), whereas real infants learn incrementally and face memory constraints. Finally, the dataset used was very small (20 minutes of interaction). Testing the model's scalability to hundreds of hours of input remains a vital challenge for future research.

Summary updated

Abstract

For infants, early word learning is a chicken-and-egg problem. One way to learn a word is to observe that it co-occurs with a particular referent across different situations. Another way is to use the social context of an utterance to infer the in- tended referent of a word. Here we present a Bayesian model of cross-situational word learning, and an extension of this model that also learns which social cues are relevant to determining reference. We test our model on a small corpus of mother-infant interaction and find it performs better than competing models. Fi- nally, we show that our model accounts for experimental phenomena including mutual exclusivity, fast-mapping, and generalization from social cues. To understand the difficulty of an infant word-learner, imagine walking down the street with a friend who suddenly says “dax blicket philbin na fivy!” while at the same time wagging her elbow. If you knew any of these words you might infer from the syntax of her sentence that blicket is a novel noun, and hence the name of a novel object. At the same time, if you knew that this friend indicated her attention by wagging her elbow at objects, you might infer that she intends to refer to an object in a nearby show window. On the other hand if you already knew that “blicket” meant the object in the window, you might be able to infer these elements of syntax and social cues. Thus, the problem of early word-learning is a classic chicken-and-egg puzzle: in order to learn word meanings, learners must use their knowledge of the rest of language (including rules of syntax, parts of speech, and other word meanings) as well as their knowledge of social situations. But in order to learn about the facts of their language they must first learn some words, and in order to determine which cues matter for establishing reference (for instance, pointing and looking at an object but normally not waggling your elbow) they must first have a way to know the intended referent in some situations. For theories of language acquisition, there are two common ways out of this dilemma. The first involves positing a wide range of innate structures which determine the syntax and categories of a language and which social cues are informative. (Though even when all of these elements are innately determined using them to learn a language from evidence may not be trivial [1].) The other alternative involves bootstrapping: learning some words, then using those words to learn how to learn more. This paper gives a proposal for the second alternative. We first present a Bayesian model of how learners could use a statistical strategy—cross-situational word-learning—to learn how words map to objects, independent of syntactic and social cues. We then extend this model to a true bootstrapping situation: using social cues to learn words while using words to learn social cues. Finally, we examine several important phenomena in word learning: mutual exclusivity (the tendency to assign novel words to novel referents), fast-mapping (the ability to assign a novel word in a linguistic context to a novel referent after only a single use), and social generalization (the ability to use social context to learn the referent of a novel word). Without adding additional specialized machinery, we show how these can be explained within our model as the result of domain-general probabilistic inference mechanisms operating over the linguistic domain. 1 Figure 1: Graphical model de- scribing the generation of words (Ws) from an intention (Is) and lexicon ((cid:96)), and intention from the objects present in a situa- tion (Os). The plate indicates multiple copies of the model for different situation/utterance pairs (s). Dotted portions indicate ad- ditions to include the generation of social cues Ss from intentions.
BibTeX
@inproceedings{frank2007bayesian,
  author = {Michael C. Frank and Noah D. Goodman and Joshua B. Tenenbaum},
  title = {{A Bayesian Framework for Cross-Situational Word-Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2007},
  url = {https://openreview.net/forum?id=HkZEpUWd-H},
}
Figure