Lexical Acquisition in the Presence of Noise and Homonymy

Jeffrey Mark Siskind

Introduction

In the study of language acquisition, the "Mapping Problem" represents one of the most significant hurdles for a developing child. This problem, famously illustrated by Quine’s "Gavagai" thought experiment, asks how a learner can determine the specific meaning of a word when the surrounding environment offers an infinite number of potential referents. When a parent says, "Mommy raised the ball," does "ball" refer to the object itself, the act of lifting, the shape, or perhaps the color?

In his landmark 1994 AAAI paper, Jeffrey Mark Siskind presents a computational model designed to solve this mapping problem under realistic conditions. Unlike previous models that relied on idealized, clean data, Siskind’s algorithm is built to handle the messiness of real-world learning: multi-word utterances, referential uncertainty, noise, and homonymy.

The Challenges of Real-World Learning

Most early computational models of lexical acquisition operated under several fragile assumptions: that children learn from single-word utterances, that meanings are always transparent in the immediate environment, and that each word corresponds to exactly one meaning. Siskind argues that these assumptions do not hold in naturalistic settings. Instead, he identifies three primary obstacles:

  1. Referential Uncertainty: A single sentence is often paired with multiple possible situational meanings.
  2. Noise: The intended meaning might occasionally be missing from the learner's set of hypotheses due to errors in perception or conceptualization.
  3. Homonymy: A single word (e.g., "bank") can map to multiple distinct, unrelated meanings.

The Proposed Algorithm

Siskind formalizes lexical acquisition as a search through a hypothesis space of logical expressions. The learner receives a sequence of utterances, each associated with a set of possible logical representations for the entire sentence. The model then employs several key mechanisms to converge on the correct lexicon.

Version-Space Filtering

The algorithm maintains two sets for each word sense $s$: $N(s)$, representing symbols that must be part of the meaning, and $P(s)$, representing symbols that could be part of the meaning. By observing the word across different contexts (cross-situational learning), the learner narrows the gap between $N$ and $P$. When $N(s) = P(s)$, the meaning is considered learned.

Compositionality and Constraints

The model assumes a "partially linear" semantic interpretation rule. It posits that the meaning of a sentence is a composition of the meanings of its constituent words. This constraint prevents the algorithm from assigning arbitrary meanings that do not contribute to the overall sentence structure.

Sense Splitting for Homonymy

To address homonymy, Siskind introduced a "sense-splitting" mechanism. If a new context renders a word's current hypothesized meaning inconsistent (i.e., no possible meaning fits), the algorithm does not fail. Instead, it creates a new sense for that word and tracks both hypotheses independently. To prevent the system from being overwhelmed by "noise" (spurious senses created by errors), a "temperature" or confidence metric is used to prune or ignore low-confidence senses over time.

Experimental Evaluation

Due to the lack of annotated child-directed speech corpora in the early 1990s, Siskind evaluated the algorithm using large-scale synthetic corpora.

  • Lexicon and Corpus: The synthetic lexicon contained 1,000 words mapped to 1,680 senses. The algorithm was tested on a corpus of 246,439 random utterances.
  • Stress Testing: The data included a 20% noise rate (where the correct meaning was absent from the hypothesis set) and a 10-way referential uncertainty (10 possible meanings per sentence).

Key Results

The results demonstrated remarkable robustness. The algorithm successfully recovered at least one correct meaning for 100% of the 1,000 words. Across all 1,680 word-to-meaning mappings, the combined error rate was a mere 2.5%. Furthermore, the algorithm proved highly scalable; as an "on-line" learner, it processed sentences one at a time without needing to store the entire corpus in memory.

Insights and Implications

Siskind’s work offers several profound insights into the nature of language learning:

  • Implicit Negative Evidence: While linguists debate whether children require explicit correction, Siskind shows that cross-situational learning provides "implicit negative evidence." By seeing what symbols are not present across multiple uses of a word, the learner can effectively rule out incorrect hypotheses.
  • The Cost of Homonymy: The experiments revealed that homonymy significantly increases the difficulty of the task. Learning a language with multiple word senses required approximately 20 times more data than a language with a one-to-one mapping between words and meanings.
  • Upper-Bound Reasoning: The model demonstrates that convergence can often be achieved by focusing on the "upper bound" (what might be true), assuming that each word sense eventually maps to a single, specific logical expression.

Conclusion and Future Work

While the model was a major step forward, Siskind noted several limitations. The "meanings" used were simplified S-expressions, which may not capture the full complexity of human conceptual thought. Additionally, the model intentionally ignored word order to test the limits of cross-situational inference. Future iterations would benefit from integrating syntactic cues (word order and grammar) to further constrain the search space. Despite these simplifications, the paper remains a foundational study in how robust, noise-tolerant lexical acquisition can be achieved through principled computational constraints.

Summary updated

Abstract

This paper conjectures a computational account of how children might learn the meanings of words in their native language. First, a simplified version of the lexical acquisition task faced by children is modeled by a precisely specified formal problem. Then, an implemented algorithm for solving this formal problem is presented. Key advances of this algorithm over previously proposed algorithms are its ability to learn homonymous word senses in the presence of noisy input and its ability to scale up to problems of the size faced by real children.
BibTeX
@inproceedings{siskind1994lexical,
  author = {Jeffrey Mark Siskind},
  title = {{Lexical Acquisition in the Presence of Noise and Homonymy}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1994},
}
Figure