Corpus-Based Induction of Lexical Representation and Meaning

Maria LapataAAAI 1999

Introduction

In the late 1990s, the field of linguistics faced a significant divide. Traditional linguistics relied on hand-crafted, theory-heavy representations to explain complex language structures, while the burgeoning field of data-driven Natural Language Processing (NLP) excelled at low-level tasks like part-of-speech tagging but struggled to capture high-level semantic meaning. This paper bridges that gap by addressing the "logical problem of language acquisition": how structured semantic representations can be induced directly from raw linguistic input. By leveraging large-scale text corpora, the research demonstrates that statistical patterns in text are not merely noise, but a valid proxy for internal human linguistic knowledge.

The Challenge of Lexical Semantics

The primary challenge addressed is the automatic induction of deep lexical knowledge—specifically, how verbs behave (subcategorization) and what they mean (semantic classes). The author seeks to move beyond simple frequency counts to extract structured information that aligns with human psycholinguistic intuitions. The goal is to prove that complex lexical semantic knowledge can be automatically derived from corpora, ensuring these computational models remain psychologically plausible.

Methodological Framework

The research employs a robust three-step pipeline to transform raw text into structured meaning:

  1. Induction: Using partial-parsing techniques, the model extracts "low-level" syntactic information from text. This includes subcategorization frames (the specific syntactic structures a verb permits, such as the double-object construction in "he gave the boy a book") and selectional restrictions (the semantic categories of nouns that typically fill those syntactic slots).
  2. Combination: These low-level features are integrated into high-level semantic representations. The author utilizes a probabilistic framework to map corpus data onto Levin’s (1993) taxonomy of verb classes. This step also incorporates WordNet to provide a semantic backbone, allowing the model to quantify generalizations like the "typicality" of specific verb behaviors.
  3. Evaluation: To ensure the model's validity, the resulting statistical representations are compared against human experimental data. This step tests whether the machine-learned patterns correlate with how humans actually process and categorize language.

Data Sources

The study utilizes several key datasets to build and validate the model:

  • British National Corpus (BNC): A 100-million-word collection of British English used as the primary source for inducing frequencies and syntactic frames.
  • WordNet: A lexical database used for taxonomic information and semantic grounding.
  • Psycholinguistic Data: Gold-standard experimental results from previous studies (e.g., Lapata & Keller 1998) used to measure the correlation between corpus-derived data and human intuition.

Key Insights: Corpora as a Mirror of the Mind

A central finding of this research is that statistical patterns in large corpora serve as a "mirror of the mind." The author discovered a high correlation between the frequency of syntactic structures in the BNC and the preferences shown by human subjects in psycholinguistic tasks. This suggests that the statistical distribution of language in the world is deeply intertwined with our internal linguistic representations.

Furthermore, the paper demonstrates that abstract linguistic concepts—such as the "productivity" of a verb alternation—can be mathematically quantified. By layering formal semantic theories (like Levin’s verb classes) over raw statistical data, the model can extract refined features that purely distributional analysis might miss. This "hybrid advantage" suggests that the future of NLP lies in the synergy between statistical methods and formal linguistic theory.

Results and Implications

The research yielded several significant results:

  • Statistical Correlation: Subcategorization frames acquired from the BNC showed a strong correlation with human-reported frequencies, validating the use of corpora for psycholinguistic modeling.
  • Productivity and Alternation: The model successfully quantified the productivity of diathesis alternations (e.g., the relationship between the causative "I broke the window" and the inchoative "The window broke").
  • Semantic Disambiguation: The study proved that verb semantic classes could be effectively disambiguated using a probabilistic model fueled by corpus-derived syntactic information.

Limitations and Future Directions

While the results are promising, the author notes that the model is currently constrained by its dependency on existing frameworks like Levin’s taxonomy. The quality of the high-level induction is inherently tied to the coverage of the underlying theory. Additionally, while the correlation with human intuition is high, identifying the specific "refined semantic features" that distinguish closely related verb classes remains an ongoing challenge. Future work will require more extensive evaluation to fully bridge the gap between data-driven induction and the nuances of traditional linguistic analysis.

Summary updated

BibTeX
@inproceedings{lapata1999corpus,
  author = {Maria Lapata},
  title = {{Corpus-Based Induction of Lexical Representation and Meaning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1999},
}
Figure