Automated Induction of Sense in Context

James Pustejovsky, Patrick Hanks, Anna RumshiskyCOLING 2004 · OpenReview

James Pustejovsky, Patrick Hanks, Anna Rumshisky
COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics

Abstract

Traditional Word Sense Disambiguation (WSD) often fails by treating word senses as static dictionary entries. This paper introduces a shift toward inducing meaning through selection context—the stereotypical patterns of usage a word inhabits. By proposing Corpus Pattern Analysis (CPA), the authors demonstrate how meanings can be assigned to syntactic patterns rather than individual words, using a shallow semantic ontology to bootstrap automatic sense induction from large-scale corpora.

Introduction: The Problem with Static Lexicons

Conventional WSD systems typically rely on "topical" features or simple N-grams to distinguish meanings. However, a word’s meaning often hinges on its structural role. For instance, the difference between "treating a patient" (medical) and "treating someone with respect" (behavioral) is defined by the specific syntactic "slots" filled by surrounding words.

Existing resources like FrameNet are often too manually intensive, while statistical clustering frequently lacks the semantic "glue" to explain why a word takes on a specific meaning in context. The researchers address this by leveraging the Generative Lexicon (GL) theory to create a more fluid model of language understanding.

Methodology: Corpus Pattern Analysis (CPA)

The core of this research is CPA, a three-step process designed to map syntactic patterns to semantic types:

  1. Lexical Discovery: A lexicographer analyzes instances of a verb (e.g., from the British National Corpus) to identify "norms" of usage. These are encoded into a formal grammar including shallow syntactic parsing and semantic typing.
  2. Shallow Semantic Typing: The system utilizes the Brandeis Shallow Ontology (BSO), a hierarchy of approximately 65 semantic types (e.g., Person, Institution, PhysObj) to tag verb arguments.
  3. Automatic Recognition and Acquisition: Machine learning models (ID3 and k-Nearest Neighbor) are used to classify new instances into established patterns. A bootstrapping phase then identifies new clusters of nouns that fill specific slots, allowing the system to "learn" selectional preferences.
The Brandeis Shallow Ontology (BSO) provides a hierarchy of semantic types used to tag verb arguments during the lexical discovery phase.

Key Insights

  • Context Over Lexicon: Senses are not pre-encoded; a word is a "meaning potential" realized only when it plugs into a syntagmatic pattern.
  • The Power of "Minor" Features: Adverbials of manner can be primary discriminators. For example, an adverbial in "treat [Person] [Adv-Manner]" distinguishes behavioral senses from medical ones.
  • Handling Coercion: By using GL theory, the system can handle "coercion," where a word's type is shifted by its context (e.g., a "therapy" acting as a "Doctor" in the context of "treating cancer").

Experimental Results

The authors evaluated their pattern recognition approach on three verbs with varying levels of complexity using the BNC and Senseval-3 verb sets.

Accuracy results for pattern recognition across three verbs using ID3 and kNN algorithms.

As shown in the table, the system achieved high accuracy (up to 87%) for the verb edit, which has distinct, well-defined patterns. The lower performance for treat reflects the more complex semantic interactions and subtle distinctions inherent in its usage patterns. Generally, kNN outperformed ID3 in handling these nuances.

Conclusion and Future Work

This study argues for a move away from "sense enumeration" toward a context-driven model. While the current scale is preliminary—relying on a BSO of 65 types and significant manual discovery—future work aims to automate pattern discovery further and scale the ontology. This approach holds promise for improving complex NLP tasks such as anaphora resolution and domain-specific inference.

Summary updated

Abstract

James Pustejovsky, Patrick Hanks, Anna Rumshisky. COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics. 2004.
BibTeX
@inproceedings{pustejovsky2004automated,
  author = {James Pustejovsky and Patrick Hanks and Anna Rumshisky},
  title = {{Automated Induction of Sense in Context}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2004},
  url = {https://openreview.net/forum?id=HUKcWWtXPqA},
}
Figure