Supervised Learning of a Probabilistic Lexicon of Verb Semantic Classes

Yusuke Miyao, Jun'ichi TsujiiEMNLP 2009 · OpenReview

Abstract

Yusuke Miyao, Jun’ichi Tsujii. Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing.

2009.

Introduction

Semantic lexicons, such as VerbNet, are foundational resources for Natural Language Processing (NLP), enabling tasks like word sense disambiguation and semantic role labeling. However, static lexicons suffer from two primary limitations: coverage and polysemy. Real-world texts frequently contain "unknown" verbs absent from these resources. Furthermore, many verbs are polysemous—belonging to multiple semantic classes—yet standard lexicons rarely provide information on the relative likelihood or "salience" of these different senses in actual usage.

This paper addresses the "all-class" problem: developing a supervised learning framework to classify any arbitrary verb into the full set of over 270 VerbNet classes while estimating a probability distribution that reflects real-world usage patterns.

Methodology

The authors frame verb classification as a supervised task using a log-linear model to estimate the probability $p(c|v)$, where $c$ is a VerbNet class and $v$ is a verb.

Training Data

The model is trained using two primary sources:

  1. VerbNet (v2.3): Provides the target classification scheme.
  2. SemLink: A resource that maps VerbNet classes to actual occurrences in the PropBank corpus, allowing the model to learn frequency-based weights for different verb senses.

Feature Engineering

The core contribution lies in the extraction of features from massive unannotated corpora (the British National Corpus and the Wall Street Journal). These features capture the syntactic and semantic behavior of verbs:

  • Syntactic Slots and Frames: Counts of subjects, objects, and prepositional phrases, along with full syntactic frames.
  • Slot Overlap: Measures participation in diathesis alternations (e.g., the causative-inchoative alternation).
  • Semantic Features: Includes distributionally similar words and "Slot Classes" (clusters of nouns appearing as arguments) to capture selectional preferences.
  • Morphosyntactic Features: Tense, voice, aspect, and Part-of-Speech (POS) of head words in syntactic slots.
Examples of features and their weights extracted from corpora.

Experimental Results

The authors evaluated their model using the British National Corpus (BNC), a 100-million-word balanced corpus, and the Wall Street Journal (WSJ), a 40-million-word news corpus.

Key Findings

  • Data Volume and Feature Quality: A critical insight was that accuracy continues to scale with corpus size. As shown in the learning curves, the model's performance had not yet plateaued even at 100 million words, suggesting that larger datasets could further improve lexical acquisition.
  • Semantics Complement Syntax: While VerbNet classes are syntactically defined, many classes share overlapping frames. The addition of semantic features (like "similar words") was essential for distinguishing these subtle boundaries.
  • Performance: The "All Features" model achieved a 10.69% error reduction in accuracy compared to previous state-of-the-art baselines. Interestingly, features from the larger, more diverse BNC outperformed those from the WSJ, even when evaluating on WSJ-based data (SemLink), highlighting the value of corpus balance.
Accuracy improves significantly as the size of the unannotated corpus increases.

Discussion and Future Work

Despite the improvements, challenges remain. Prepositional ambiguity is a major source of error; verbs taking the same prepositions but belonging to different semantic classes (e.g., "with" phrases in different contexts) are difficult to distinguish. Additionally, large "catch-all" classes in VerbNet act as noise sinks for misclassifications.

Future work suggests using this probabilistic model to assist human lexicographers in semi-automatically expanding resources like VerbNet, effectively bridging the gap between static expert-built lexicons and the dynamic nature of real-world language.

Summary updated

Abstract

Yusuke Miyao, Jun’ichi Tsujii. Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. 2009.
BibTeX
@inproceedings{miyao2009supervised,
  author = {Yusuke Miyao and Jun'ichi Tsujii},
  title = {{Supervised Learning of a Probabilistic Lexicon of Verb Semantic Classes}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2009},
  url = {https://openreview.net/forum?id=iG9I2nNg3K8},
}
Figure