Broad-Coverage Sense Disambiguation and Information Extraction with a Supersense Sequence Tagger

Massimiliano Ciaramita, Yasemin AltunEMNLP 2006 · OpenReview

Abstract

In this paper we approach word sense disambiguation and information extraction as a unified tagging problem. The task consists of annotating text with the tagset defined by the 41 Wordnet supersense classes for nouns and verbs. Since the tagset is directly related to Wordnet synsets, the tagger returns partial word sense disambiguation. Furthermore, since the noun tags include the standard named entity detection classes -- person, location, organization, time, etc. -- the tagger, as a by-product, returns extended named entity information. We cast the problem of supersense tagging as a sequential labeling task and investigate it empirically with a discriminatively-trained Hidden Markov Model. Experimental evaluation on the main sense-annotated datasets available, i.e., Semcor and Senseval, shows considerable improvements over the best known "first-sense" baseline.

Introduction: Bridging the Semantic Gap

In Natural Language Processing (NLP), researchers often face a trade-off between the high accuracy of coarse-grained Named Entity Recognition (NER) and the extreme difficulty of fine-grained Word Sense Disambiguation (WSD). While NER is typically limited to a few categories like Person or Location, WSD attempts to identify the exact dictionary definition of every word.

WSD is notoriously difficult because WordNet contains tens of thousands of hyper-specific senses, leading to data sparsity and low human agreement. The authors propose a "middle way": tagging words with Supersenses—the 41 broad lexicographer categories used by WordNet (e.g., noun.artifact, verb.motion, noun.food). This provides more semantic depth than standard NER while remaining computationally feasible.

The Supersense Tagset

The tagset consists of 26 categories for nouns and 15 for verbs. This broad categorization allows the model to identify "water" as a substance, "box" as an artifact, and "stood up" as a motion, effectively extending the scope of traditional information extraction.

The 41 WordNet supersense categories for nouns and verbs.

Methodology: Sequential Labeling with HMMs

The problem is cast as a sequential labeling task, similar to Part-of-Speech tagging. The core of the system is a Perceptron-trained Hidden Markov Model (HMM). Unlike traditional generative HMMs, this discriminative approach allows for the integration of various overlapping features:

  • Contextual Features: The target word and its immediate neighbors.
  • Morphological Features: Part-of-Speech (POS) tags and "word shape" (e.g., capitalization, digits).
  • The First-Sense Feature: The most frequent sense from WordNet is included as a feature, providing a powerful prior that the model then refines.

By treating the problem as a sequence, the model captures vital context—for instance, the likelihood of a motion verb being followed by a location noun.

Experimental Evaluation

The tagger was evaluated on two primary benchmarks:

  1. Semcor: A subset of the Brown Corpus manually annotated with WordNet senses, serving as the largest available resource.
  2. Senseval-3 (English All-Words): A standard WSD benchmark featuring Wall Street Journal articles and fiction excerpts.

Key Results

The Supersense Tagger consistently outperformed the notoriously difficult "first-sense" baseline, which always selects the most common meaning of a word.

Performance comparison of the Supersense-Tagger against random and first-sense baselines.

On the Semcor dataset, the tagger achieved an F-score of 77.18%, representing a 31% error reduction over the baseline (66.47%). On Senseval-3, it reached 70.54%, compared to the 64.09% baseline. Furthermore, on categories comparable to standard NER (Person, Group, Location, Time), the system achieved an average F-score of 82.46%, proving its utility for broad-coverage information extraction.

Conclusion and Future Directions

This work demonstrates that supersense tagging is a viable "sweet spot" for semantic labeling. While 41 classes may not capture every nuance of language, they provide a robust foundation for broad-coverage disambiguation.

Future work may address the limitations of linear sequential models by incorporating syntactic dependencies to better capture relationships between distant predicates and arguments. Additionally, more sophisticated feature engineering or domain-specific adaptation could further enhance the accuracy of this unified approach to WSD and IE.

Summary updated

Abstract

In this paper we approach word sense disambiguation and information extraction as a unified tagging problem. The task consists of annotating text with the tagset defined by the 41 Wordnet supersense classes for nouns and verbs. Since the tagset is directly related to Wordnet synsets, the tagger returns partial word sense disambiguation. Furthermore, since the noun tags include the standard named entity detection classes -- person, location, organization, time, etc. -- the tagger, as a by-product, returns extended named entity information. We cast the problem of supersense tagging as a sequential labeling task and investigate it empirically with a discriminatively-trained Hidden Markov Model. Experimental evaluation on the main sense-annotated datasets available, i.e., Semcor and Senseval, shows considerable improvements over the best known "first-sense" baseline.
BibTeX
@inproceedings{ciaramita2006broad,
  author = {Massimiliano Ciaramita and Yasemin Altun},
  title = {{Broad-Coverage Sense Disambiguation and Information Extraction with a Supersense Sequence Tagger}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2006},
  url = {https://openreview.net/forum?id=HkZHVffu-H},
}
Figure