Extracting Glosses to Disambiguate Word Senses

Weisi Duan, Alexander YatesNAACL 2010

TL;DR

This paper introduces GLOSSY, an unsupervised system that automatically extracts "sense indicators" (short phrases) from unlabeled text to define different meanings of ambiguous words. These extracted indicators can then be used to train a word sense disambiguation (WSD) system, achieving performance close to supervised methods without requiring any manually labeled data.

What Problem Does This Paper Address?

Word Sense Disambiguation (WSD) is a crucial task in natural language processing, aiming to determine the correct meaning of a word in a given context. For example, the word "bank" can refer to a financial institution or the side of a river. Accurately disambiguating word senses requires world knowledge. However, acquiring this knowledge is a major bottleneck. Traditional methods rely on:

  • Supervised learning: Requires large amounts of manually labeled data, which is expensive and time-consuming to create, and doesn't scale well to the vast number of words and their senses.
  • Knowledge-based methods: Use resources like dictionaries or WordNet. These are limited by the coverage of these resources and often require manual curation.
  • User-contributed knowledge: Like Wikipedia, which can be noisy and may not cover all word senses comprehensively.

The core problem is the "open-ended knowledge acquisition" challenge: the knowledge needed for WSD is vast and constantly evolving, making it impractical to manually create comprehensive resources for all words. This paper tackles this by proposing an unsupervised method to automatically extract the necessary world knowledge.

Methods

The paper proposes the GLOSSY system, which operates in two phases:

  1. Collocation Detection: This phase aims to identify potential "sense indicators" (features) that are strongly associated with specific senses of an ambiguous word. The key insight here is that features that rarely or never co-occur in the same document are likely to belong to different senses of the ambiguous word. GLOSSY identifies these "non-overlapping" feature pairs by analyzing co-occurrence statistics in a large corpus. It then selects the most frequent non-overlapping pairs as potential features. Potential features include stems, words, and bigrams.
  2. Arranging Glosses: This phase takes the detected features and groups them into coherent "glosses" (sets of sense indicators) for distinct senses of the word. GLOSSY uses a scoring function that balances several statistics, such as the number of clusters (senses) and the number of documents explained by the glosses. It employs a greedy hill-climbing search with random restarts to find an optimal arrangement of features into glosses. The goal is to create glosses that are distinct and cover as many senses as possible.

The paper also describes a Bootstrapping WSD System. This system uses the glosses extracted by GLOSSY to automatically label a corpus. Documents are labeled based on which cluster of features (gloss) appears in them. This "bootstrap corpus" is then used to train a standard supervised WSD classifier (a multi-class SVM) using a rich set of features from the surrounding context.

Datasets

  • Training/Development Data: A dataset of biomedical literature abstracts from Duan et al. (2009) was used. This dataset contained 21 ambiguous terms. One term ("MCP") was reserved for parameter tuning.
  • Unlabeled Corpus: For each ambiguous term, a large collection of unlabeled documents was gathered by searching PubMed Central.
  • Evaluation Data: The evaluation used the same biomedical abstracts dataset. For comparison, a "sampled data" set was also created by manually labeling a random sample of 100 documents for each ambiguous term.
  • External Resource: The Unified Medical Language System (UMLS) Metathesaurus was used as a gold standard for evaluating the extracted glosses.

Key Insights and Ideas

  • Non-overlapping features as sense indicators: The core idea that features rarely co-occurring are likely tied to different senses is a powerful heuristic for unsupervised sense discovery.
  • Automatic knowledge acquisition for WSD: The paper demonstrates that it's possible to bypass manual annotation for WSD by automatically extracting relevant knowledge from unlabeled text.
  • Bootstrapping WSD: Using automatically extracted glosses to create labeled data for a supervised classifier is an effective way to leverage unsupervised extraction for a supervised task.
  • GLOSSY's ability to discover novel senses: The system can identify valid senses that are not present in established dictionaries like UMLS, highlighting its potential for extending lexical resources.

Key Results

  • Gloss Extraction Performance: GLOSSY achieved 53% precision and 47% recall (Macro average) in identifying glosses for ambiguous biomedical terms, using only unlabeled corpora.
  • Novel Sense Discovery: The system identified 7 valid senses missing from the UMLS Metathesaurus for the 20 terms evaluated.
  • WSD Performance: The bootstrapping WSD system achieved accuracies close to supervised SVMs. On one dataset, it outperformed a strong unsupervised baseline (SENSATIONAL) by 16.9% and the standard BASELINE by 22.7%. On a more challenging "sampled data" set, it also outperformed SENSATIONAL and the BASELINE, achieving accuracy within 3% of a supervised SVM.

Limitations and Future Work

  • Data Skew: The performance of the bootstrapping WSD system can be affected if the distribution of senses in the unlabeled corpus used for gloss extraction differs significantly from the distribution in the labeled test data. This was observed with the term "mole," where the "unit of measurement" sense was overrepresented in the unlabeled data.
  • Granularity of Glosses: The current system extracts "sense indicators" (features) rather than full logical forms. The authors suggest that future work should focus on extracting richer relational information to form more complete definitions.
  • Matching Glosses to Existing Resources: The evaluation against UMLS was potentially unfair, as GLOSSY might discover valid senses not present in UMLS, or senses might differ in granularity. While they addressed this by incorporating newly discovered senses, it remains a challenge.
  • Future Work: The authors propose extending the system to extract relations, not just n-grams, to create more comprehensive glosses. They also aim to improve the ability of gloss extraction systems to more closely match the meanings of words.

Summary updated

BibTeX
@inproceedings{duan2010extracting,
  author = {Weisi Duan and Alexander Yates},
  title = {{Extracting Glosses to Disambiguate Word Senses}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2010},
}
Figure