WordNet::SenseRelate::AllWords - A Broad Coverage Word Sense Tagger that Maximizes Semantic Relatedness

Ted Pedersen, Varada KolhatkarNAACL 2009

TL;DR

This paper introduces WordNet::SenseRelate::AllWords (SR-AW), a Perl package that automatically assigns the correct meaning (sense) to every content word in a text using only existing knowledge in WordNet. This approach bypasses the need for expensive manual annotation, offering a practical solution for broad-coverage word sense disambiguation (WSD).

What Problem Does This Paper Address?

Word Sense Disambiguation (WSD) is a fundamental challenge in Natural Language Processing (NLP). It's the task of figuring out which meaning of a word is intended in a given context. For example, "bank" can refer to a financial institution or the side of a river. While supervised learning methods have achieved good accuracy for specific words, they require vast amounts of manually labeled data, which is time-consuming and expensive to create. This makes it impractical to build WSD systems that cover all words in a language. SR-AW aims to solve this by leveraging the rich semantic information already present in WordNet, a lexical database, to achieve broad coverage without manual annotation.

Methods

SR-AW processes text sentence by sentence. For each sentence, it iterates through the words, considering each "content word" (nouns, verbs, adjectives, adverbs) as a potential target. It uses a "window" of surrounding words to determine the context. The core idea is to find the sense of the target word that is most semantically related to the senses of the words within its context window.

The system utilizes measures from the WordNet::Similarity package, which offers ten different ways to calculate semantic relatedness or similarity between WordNet concepts. These measures can be based on path lengths within WordNet's hierarchy, information content derived from corpora, or even the textual definitions (glosses) of words. The sense of the target word that achieves the highest sum of pairwise relatedness scores with its neighbors is chosen. SR-AW can also optionally fix the senses of previously disambiguated words to improve accuracy.

Datasets

The paper evaluates SR-AW using three corpora:

  • SemCor: A corpus of over 200,000 words from the Brown Corpus, manually annotated with WordNet senses.
  • SENSEVAL-2 and SENSEVAL-3: Datasets from well-known WSD evaluation campaigns, each containing approximately 4,000 words from the Wall Street Journal, also annotated with WordNet senses.

The authors note that only words known to WordNet in these corpora were sense-tagged, resulting in 185,273 tagged words in SemCor, 2,260 in SENSEVAL-2, and 1,937 in SENSEVAL-3. They used versions of these corpora where senses were mapped to WordNet 3.0.

Key Insights and Ideas

  • Leveraging existing knowledge: The most significant insight is that the vast semantic network within WordNet, combined with robust similarity measures, is sufficient for broad-coverage WSD, eliminating the need for manual training data.
  • Context window flexibility: The ability to adjust the size of the context window allows for a trade-off between capturing broader context and potentially introducing noise.
  • Richness of similarity measures: The ten different similarity/relatedness measures available in WordNet::Similarity provide a flexible toolkit, allowing users to tailor the disambiguation process to their specific needs and data characteristics.
  • Importance of compounds: The paper highlights that correctly identifying multi-word expressions (compounds) like "White House" is crucial for accurate disambiguation, as treating them as separate words can lead to incorrect sense assignments.

Key Results

SR-AW achieved competitive results on recent evaluations. Using the lesk similarity measure with a window size of 15, the system achieved F-measures of:

  • 61% for SemCor
  • 59% for SENSEVAL-2
  • 54% for SENSEVAL-3

These results would have placed SR-AW among the top-performing systems in the respective SENSEVAL events. The paper also establishes baselines:

  • Sense one baseline: Assigning the most frequent sense resulted in F-measures of 76% (SemCor), 69% (SENSEVAL-2), and 68% (SENSEVAL-3).
  • Random baseline: Randomly assigning senses yielded F-measures of 41% (SemCor), 41% (SENSEVAL-2), and 37% (SENSEVAL-3).

SR-AW significantly outperforms the random baseline and, while lagging behind the sense-one baseline (a common characteristic of broad-coverage systems), demonstrates its effectiveness.

Limitations and Future Work

Limitations: * Sense-one baseline performance: SR-AW's performance lags behind the "sense one" baseline, which is a common observation for broad-coverage WSD systems. This suggests that while SR-AW is effective, it doesn't fully capture the frequency bias that often drives the sense-one heuristic. * Dependency on WordNet: The system's performance is inherently tied to the coverage and quality of WordNet itself. Words not present in WordNet or compounds not identified will not be disambiguated. * Window size trade-offs: While flexible, choosing the optimal window size requires experimentation.

Future Work: The paper mentions that SR-AW is "being significantly enhanced," implying ongoing development. Specific future directions are not detailed, but potential areas could include:

  • Improving compound detection.
  • Integrating external knowledge sources beyond WordNet.
  • Exploring more sophisticated context modeling techniques.
  • Further optimizing the selection of similarity measures and window sizes for different types of text.

Summary updated

BibTeX
@inproceedings{pedersen2009wordnet,
  author = {Ted Pedersen and Varada Kolhatkar},
  title = {{WordNet::SenseRelate::AllWords - A Broad Coverage Word Sense Tagger that Maximizes Semantic Relatedness}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2009},
}
Figure