Word vs. Class-Based Word Sense Disambiguation

Rubén Izquierdo, Armando Suárez, German RigauJAIR 2015 · DOI · OpenReview

TL;DR

This paper tackles the long-standing challenge of Word Sense Disambiguation (WSD) by proposing a novel approach that uses intermediate-level semantic classes derived from WordNet. Their method not only outperforms traditional word-sense-based WSD but also demonstrates robustness across different domains and significantly reduces the need for training data.

What Problem Does This Paper Address?

Word Sense Disambiguation (WSD) is the task of identifying the correct meaning of a word in its context. Despite decades of research and numerous evaluation exercises like SensEval and SemEval, achieving high accuracy has remained elusive. A key challenge identified by the authors is the potential mismatch between the fine-grained senses defined in resources like WordNet and the actual needs of downstream NLP applications. Existing approaches either use these fine-grained senses directly or rely on very coarse-grained "SuperSenses." The authors hypothesize that an intermediate level of abstraction, using semantically coherent groups of senses, could offer a better balance, leading to improved WSD performance and better integration with other NLP tasks. They also address the "acquisition bottleneck" problem, where supervised WSD systems require large amounts of annotated data, which is expensive and time-consuming to create.

Methods

The core of the paper introduces a method for automatically deriving "Basic Level Concepts" (BLCs) from WordNet. These BLCs represent intermediate levels of semantic abstraction. The method exploits the hierarchical structure of WordNet, specifically the hypernymy (is-a) relations. It identifies candidate BLCs by looking for local maxima in the number of relations (either all relations or just hyponymy relations) within hypernym chains. A threshold parameter ($\lambda$) is used to filter out concepts that don't subsume a minimum number of descendant synsets, ensuring a certain level of abstraction.

For the WSD system itself, the authors employ a supervised machine learning approach using Support Vector Machines (SVMs). Instead of training a classifier for each word sense (as in traditional word-based WSD), they train a classifier for each semantic class. This means that all words belonging to a particular semantic class can contribute to training that class's classifier. They also explore using semantic classes as additional features within their WSD framework.

Datasets

The paper uses the following datasets:

  • SemCor: Used for training the supervised WSD system. It's a subset of the Brown Corpus, annotated with WordNet 1.6 senses.
  • SensEval-2 (SE2) and SensEval-3 (SE3): Used for testing the WSD system. These are standard "all-words" WSD datasets with WordNet 1.7 and 1.7.1 senses, respectively.
  • SemEval-2007 "All-words Word Sense Disambiguation on a Specific Domain" task: Used for evaluating the system's robustness and domain adaptation capabilities on an environmental domain corpus.

Key Insights and Ideas

  • Intermediate Abstraction is Key: The central idea is that neither overly fine-grained senses nor overly coarse-grained SuperSenses are optimal for WSD. An intermediate level of semantic abstraction, captured by their automatically derived BLCs, strikes a better balance.
  • Automatic BLC Generation: The proposed method for automatically generating BLCs from WordNet's hierarchical structure is a significant contribution, offering a scalable way to create useful semantic groupings.
  • Class-Based WSD Reduces Data Needs: By training classifiers on semantic classes rather than individual word senses, the system can leverage more training examples, significantly reducing the "acquisition bottleneck" and enabling competitive performance with less data.
  • Semantic Features Boost Performance: Incorporating semantic class information as features within the WSD system further improves performance, demonstrating the value of semantic context.
  • Robustness Across Domains: The class-based approach shows remarkable robustness, performing well even when tested on domain-specific data that differs from the general-domain training corpus.

Key Results

  • Outperforming Baselines: The proposed class-based WSD system, particularly with their automatically derived BLCs (BLC20 and BLC50), consistently outperforms traditional word-sense-based approaches and most SensEval participants on SE2 and SE3 datasets. For instance, on SE3, their class-based systems achieved the best results for nouns and second place for verbs.
  • Competitive with State-of-the-Art: When tested on a specific domain (SemEval-2 "All-words Word Sense Disambiguation on a Specific Domain" task), their system achieved results comparable to the state-of-the-art ItMakesSense system, even outperforming it in some configurations.
  • Reduced Training Data: The class-based approach significantly reduces the amount of training data required. Learning curves show that the system achieves near-peak performance with a relatively small portion of the training corpus (e.g., 25-30% of SemCor).
  • Domain Adaptation: The system demonstrates effective domain adaptation by simply incorporating unlabeled domain-specific text as training data, highlighting the flexibility of the class-based approach.

Limitations and Future Work

  • Potential for Noise: The authors acknowledge that grouping diverse words into a single semantic class might introduce noise into the learning process. However, their results suggest that this is manageable and often outweighed by the benefits of increased data and abstraction.
  • Further Exploration of Abstraction Levels: While they explored several levels of abstraction (BLC20, BLC50, WND, SUMO, SuperSenses), there's potential for further investigation into optimal abstraction levels for different tasks or domains.
  • Exploiting Inconsistencies: The authors propose future work to exploit inconsistencies between different class-based classifiers to build a more robust system, potentially by predicting which classifier is best suited for a given context.
  • Feature Engineering: While their system uses a relatively simple feature set, future work could explore more complex features, especially for verbs, which might further boost performance.

Summary updated

Abstract

As empirically demonstrated by the Word Sense Disambiguation (WSD) tasks of the last SensEval/SemEval exercises, assigning the appropriate meaning to words in context has resisted all attempts to be successfully addressed. Many authors argue that one possible reason could be the use of inappropriate sets of word meanings. In particular, WordNet has been used as a de-facto standard repository of word meanings in most of these tasks. Thus, instead of using the word senses defined in WordNet, some approaches have derived semantic classes representing groups of word senses. However, the meanings represented by WordNet have been only used for WSD at a very fine-grained sense level or at a very coarse-grained semantic class level (also called SuperSenses). We suspect that an appropriate level of abstraction could be on between both levels. The contributions of this paper are manifold. First, we propose a simple method to automatically derive semantic classes at intermediate levels of abstraction covering all nominal and verbal WordNet meanings. Second, we empirically demonstrate that our automatically derived semantic classes outperform classical approaches based on word senses and more coarse-grained sense groupings. Third, we also demonstrate that our supervised WSD system benefits from using these new semantic classes as additional semantic features while reducing the amount of training examples. Finally, we also demonstrate the robustness of our supervised semantic class-based WSD system when tested on out of domain corpus.
BibTeX
@article{izquierdo2015word,
  author = {Rubén Izquierdo and Armando Suárez and German Rigau},
  title = {{Word vs. Class-Based Word Sense Disambiguation}},
  journal = {J. Artif. Intell. Res.},
  year = {2015},
  doi = {10.1613/jair.4727},
}
Figure