Knowledge-Based WSD and Specific Domains: Performing Better than Generic Supervised WSD

Eneko Agirre, Oier Lopez de Lacalle, Aitor SoroaIJCAI 2009 · OpenReview

Abstract

Word Sense Disambiguation (WSD) remains a foundational challenge in Natural Language Processing. While supervised systems currently lead in general-purpose benchmarks, they often falter when applied to specific domains due to data sparseness and corpus mismatch. This paper demonstrates that knowledge-based methods, specifically Personalized PageRank (PPR) applied to WordNet, can significantly outperform state-of-the-art supervised models in specialized fields like Sports and Finance. By leveraging domain-specific "related words" harvested from unlabelled text, the authors show a viable path for high-performance WSD without the prohibitive cost of hand-labeling new training data.

Introduction

Word Sense Disambiguation (WSD) is the task of identifying the correct sense of a polysemous word within a given context. State-of-the-art supervised systems, while effective on general datasets like SemCor, face two critical hurdles: the high cost of hand-labeling data (data sparseness) and the "domain gap." A model trained on general news often fails in specialized domains because the "most frequent sense" (MFS) of a word changes. For instance, "coach" typically refers to a vehicle in general text but almost exclusively to a trainer in sports news. Supervised models fail because they cannot easily "unlearn" these general biases without new labeled data.

The authors propose a knowledge-based approach using Personalized PageRank (PPR) over the WordNet graph. WordNet is treated as a network where nodes represent concepts (synsets) and edges represent semantic relations (e.g., "is-a" or "part-of"). PPR works by injecting probability mass into specific context nodes and observing the resulting distribution across the graph.

The core innovation lies in the context used to "prime" the PPR algorithm. Traditional methods use the immediate surrounding words (occurrence context). This study introduces the use of related words—a set of terms automatically extracted from the target domain's unlabelled text using a distributional thesaurus.

Accuracy of the PPR method as a function of the number of related words used as context.

As shown in the figure above, the accuracy of the system improves significantly as the number of related words increases, typically peaking between 30 and 50 words before declining as more noise is introduced.

Experimental Results

The study evaluated several systems across three corpora: the British National Corpus (BNC) for general text, and two specialized corpora for Sports and Finance. The supervised baselines (SVM and k-NN) were trained on the general-purpose SemCor dataset.

Comparison of WSD systems across general (BNC) and specific (Sports, Finance) domains.

The results indicate a substantial performance gain for knowledge-based methods in specialized domains:

  • Sports Domain: The best supervised model (k-NN) achieved 30.3% recall, whereas the PPR method using related words reached 51.5%.
  • Finance Domain: Supervised k-NN reached 43.4%, while the PPR method achieved 59.3%.
  • General Domain (BNC): Supervised models and knowledge-based models performed similarly (42.8% for k-NN vs. 43.8% for PPR), suggesting that supervised models are heavily tuned to general distributions but lack portability to specific domains.

Discussion and Conclusion

The findings suggest that knowledge-based systems are more robust to domain shifts because they rely on the underlying semantic structure of language rather than frequency biases in training data. The use of a distributional thesaurus to provide domain context effectively "primes" the system, allowing it to overcome the "MFS trap" that plagues supervised models. Future work will focus on extending these findings to "all-words" tasks and exploring hybrid systems that use knowledge-based methods to adapt supervised models to new domains without manual tagging.

Summary updated

BibTeX
@inproceedings{agirre2009knowledge,
  author = {Eneko Agirre and Oier Lopez de Lacalle and Aitor Soroa},
  title = {{Knowledge-Based WSD and Specific Domains: Performing Better than Generic Supervised WSD}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2009},
  url = {https://openreview.net/forum?id=Skbp8VfdWr},
}
Figure