A Fully Unsupervised Word Sense Disambiguation Method Using Dependency Knowledge

Ping Chen, Wei Ding, Chris Bowes, David BrownNAACL 2009

TL;DR

This paper introduces a novel, fully unsupervised method for Word Sense Disambiguation (WSD) that leverages dependency knowledge extracted from unannotated text and an electronic dictionary. The approach achieves performance comparable to supervised methods, overcoming the limitations of data scarcity and brittleness in existing WSD systems.

What Problem Does This Paper Address?

Word Sense Disambiguation (WSD) is the crucial task of identifying the correct meaning of a word in a given context, especially for words with multiple senses (homographs). This is fundamental for understanding natural language and impacts applications like machine translation and information retrieval. While supervised methods currently offer the highest accuracy, they require large, manually annotated datasets, which are expensive to create and lead to systems that are "brittle" – performing poorly outside their specific training domains. Unsupervised methods, while avoiding annotation costs, often struggle with noisy data and achieve lower accuracy. The core challenge addressed here is how to achieve broad-coverage and high-quality WSD without relying on manually annotated data, thus overcoming the knowledge acquisition bottleneck.

Methods

The proposed method is fully unsupervised and relies on two primary knowledge sources: an electronic dictionary (specifically, WordNet for sense inventories) and unannotated text. The process involves several key steps:

  1. Corpus Building via Web Search: The system queries web search engines for instances of the word to be disambiguated, retrieving a large number of sentences.
  2. HTML Cleaning and Sentence Segmentation: Raw web pages are cleaned of HTML tags, and sentences are extracted based on punctuation.
  3. Parsing Sentences: Each extracted sentence is parsed using a dependency parser (Minipar) to identify grammatical relationships between words.
  4. Merging Dependency Relations: Dependency relations from all sentences for a given word are merged to create a weighted directed graph, where nodes are words and edges represent dependency relations with weights indicating their frequency. This graph serves as the "context knowledge base."
  5. WSD Algorithm (TreeMatching): For a target word in a sentence, the algorithm compares the sentence's dependency parse tree with the parse trees of the word's dictionary glosses. A "semantic coherence score" is calculated by matching dependent words of the target word in the sentence with words in the glosses, and vice-versa. This matching can be based on direct dependency, backward dependency, or synonym matching. The gloss that yields the highest coherence score is selected as the correct sense.

Datasets

  • WordNet: Used as the sense inventory for words.
  • SemEval-2007 Task 7 (Coarse-grained English All-words Task) test set: This dataset, comprising five different text articles (news, book review, travelogue, computer programming, biography), was used for evaluation. The authors note that their method does not require the training data provided by SemEval.

Key Insights and Ideas

  • Dependency Knowledge as a Powerful Signal: The paper demonstrates that analyzing dependency relations between words in both the target sentence and dictionary glosses is a highly effective way to measure semantic coherence.
  • Unsupervised Approach for Broad Coverage: By leveraging readily available unannotated text and dictionaries, the method sidesteps the need for expensive manual annotation, making broad-coverage WSD feasible.
  • Brittleness Mitigation: The unsupervised nature and broad data acquisition strategy help overcome the "brittleness" problem common in supervised WSD systems.
  • Web Search as a Rich Source: Despite potential noise, web search engines provide a vast and dynamic source of contextual information for WSD.
  • Impact of Knowledge Base Size: The experiments show a positive correlation between the size of the context knowledge base (number of word instances) and disambiguation quality.

Key Results

The proposed method, named "TreeMatch," was evaluated on the SemEval-2007 Task 7 dataset.

  • It significantly outperformed the best unsupervised systems participating in SemEval-2007.
  • Its performance approached that of the top-performing supervised systems, achieving an F1 score of 73.65%.
  • The paper also investigated the impact of different matching strategies, finding that combining multiple strategies (dependency, backward, and synonym matching) generally improved disambiguation quality. Strategy 5, which combined all four matching types (dependency, backward, synonym backward, and synonym dependency), achieved the highest overall precision and recall.

Limitations and Future Work

  • Limitations:
    • The current method was tested with coarse-grained senses; its direct applicability to fine-grained WSD is mentioned but not yet evaluated.
  • Parsing is computationally expensive, although it's done offline and doesn't impact real-time disambiguation speed.
  • The quality of dictionary definitions significantly affects disambiguation quality.
  • Future Work:
    • Continue building and enlarging the knowledge base to improve coverage and performance.
  • Develop smarter and more selective matching strategies, as human WSD is often an unconscious process.
  • Apply and evaluate the system on fine-grained WSD tasks.

Summary updated

BibTeX
@inproceedings{chen2009fully,
  author = {Ping Chen and Wei Ding and Chris Bowes and David Brown},
  title = {{A Fully Unsupervised Word Sense Disambiguation Method Using Dependency Knowledge}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2009},
}
Figure