Unsupervised Domain Relevance Estimation for Word Sense Disambiguation

Alfio Massimiliano Gliozzo, Bernardo Magnini, Carlo StrapparavaEMNLP 2004

Introduction

Word Sense Disambiguation (WSD) remains a cornerstone of Natural Language Processing, tasked with identifying the correct meaning of a word based on its context. A powerful signal for WSD is the topical domain of the text. For instance, in a financial context, the word "bank" almost certainly refers to a financial institution rather than a riverbank. However, automatically identifying these domains is complicated by the very problem it seeks to solve: polysemy.

The Challenge of Polysemous Noise

Simple frequency-based domain detection often fails because common words appear across many contexts. A word like "check" might appear in a sports report but carry a financial sense, creating statistical "noise." Most state-of-the-art systems mitigate this using large, hand-labeled datasets, which are costly and difficult to scale across different languages or specialized fields.

Methodology: Domain Relevance Estimation (DRE)

This paper proposes Domain Relevance Estimation (DRE), a fully unsupervised approach to filter out polysemous noise. The core innovation is modeling the frequency of domain-specific words as a mixture of two distributions using a Gaussian Mixture Model (GMM):

  1. Noise Distribution: Represents occurrences of domain words in non-relevant texts due to chance or ambiguity.
  2. Relevant Distribution: Represents occurrences in texts truly belonging to the target domain.
Figure 1: Density functions for relevant and non-relevant distributions within a domain.

By applying the Expectation-Maximization (EM) algorithm to a large, unlabeled corpus like the British National Corpus (BNC), the system learns to statistically distinguish between a genuine topical signal and background noise. This relevance score is then integrated into a Domain Driven Disambiguation (DDD) framework, which uses cosine similarity between context vectors and sense vectors to select the most likely meaning.

Experimental Setup

The authors utilized WORDNET DOMAINS to map word senses to approximately 40 broad categories. The system was evaluated on the Senseval-2 All-Words Task, a gold-standard benchmark for WSD. To estimate prior probabilities of word senses, the authors used SemCor, a subset of the Brown Corpus.

Key Results

The DRE-based system demonstrated a significant performance leap over previous iterations. By effectively filtering noise, the "new" DDD system achieved 79% precision at 35% recall.

Figure 2: Precision-Recall comparison between the new DRE-based system and the previous baseline.

This represents a 4-point improvement over the "old" DDD baseline, which relied on simple frequency counts. While the system shows high precision, the recall is lower because domain information is most effective for specialized terminology rather than generic functional language.

Insights and Limitations

A key finding of the study is the disparity between parts of speech. Nouns are highly "domain-oriented" (e.g., "scalpel" in MEDICINE), whereas verbs like "run" are used ubiquitously, making them harder to disambiguate through domain signals alone.

Furthermore, the system faces the "Factotum" problem—many WordNet senses are labeled as generic, providing no strong domain signal. Future work suggests that combining DRE with syntactic features or increasing the granularity of domain categories could further refine performance.

Summary updated

BibTeX
@inproceedings{gliozzo2004unsupervised,
  author = {Alfio Massimiliano Gliozzo and Bernardo Magnini and Carlo Strapparava},
  title = {{Unsupervised Domain Relevance Estimation for Word Sense Disambiguation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2004},
}
Figure