Word Sense Disambiguation for Information Retrieval

Özlem Uzuner, Boris Katz, Deniz YuretAAAI 1999 · OpenReview

Abstract

Despite their increasing importance as data retrieval tools, most Information Retrieval (IR) systems are deficient in precision and recall. Lack of disambiguation power is one reason for the poor performance of these systems. Correctly disambiguating and expanding a query with intended synonyms before retrieval may improve the performance. We use the local context of a word to identify its sense. In our case, the local context of a word is the ordered list of words from the closest content word on each side of the target word up to the target word which is expressed as a placeholder.

Introduction

The core challenge in Information Retrieval (IR) is that language is inherently ambiguous. A single word can have multiple meanings, a phenomenon known as polysemy. For instance, the word "charge" could refer to a legal accusation, a battery's energy, or a financial bill. Most IR systems treat words as simple character strings, leading to "noisy" results where a user searching for legal "charges" might receive documents about "charging" a laptop.

The authors aim to bridge this gap by correctly identifying the intended sense of a word within a query and expanding that query with relevant synonyms. The goal is to increase both precision (the relevance of results) and recall (the completeness of results).

Methods: The Selector Algorithm

The authors introduce a Word Sense Disambiguation (WSD) algorithm based on the concept of local context and selectors. The process follows these steps:

  1. Defining Context: The local context of a target word is defined as the ordered list of surrounding content words. For example, in the phrase "...the jury had been charged to investigate...", the left context is "jury had been X" and the right context is "X to investigate."
  2. Extracting Selectors: Using a large corpus (such as Associated Press articles), the system identifies other words that appear in the exact same context. If words like "appointed" or "assigned" frequently appear in the "X to investigate" slot, these are labeled as "selectors."
  3. WordNet Mapping: The algorithm compares these selectors against the various "synsets" (synonym sets) provided by WordNet. The sense that shares the most overlap with the extracted selectors is chosen as the intended meaning.
  4. Query Expansion: Once a sense is identified, the system expands the original IR query by adding synonyms from the chosen WordNet synset.
The process of extracting selectors from local context for the word 'charged' in a legal context.

Key Insights and Ideas

Distributional Similarity

The "selector" method is an early implementation of the distributional hypothesis: "you shall know a word by the company it keeps." By identifying which words are interchangeable in a specific sentence structure, the system can infer meaning without requiring deep linguistic parsing.

WordNet Granularity and the Baseline Hurdle

A significant challenge in WSD is the granularity of dictionaries. WordNet often provides senses that are too fine-grained for the needs of IR, making distinctions that are too subtle to impact retrieval performance significantly.

WordNet senses for the word 'charge', illustrating the fine-grained nature of sense definitions that the algorithm must navigate.

Furthermore, the "most frequent sense" heuristic remains a powerful baseline. Simply guessing the most common meaning of a word (Sense 1 in WordNet) often outperforms complex algorithms.

Key Results

The study evaluated the algorithm using the Semcor dataset for WSD accuracy and the SMART system on benchmark collections like CACM, CISI, and CRAN for IR performance.

  • WSD Accuracy: The proposed algorithm achieved 45% accuracy on Semcor. However, it failed to beat the "most frequent sense" baseline, which achieved 54% accuracy.
  • IR Performance: When integrated into the SMART retrieval system, the results were disappointing. Whether the system expanded the query with multiple synonyms or replaced words with single synonyms, the overall IR performance decreased compared to the baseline keyword search.

Limitations and Conclusion

The authors identify low WSD accuracy as the primary bottleneck. If disambiguation is even slightly incorrect, it introduces "garbage" synonyms into the query, which hurts performance more than the original ambiguity. The paper concludes that until disambiguation becomes significantly more reliable, its deleterious effects will continue to outweigh its benefits in Information Retrieval. Future work may involve using less granular dictionaries or weighting selectors based on their informativeness.

Summary updated

Abstract

Word Sense Disambiguation for Information Retrieval Ozlem Uzuner, Boris Katz, and Deniz Yuret, MIT Artificial Intelligence Laboratory Despite their increasing importance as data retrieval tools, most Information Retrieval (IR) systems are deficient in precision and recall. Lack of disambiguation power is one reason for the poor performance of these systems. Correctly disambiguating and expanding a query with intended synonyms before retrieval may improve the performance. We use the local context of a word to identify its sense. In our case, the local context of a word is the ordered list of words from the closest content word on each side of the target word up to the target word which is expressed as a placeholder. This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.
BibTeX
@inproceedings{uzuner1999word,
  author = {Özlem Uzuner and Boris Katz and Deniz Yuret},
  title = {{Word Sense Disambiguation for Information Retrieval}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1999},
  url = {https://openreview.net/forum?id=ryW3Hlbdbr},
}
Figure