An Information Retrieval Approach to Sense Ranking

Mirella Lapata, Frank KellerNAACL 2007 · OpenReview

Abstract: Mirella Lapata, Frank Keller. Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference.

2007.

Introduction

Word Sense Disambiguation (WSD) is the task of determining the correct meaning of an ambiguous word in context. One of the most effective strategies for WSD is the "first sense heuristic," which involves simply predicting the most frequent sense of a word. While powerful, this heuristic typically requires large amounts of hand-labeled data to determine sense frequencies, creating an "annotation bottleneck." Furthermore, sense distributions are domain-dependent; the dominant sense of a word like "court" differs significantly between legal and sports contexts.

This paper introduces a "knowledge-lean" method to automatically induce sense rankings from raw, unlabeled text. By leveraging Information Retrieval (IR) techniques and the distributional hypothesis—that words with similar meanings appear in similar contexts—the authors develop a system that identifies dominant senses without human annotation or complex linguistic parsing.

Methodology

The proposed approach relies on WordNet to identify sense inventories and associated synonyms (synsets). The core mechanism involves treating a large corpus, such as the British National Corpus (BNC), as a searchable database. To estimate the frequency of a specific sense, the system queries an IR engine to find how often a target word co-occurs with its synonyms at the document level.

The researchers evaluated 12 model variations based on three key parameters:

  1. Aggregation: Determining whether to Sum synonym frequencies, take the Average, or use the Highest frequency synonym as the sense representative.
  2. Hypernyms: Testing whether including broader category terms (e.g., "furniture" for "chair") improves the signal or introduces noise.
  3. Normalization: Adjusting counts based on the global frequency of synonyms to ensure common words do not disproportionately bias the results.
Table 2: Parameter tuning on Senseval-2 showing the impact of normalization and hypernyms. The best configuration (High, +Norm, -Hyp) is highlighted.

Key Results and Insights

The experimental results demonstrate that document-level co-occurrence is a remarkably robust signal for sense ranking. The best-performing configuration—utilizing the Highest frequency synonym, Normalization, and No hypernyms—achieved a precision of 59.7% on the Senseval-2 benchmark.

Several critical insights emerged from the study:

  • Normalization is Essential: Raw co-occurrence counts are often deceptive. Adjusting for the global frequency of synonyms is necessary to isolate the specific relationship between a word and its intended sense.
  • Hypernyms Add Noise: Including broader terms generally degraded performance, as hypernyms are often shared across multiple senses, muddying the distinction between specific meanings.
  • Competitive Performance: The IR-based model significantly outperformed previous automated methods, such as the dependency-parser-based model by McCarthy et al. (2004), while being computationally simpler and easier to adapt to new languages.
Table 5: Comparison of the IR-Model against other state-of-the-art sense ranking systems on Senseval-3 data.

Beyond benchmark performance, the model's rankings showed a significant correlation ($\rho = 0.384$) with human "free association" data (Durkin and Manning, 1989). This suggests the model captures genuine human intuitions regarding word prominence rather than just statistical artifacts of the IR engine.

Discussion and Future Work

While the IR approach is highly effective, it remains "tethered" to predefined sense inventories like WordNet and cannot discover entirely new meanings. Additionally, the granularity of the sense inventory can impact results; extremely fine-grained distinctions may weaken the co-occurrence signal.

Future research could explore more sophisticated association measures, such as Pointwise Mutual Information (PMI), or test the model's efficacy in specialized domains (e.g., medical or technical corpora) where dominant senses shift dramatically. Nonetheless, this work proves that a simple, knowledge-lean IR approach can match or exceed the performance of much more complex, resource-intensive models in the task of sense ranking.

Summary updated

Abstract

Mirella Lapata, Frank Keller. Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference. 2007.
BibTeX
@inproceedings{lapata2007information,
  author = {Mirella Lapata and Frank Keller},
  title = {{An Information Retrieval Approach to Sense Ranking}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2007},
  url = {https://openreview.net/forum?id=B1VUHQ-dWS},
}
Figure