Word Sense Disambiguation for Information Retrieval

87 concepts82 papers1 proposal

When you type a query into a traditional search engine, the system treats your words as literal character strings. If you search for "apple," it looks for documents containing the sequence of letters a-p-p-l-e.

This literal string matching is the foundation of classical Information Retrieval (IR), but it suffers from a fundamental linguistic flaw: words are highly ambiguous. A single word can mean entirely different things depending on its context. This phenomenon—encompassing both homonymy (unrelated meanings, like a financial "bank" versus a river "bank") and polysemy (related but distinct meanings, like "apple" the fruit versus "Apple" the company)—creates a massive bottleneck for search precision and recall.

To build search systems that truly understand user intent, researchers have developed techniques to transition from string matching to sense matching. This is the domain of Word Sense Disambiguation (WSD) in Information Retrieval.


The Failure of Surface-Form Statistics

Traditional retrieval models, such as TF-IDF or standard BM25, rely heavily on term frequency statistics across a corpus. The underlying assumption is that if a query term appears frequently in a document, that document is highly likely to be relevant.

However, as Jin Xu et al. observe in NLPR at TREC 2004: Robust Experiments, these statistical models treat highly specific terms and highly polysemous terms identically if their corpus frequencies match. A word with five distinct meanings is treated as a single, uniform signal. As a result, a query containing an ambiguous term will inadvertently retrieve documents spanning all of its possible meanings, flooding the user with irrelevant results.

This lack of semantic awareness makes it incredibly difficult for systems to evaluate how challenging a search query actually is. As demonstrated by David Carmel et al. in What makes a query difficult?, when a model relies strictly on word distributions without resolving polysemy, it cannot accurately predict query difficulty because it cannot see that a single word form is mapping to entirely unrelated concepts in the corpus.


The negative impact of lexical ambiguity compounds when systems attempt to optimize search queries through automation. Consider query expansion, a common technique where a search engine automatically adds synonyms to a user's query to catch relevant documents that use alternative phrasing (e.g., expanding "automobile" to include "car").

Without WSD, query expansion can completely derail a search. If a user searches for "crane" (meaning the bird) and the system blindly expands the query using a dictionary like WordNet, it might add terms like "hoist" or "construction equipment."

[ User Query: "crane" ]
       │
       ▼ (Blind Query Expansion)
[ Expanded Query: "crane" + "bird" + "hoist" + "construction" ]
       │
       ▼ (Result)
[ Massive Semantic Noise / Irrelevant Documents ]

In A Re-examination of Query Expansion Using Lexical Resources, Hui Fang notes that lexical query expansion frequently introduces severe noise because it fails to isolate the single, intended semantic context. In fact, experiments by Paolo Rosso in NLEL at CLEF 2009 Robust WSD Task revealed that blindly enriching a search index with synonyms from a lexical database actually degraded retrieval performance, dropping Mean Average Precision by 14–16% compared to a basic keyword baseline.


Resolving Ambiguity via Local Context

If literal strings fail us, how do we determine which "sense" of a word is intended? The answer lies in the word's immediate neighborhood. Words do not appear in isolation; their meaning is anchored by the words around them.

Trigger Words and Neighborhood Models

One powerful intuition is the concept of "trigger" words. If the word "bank" appears near "river," "water," or "sediment," the geographic sense is triggered. If it appears near "interest," "vault," or "transaction," the financial sense is triggered.

Junlin Zhang et al. operationalized this in A Trigger Language Model-based IR System by using mutual information to identify context-defining, co-occurring words for a given query. By factoring these triggered words into the document scoring process, the system can mathematically steer the search toward the correct topical domain.

Similarly, Edward Kai Fung Dang et al. introduced the Context-Dependent Relevance Model (CDRM) in A context-dependent relevance model. Instead of evaluating query terms globally, CDRM dynamically adjusts document weights based on the specific local neighborhood of words surrounding the query terms within each document, effectively filtering out passages where the word is used in the wrong context.

The Power of Local Embeddings

Modern dense retrieval often relies on word embeddings like Word2Vec or GloVe. However, standard global embeddings assign a single vector to each word string, which forces all meanings of a polysemous word to collapse into a single "average" vector.

To fix this, Bhaskar Mitra 0001 et al. proposed a clever alternative in Query Expansion with Locally-Trained Word Embeddings. Instead of using massive, globally trained models, the authors trained embedding models on a small, query-specific local corpus retrieved during an initial search phase. Because this local corpus is already constrained by the query's broader topic, the resulting embeddings naturally capture the precise, context-specific nuances of the ambiguous terms, preventing the model from drifting into irrelevant semantic domains.


Mapping Strings to Explicit Concepts

An alternative to guessing intent from local context is to explicitly map text strings to structured, unambiguous knowledge bases or ontologies (such as WordNet or DBpedia). Instead of indexing words, the system indexes concepts.

Text: "The apple fell from the tree."
        │
        ▼ (Concept Mapping / WSD)
Indexed Concepts: [ Concept_Fruit_Apple, Concept_Plant_Tree ]

In Intégration probabiliste de sens dans la représentation de textes, Romaric Besançon et al. demonstrated that document retrieval systems utilizing disambiguated concept frequencies and co-occurrences significantly outperform standard models based on raw word frequencies. By shifting the underlying representation from character strings to explicit senses, the negative impact of polysemy is fundamentally mitigated.

Taking this a step further, Susan Gauch et al. developed the KeyConcept engine, detailed in KeyConcept: Conceptual Search and Pruning Exploiting Concept Relationships. This system automatically classifies documents into a hierarchical ontology. When a user executes a search, the engine combines traditional keyword matching with conceptual similarity and hierarchical "pruning." If a keyword matches a document but the document's broader ontological classification doesn't align with the query's conceptual domain, the document is pruned. This hybrid approach yielded a massive jump in search precision—from 36.7% to 63.77%.

By forcing retrieval systems to look past the surface form of language and resolve the underlying senses of words, WSD transforms information retrieval from a game of statistical string matching into a process of genuine semantic alignment.

Key Foundational Work

Une Experience Pratique D’utilisation De L’analyse Linguistique En Recherche D’information: Bilan & Perspectives (1980) — contributed 1 concept elements to this direction.
An intelligent system for information retrieval over the internet through spoken dialogue (1997) — contributed 1 concept elements to this direction.
Homonymy and Polysemy in Information Retrieval (1997) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 79 papers
  2. Version 1 · 2026-10-04 · 75 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_lexical_ambiguity_9cb8c751,
  title = {lexical ambiguity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/lexical-ambiguity/rdr_9cb8c7515fa54657bef83246ddaf4200},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_lexical_ambiguity_9cb8c751_v2,
  title = {lexical ambiguity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/lexical-ambiguity/rdr_9cb8c7515fa54657bef83246ddaf4200/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake