Contextual Word Sense Disambiguation

This direction is merged into another direction: word sense disambiguation. Its last version stays readable and citable here.

Version 1 of 1135 papers139 conceptsfrozen 2026-10-04

Natural language is inherently messy because words are rarely monosemous. A single surface form—like "bank," "interest," or "cell"—acts as a container for multiple, often unrelated, concepts. This phenomenon, known as polysemy, creates a fundamental bottleneck for machines: how can a system determine which specific "sense" of a word is intended in a given sentence?

The research direction of Word Sense Disambiguation (WSD) seeks to bridge this gap by treating word meaning not as a static dictionary entry, but as a dynamic property derived from surrounding context.

From Rules to Word Experts

Early attempts at solving ambiguity relied on "Good Old Fashioned AI" (GOFAI), where linguistic knowledge was encoded into rigid frames or semantic networks. Philip J. Hayes argued in On Semantic Nets, Frames and Associations that systems must identify the correct "conceptual frame" of a discourse to avoid misinterpretation.

A more radical early intuition came from Chuck Rieger and Steven L. Small in their work on the Word Expert Parser (WEP). Instead of a central controller applying rules, they envisioned every word as an "active program" or "expert." When a sentence is read, these word-experts "negotiate" with their neighbors to resolve ambiguity. If you see the word "suit" next to "cards," the "suit-expert" suppresses the "clothing" meaning and promotes the "playing cards" meaning.

The Power of Local Constraints

In the 1990s, the field shifted from manual rules to statistical bootstrapping. David Yarowsky revolutionized WSD with two simple but powerful observations in Unsupervised Word Sense Disambiguation Rivaling Supervised Methods:

  1. One sense per collocation: A word’s meaning is strongly constrained by its immediate neighbors (e.g., "clerical" near "error" vs. "clerical" near "vestments").
  2. One sense per document: If a word appears multiple times in a single article, it almost always maintains the same sense throughout.

By leveraging these constraints, Yarowsky showed that models could "self-train" on massive amounts of unlabelled text, reaching 96% accuracy without the need for expensive, hand-tagged datasets. Similarly, Eneko Agirre and German Rigau used the "Conceptual Density" of WordNet hierarchies to let context words "vote" on the most likely meaning based on how closely they clustered in a semantic tree (Word Sense Disambiguation using Conceptual Density).

Contextualized Representations and Definition Modeling

The modern era of WSD has moved away from picking a sense from a fixed list (like a dictionary) and toward generating meaning on the fly. Christian Hadiwinoto and colleagues demonstrated that pre-trained models like BERT can shatter previous records by using "contextualized word representations"—vectors that change based on the words around them (Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations).

This led to the rise of definition modeling, where a model is asked to write a dictionary definition for a word based on its use in a specific sentence. Timothee Mickus and others showed in Mark my Word that the way you "mark" the target word for the model—essentially how you tell the model which word to focus on—is critical for capturing the specific nuance of that usage.

Why Disambiguation Matters

WSD is not just a theoretical exercise; it is the "glue" for many downstream tasks:

  • Paraphrasing: To replace a word with a synonym, you must first know its sense. Colin J. Bannard used bilingual data as a "pivot" to find paraphrases, noting that failing to disambiguate leads to semantically incorrect substitutions (Paraphrasing with Bilingual Parallel Corpora).
  • Semantic Expansion: When searching for "apple," a system needs to know if you want "fruit" or "technology." Aixin Sun introduced CASE (Context-Aware Semantic Expansion) to suggest relevant terms based on the specific sentence context (CASE: Context-Aware Semantic Expansion).
  • Language Learning: For ESL learners, static dictionaries are overwhelming. Mahmoud Azab and Rada Mihalcea developed tools to provide the single correct definition for a word in its current context, reducing the cognitive load on the reader (Using Word Semantics To Assist English as a Second Language Learners).

The Context Gap

Despite these advances, challenges remain. Seunghyun Yoon identified a "context gap" in modern models, showing that while they are good at finding phrases, they still struggle to distinguish between different meanings of the same phrase in different situations (PiC: A Phrase-in-Context Dataset). Whether it is resolving regional slang in Nigerian tweets (Taiwo Kolajo) or disambiguating homonyms in ancient Sanskrit (Sebastian Nehrdich), the core mission remains the same: transforming ambiguous surface forms into precise, context-aware concepts.

Papers in this version

Showing 60 of 135 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_contextual_word_sense_disambiguation_045c5e3f,
  title = {Contextual Word Sense Disambiguation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/contextual-word-sense-disambiguation/rdr_045c5e3f2a954d349aa71ed57176e063},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_contextual_word_sense_disambiguation_045c5e3f_v1,
  title = {Contextual Word Sense Disambiguation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/contextual-word-sense-disambiguation/rdr_045c5e3f2a954d349aa71ed57176e063/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}