Human language is inherently ambiguous. A single word like "bank" can mean a financial institution, the side of a river, or a slope in a landscape. Word Sense Disambiguation (WSD) is the task of algorithmically determining which meaning of a word is intended within a specific context.
While supervised machine learning models achieve high accuracy on this task, they suffer from a severe knowledge acquisition bottleneck. As noted by Samuel Brody et al. in Ensemble Methods for Unsupervised WSD, it would take an estimated 27 person-years of human effort to manually annotate a corpus large enough to build a truly robust, domain-independent supervised WSD system. This heavy dependency on hand-labeled data makes it nearly impossible to scale supervised systems to new languages, specialized technical domains, or full vocabularies.
To bypass this bottleneck, a major research direction focuses on unsupervised and knowledge-based alternatives. By extracting sense distinctions directly from raw text or leveraging existing structured knowledge bases, these approaches eliminate the need for human annotation.
Word Sense Discrimination via Contextual Clustering
If we lack manual labels telling us what a word means in a sentence, how can a machine distinguish its senses? The core intuition relies on the distributional hypothesis: words used in similar contexts tend to have similar meanings. Instead of mapping a word to a predefined dictionary definition, we can group occurrences of a word into clusters based on contextual similarity—a task known as Word Sense Discrimination.
Early foundational work by Ted Pedersen et al. in Distinguishing Word Senses in Untagged Text evaluated unsupervised clustering algorithms like Expectation-Maximization (EM) to segregate word meanings without human intervention. To capture context, these systems look at immediate lexical features. For instance, Ted Pedersen demonstrated in A Decision Tree of Bigrams is an Accurate Predictor of Word Sense that simple two-word sequences (bigrams) surrounding an ambiguous word provide highly accurate signals for classification.
However, raw text is often sparse. If "bank" appears next to "river" in one sentence and "stream" in another, a simple keyword matcher might fail to see the connection. To solve this, Amruta Purandare et al. introduced a distinction between first-order and second-order contexts in Discriminating Among Word Meanings by Identifying Similar Contexts.
- First-order features look at words directly co-occurring with the target word.
- Second-order features look at the contexts of those co-occurring words.
If "river" and "stream" frequently share the same neighboring words in a large corpus, their second-order relationship allows the open-source system SenseClusters to successfully group sparse data into a shared semantic cluster. Modern iterations of this idea, such as the Watasense system proposed by Alexander Panchenko et al. in An Unsupervised Word Sense Disambiguation System for Under-Resourced Languages, scale this intuition to under-resourced languages by averaging dense word embeddings to represent sense clusters mathematically.
Graph-Based Propagation over Lexical Knowledge Bases
Purely data-driven clustering discovers sense groups, but it does not inherently link them back to an established dictionary or Lexical Knowledge Base (LKB) like WordNet. To achieve true disambiguation without labeled data, another family of algorithms leverages the structural topology of the dictionaries themselves.
The intuition here is that a dictionary is not just a list of definitions, but a massive social network of concepts. If a sentence contains the words doctor, nurse, and clinic, the correct senses of these words should sit close to one another within the LKB graph.
[Doctor: Sense 1] <---> [Clinic: Sense 1] <---> [Nurse: Sense 1]
|
v
[Doctor: Sense 2 (Ph.D.)] ----> [University]
Aitor Soroa operationalized this in Random Walks for Knowledge-Based Word Sense Disambiguation by running Personalized PageRank over the WordNet graph. By treating the words in a target text as "energy sources" that initiate a random walk, the algorithm allows probability mass to flow through the graph's edges. Senses that receive the highest steady-state probability are selected as the correct meanings because they represent the global consensus of the context.
The performance of these graph algorithms depends heavily on the density of the underlying network. To supercharge this approach, Simone Paolo Ponzetto et al. developed WordNet++ in Knowledge-Rich Word Sense Disambiguation Rivaling Supervised Systems. By automatically mapping WordNet concepts to Wikipedia articles, the authors injected millions of new semantic relations into the graph. This massive structural expansion allowed simple, unsupervised graph-based algorithms to rival the accuracy of state-of-the-art supervised models.
Lexical Substitution and Global Optimization
A more recent evolution combines pre-trained language models with structured knowledge via the intuition of lexical substitution. If you mask an ambiguous word in a sentence, a language model can predict which synonyms best fit the blank space based on context.
In Try to Substitute: An Unsupervised Chinese Word Sense Disambiguation Method Based on HowNet, Bairu Hou et al. leveraged Masked Language Models alongside the sememe structure of HowNet. By scoring how naturally a specific sense's synonymous substitutes fit into the exact context window, the system determines the correct meaning without ever seeing a single human-labeled training example.
Alternatively, instead of deciding each word's meaning in isolation, the entire sentence or document can be optimized simultaneously. Rocco Tripodi et al. framed this in A Game-Theoretic Approach to Word Sense Disambiguation as a strategic game. Each word in a text acts as a player, and its potential senses are its available strategies. Through evolutionary game theory, words iteratively adjust their chosen senses to maximize their "payoff"—defined as the semantic coherence with neighboring words. The system converges on a Nash equilibrium where all words are mutually disambiguated in harmony, proving that global structural constraints can effectively replace manual supervision.
Key Foundational Work
Distinguishing Word Senses in Untagged Text (1997) — contributed 1 concept elements to this direction.
A Decision Tree of Bigrams is an Accurate Predictor of Word Sense (2001) — contributed 1 concept elements to this direction.