Overcoming the Knowledge Acquisition Bottleneck in Word Sense Disambiguation

42 concepts41 papers1 proposal

Word Sense Disambiguation (WSD)—the task of determining which specific meaning of a polysemous word is intended in a given context—is a cornerstone of natural language understanding. However, the field has long been constrained by what researchers call the knowledge acquisition bottleneck.

Supervised WSD models typically require vast amounts of manually sense-tagged data to achieve high accuracy. As noted in A Client/Server Architecture for Word Sense Disambiguation, Caroline Brun argues that this reliance on human annotation is prohibitively expensive and time-consuming. This bottleneck prevents models from scaling to the thousands of world languages that lack resources or to specialized domains where expert annotation is required.

To bypass this, researchers have developed strategies to acquire "silver-standard" labels or induce senses without direct human intervention.

Leveraging Cross-Lingual Divergence

One of the most intuitive solutions to the data bottleneck is to look at how other languages resolve ambiguity. A word that is ambiguous in English, like "bank," often translates to distinct, unambiguous words in another language (e.g., banco for a financial institution vs. orilla for a river bank in Spanish).

Xinglong Wang proposes using a second language as a "linguistic bridge" to automatically acquire topic signatures—sets of words that define a sense—without any manual tagging (Automatic Acquisition of English Topic Signatures Based on a Second Language). Similarly, Mitesh M. Khapra et al. demonstrate a "bilingual bootstrapping" technique where two resource-poor languages help each other learn by projecting parameters across a shared dictionary, reducing the need for annotated data by nearly half (Together We Can: Bilingual Bootstrapping for WSD).

Harvesting Synthetic Supervision from the Web

If humans cannot tag enough data, models can be trained on "found" data. Wikipedia and the broader web serve as massive, self-organizing sources of sense-annotated text.

Moving Beyond Word-Expert Models

Traditional WSD often relied on "word-experts"—individual classifiers trained for one specific word. This approach exacerbates the bottleneck because a new model must be trained for every word in the dictionary.

Recent shifts favor universal architectures. Prafulla Kumar Choubey et al. developed a single neural network that disambiguates all words by leveraging sense definitions (glosses). This allows the model to transfer knowledge from common, well-documented words to rare ones in the "long tail" of language (One Classifier for All Ambiguous Words: Overcoming Data Sparsity by Utilizing Sense Correlations Across Words).

Unsupervised Induction and Latent Knowledge

Finally, some researchers argue that explicit labels may not be necessary at all. By using Word Sense Induction (WSI), models can cluster word occurrences into groups based on context similarity. David Jurgens et al. demonstrate that combining these unsupervised clusters with a tiny amount of labeled data can outperform fully supervised systems in data-sparse scenarios (Semi-supervised Learning with Induced Word Senses for State of the Art Word Sense Disambiguation).

Even more radically, Oscar Sainz suggests that modern pre-trained language models already possess latent knowledge of word senses. By reframing WSD as a "textual entailment" problem, the author shows that models like BERT can perform WSD in a zero-shot setting, effectively bypassing the acquisition bottleneck entirely (What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories).

Key Foundational Work

A Client/Server Architecture for Word Sense Disambiguation (2000) — contributed 1 concept elements to this direction.
Word Sense Disambiguation in Untagged Text based on Term Weight Learning (1999) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_knowledge_acquisition_bottleneck_supervi_52341b4f,
  title = {knowledge acquisition bottleneck (supervised data bottleneck)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/knowledge-acquisition-bottleneck-supervised-data-bottleneck/rdr_52341b4fb4fa4fe3b0b1b80494db490f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_knowledge_acquisition_bottleneck_supervi_52341b4f_v1,
  title = {knowledge acquisition bottleneck (supervised data bottleneck)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/knowledge-acquisition-bottleneck-supervised-data-bottleneck/rdr_52341b4fb4fa4fe3b0b1b80494db490f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake