Overcoming the Supervision Bottleneck in Word Sense Disambiguation

This direction is split into other directions: supervised data dependency (knowledge acquisition bottleneck), knowledge acquisition bottleneck (supervised data bottleneck). Its last version stays readable and citable here.

Version 1 of 1243 papers274 conceptsfrozen 2026-10-04

Word Sense Disambiguation (WSD)—the task of determining which "sense" of a word is meant in a given context—is a cornerstone of natural language understanding. However, the field has long been haunted by the "knowledge acquisition bottleneck." As noted by researchers like Ted Pedersen and Sanae Fujita, supervised models require massive, manually sense-tagged corpora that are prohibitively expensive to produce and rarely exist for specialized domains or low-resource languages.

To solve this, researchers have moved toward unsupervised and semi-supervised strategies that treat language itself, or structured knowledge bases, as a source of "free" supervision.

Dictionaries as a Semantic Bridge

One of the most intuitive ways to bypass manual labeling is to use the resources we already have: dictionaries. A dictionary definition (or "gloss") provides a textual bridge between a word and its meaning.

Felix Hill, Kyunghyun Cho, and colleagues demonstrated in Learning to Understand Phrases by Embedding the Dictionary that neural networks can learn to map multi-word definitions directly to the vector representation of the word they define. This allows a model to "understand" a concept by looking at its components. Similarly, the EWISE model by Sawan Kumar and Partha Talukdar in Zero-shot Word Sense Disambiguation using Sense Definition Embeddings maps senses into a continuous embedding space. By leveraging these glosses, models can perform "zero-shot" WSD, identifying meanings they never saw during training because they can "read" the definition of a new sense and compare it to the current context.

Discovery through Distributional Clustering

If we don't have labels, we can look at the company a word keeps. The intuition is that a word used in a specific sense will consistently appear near a specific set of "expert" context words.

Early work by Ted Pedersen and Amruta Purandare with SenseClusters showed that by clustering similar contexts in raw, untagged text, we can discriminate between senses without any human intervention. Jordan Lee Boyd-Graber and David Blei extended this logic to topic modeling in PUTOP, arguing that a word's sense is often a function of the document's global topic. By making sense selection sensitive to these latent topics, they moved beyond the "most frequent sense" (MFS) baseline, which often fails in specialized domains.

Leveraging Multilingual and Visual Signals

When one language lacks data, another language—or even another medium—can provide the necessary signal. This is known as "bootstrapping."

In Together We Can: Bilingual Bootstrapping for WSD, Mitesh M. Khapra and Pushpak Bhattacharyya showed that two resource-poor languages can help each other learn by projecting parameters across a shared dictionary. Similarly, Multilingual Word Sense Disambiguation Using Wikipedia treats Wikipedia’s cross-lingual links as a massive, self-organizing source of sense-annotated data.

Even non-textual data can help. Sanja Fidler and Suzanne Stevenson demonstrated in Unsupervised Disambiguation of Image Captions that visual features in an image can clarify ambiguous words in its caption. If a caption mentions a "bank," the presence of water pixels in the image provides an unsupervised cue that we are discussing a riverbank, not a financial institution.

Unlocking Pre-trained Representations

The advent of contextualized embeddings like BERT changed the landscape, but the supervision bottleneck remained: how do we map these high-dimensional vectors to specific dictionary senses?

Christian Hadiwinoto and Hwee Tou Ng showed in Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations that linear projection strategies can bridge this gap. More recently, Fangyu Liu, Ivan Vulić, and others developed MirrorWiC, a fully unsupervised method that uses contrastive learning on raw text to "unlock" the sense-discriminative power already latent within BERT. By forcing the model to recognize when the same word is used in different contexts, it learns to represent senses distinctly without ever seeing a human-labeled example.

Synthetic Supervision and Weak Labels

Finally, when real data is scarce, we can manufacture it. For tasks like Chinese Spelling Check, Dingmin Wang and Jialong Han showed in A Hybrid Approach to Automatic Corpus Generation that simulating OCR and ASR errors can create a massive synthetic corpus. This "weak supervision" allows deep learning models to reach the scale they need to outperform traditional unsupervised methods, effectively trading human effort for computational simulation.

Papers in this version

Showing 60 of 243 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_overcoming_the_supervision_bottleneck_in_a0608d6c,
  title = {Overcoming the Supervision Bottleneck in Word Sense Disambiguation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/overcoming-the-supervision-bottleneck-in-word-sense-disambiguation/rdr_a0608d6c7cac4e5c9a3c2d328b739257},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_overcoming_the_supervision_bottleneck_in_a0608d6c_v1,
  title = {Overcoming the Supervision Bottleneck in Word Sense Disambiguation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/overcoming-the-supervision-bottleneck-in-word-sense-disambiguation/rdr_a0608d6c7cac4e5c9a3c2d328b739257/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}