Word Sense Disambiguation: Towards Interactive Context Exploitation from Both Word and Sense Perspectives

Ming Wang, Yinglin WangACL 2021

TL;DR

This paper introduces SACE, a novel Word Sense Disambiguation (WSD) architecture that tackles the problem of isolated word sense decisions by enabling interactive context exploitation. By considering relationships between senses within a document and leveraging broader context beyond a single sentence, SACE significantly improves performance, especially in low-resource scenarios like few-shot and zero-shot learning.

What Problem Does This Paper Address?

Word Sense Disambiguation (WSD) is the task of identifying the correct meaning of a word in a given context. While recent advancements using contextualized embeddings have pushed performance close to human agreement levels, existing systems often treat the disambiguation of each word in a document as an independent problem. This isolation limits their ability to fully exploit the rich contextual information available within a document, particularly the interplay between different word senses. The paper argues that the disambiguation of one word's sense can and should inform the disambiguation of others, especially when senses appear in proximity or share semantic relationships within the same document. This lack of inter-sense interaction and limited context exploitation is a key bottleneck that SACE aims to address.

Methods

SACE introduces two main innovations to enable interactive context exploitation:

  • Sense-level Context (SIC): This component enhances sense embeddings by making them aware of other senses present in the document. It uses a selective attention layer that learns to connect senses based on their co-occurrence in text, rather than relying solely on manually defined sense relations. This attention mechanism considers both the sentence-level representation and the gloss relevance of senses.
  • Word-level Context (WIC): This module broadens the context used for disambiguation beyond a single sentence. For a given sentence, SACE selects several highly related sentences from the same document (based on proximity and semantic relatedness) and uses them to enrich the context embedding of the target word. This allows the model to leverage a more comprehensive document-level understanding.

Additionally, the paper incorporates a Try-again Mechanism (TaM), which reconstructs a previous mechanism to further enhance interactivity by considering related senses based on a Coarse Sense Inventory (CSI).

Datasets

The paper uses several standard datasets for training and evaluation:

  • SemCor: A widely used dataset for training WSD models.
  • English all-words WSD benchmarks: SensEval-2 (SE2), SensEval-3 (SE3), SemEval-2007 Task-17 (SE07), SemEval-2013 (SE13), and SemEval-2015 (SE15). These are combined into an "ALL" dataset.
  • Few-shot and Zero-shot benchmarks: ALLWN_1st, ALLWN_others, ALLzss, and ALLZSL, derived from the ALL dataset to specifically evaluate performance in low-resource scenarios.
  • Cross-lingual datasets: Test datasets for Spanish, Italian, French, and German, derived from SemEval-2013 and SemEval-2015.

Key Insights and Ideas

  • Interactivity is key: The core idea is that word sense disambiguation is not an isolated task. Senses within a document interact, and exploiting these interactions significantly boosts performance.
  • Beyond sentence-level context: Limiting context to a single sentence is insufficient. Leveraging related sentences within the same document provides richer information for disambiguation.
  • Implicit sense relations are powerful: The model learns to connect senses based on their co-occurrence in text, bypassing the need for manually curated, potentially incomplete, sense relation knowledge bases.
  • Few-shot and zero-shot learning benefits greatly: The interactive approach is particularly effective when labeled data is scarce, demonstrating a strong ability to generalize to unseen senses and words.

Key Results

  • SACE achieves state-of-the-art performance on English all-words WSD benchmarks, surpassing previous methods by significant margins (e.g., 3.7% on ALL).
  • The model shows remarkable improvements in few-shot (14.3% increase) and zero-shot (35.9% increase) scenarios, outperforming strong baselines.
  • For cross-lingual WSD, SACE (specifically SACEmul) also sets new state-of-the-art results, demonstrating strong multilingual transferability.
  • Ablation studies confirm the effectiveness of each component: WIC provides the largest performance boost (1.5 F1), followed by SIC (1.2 F1) and the gloss word attention within SIC (0.5 F1).

Limitations and Future Work

  • Noise injection: The paper acknowledges that in some cases, the proposed methods might inject "excessive noise" for disambiguation, leading to incorrect predictions. This suggests that selective exploitation of context for different instances might be beneficial.
  • Future work: The authors intend to explore reinforcement learning to enhance current interactive WSD by customizing context exploitation for different instances.

Summary updated

BibTeX
@inproceedings{wang2021word,
  author = {Ming Wang and Yinglin Wang},
  title = {{Word Sense Disambiguation: Towards Interactive Context Exploitation from Both Word and Sense Perspectives}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure