Abstract
Word Sense Disambiguation (WSD) remains a fundamental challenge in Natural Language Processing, requiring models to determine the intended meaning of a word based on its context. While Large Language Models (LLMs) like GPT-4 show promise, they often falter on out-of-distribution data and rare word senses. The SANDWiCH framework addresses these limitations by reframing WSD from a single-sense selection task to a cluster discrimination problem over semantic networks. By grouping related senses and utilizing specialized models for different parts of speech, SANDWiCH establishes a new state-of-the-art across multiple benchmarks while being significantly more parameter-efficient.
Introduction
Word Sense Disambiguation is the classic NLP task of identifying which meaning of a polysemous word is intended in a specific sentence—for instance, distinguishing between "bank" as a financial institution versus a riverside. Despite the conversational prowess of modern LLMs, they frequently struggle with the "long tail" of rare definitions and out-of-distribution data. Specialized models also face hurdles, particularly with verbs and low-resource languages. The authors of SANDWiCH argue that the key to overcoming these issues lies in shifting focus from individual sense definitions to the broader semantic neighborhood of a word.
Methodology
The SANDWiCH (Semantical Analysis of Neighbours for Disambiguating Words in Context ad Hoc) framework is built upon three innovative pillars:
- Cluster Discrimination: Instead of treating senses as isolated islands, SANDWiCH leverages BabelNet to group related senses into clusters. The model is trained to identify the correct cluster, making it more robust to rare senses.
- Sense-Separability: The framework mathematically ensures these semantic clusters are disjoint by pruning edges in the semantic graph, allowing for more efficient disambiguation.
- POS-Separated Cross-Encoders: Recognizing that different parts of speech (POS) require different disambiguation strategies, the architecture uses specialized cross-encoders based on DeBERTa-v3. One encoder focuses on nouns and verbs, while another handles nouns, adjectives, and adverbs.
Experimental Results
SANDWiCH was evaluated against a rigorous battery of benchmarks, including the English All-Words WSD unified benchmark, the Rare Senses Benchmark, and the multilingual XL-WSD dataset.
State-of-the-Art Performance
As shown in the table below, SANDWiCH achieved a significant breakthrough with an 89.0 F1 score on the English all-words benchmark, a 7-point improvement over the previous state-of-the-art (CoNSeC).
Robustness and Efficiency
- Hard Cases: On the challenging hardEN dataset, where previous models often failed entirely, SANDWiCH achieved an F1 of 53.4. It also improved performance on out-of-domain data (42D) by over 20 points.
- Multilingual Mastery: The model set new records for all nine languages in the XL-WSD dataset, showing massive gains in Japanese (+22 points) and Estonian (+19 points).
- Parameter Efficiency: Remarkably, SANDWiCH outperformed the previous leader while using 72% fewer parameters (utilizing a DeBERTa-v3-small backbone), demonstrating that architectural specialization can be more effective than sheer scale.
Conclusion
SANDWiCH demonstrates that reframing WSD as a cluster discrimination problem grounded in semantic networks significantly improves robustness and efficiency. While the model currently relies on the availability of high-quality semantic graphs like BabelNet, its success across multiple languages and difficult benchmarks marks a major step forward for specialized NLP tasks. Future work will explore generating sense clusters without fixed graphs to extend these benefits to even lower-resource languages.
Summary updated