Abstract
Word Sense Disambiguation (WSD) is a fundamental task in Natural Language Processing, yet it faces a persistent challenge: the long-tail distribution of word senses. Most existing models excel at identifying common "Head Senses" but struggle with rare "Long-Tail Senses" (LTS) due to a lack of training data. This paper presents QR-WSD, a novel approach that utilizes quantum-inspired mathematical frameworks to enhance sense representations. By mapping embeddings into a complex Hilbert space and employing superposition states, QR-WSD provides a more robust mechanism for learning from sparse data. Our model achieves state-of-the-art results on standard WSD benchmarks and demonstrates a significant performance boost specifically for rare word senses.
Introduction
The task of Word Sense Disambiguation (WSD) involves determining the specific meaning of a polysemous word based on its context. For instance, the word "plant" could refer to a living organism or a manufacturing facility. While modern deep learning models like BERT have advanced WSD, they remain data-hungry. In natural language, sense frequency follows a Zipfian distribution: a few senses appear frequently, while the vast majority are rare. This "long-tail" problem means that for many senses, there are insufficient training examples to learn high-quality vector representations in standard Euclidean space.
Methodology
The researchers developed QR-WSD (Quantum-Inspired Representation for WSD) to address these limitations. The architecture bridges classical deep learning with quantum-inspired processing.
Hilbert Space and Constraints
Unlike traditional models that use unconstrained Euclidean space, QR-WSD maps word and sense embeddings into a Hilbert space. Specifically, vectors are normalized to lie on the surface of a unit sphere (a Bloch sphere representation). This geometric constraint limits the parameter space, which helps the model generalize better when training data is scarce.
Superposition and Refinement
A core innovation is the use of superposition states. A sense representation is constructed as a combination of its initial state (derived from BERT) and an "anti-state" produced by a quantum NOT gate. A learnable parameter, $\theta$, controls the weight of this superposition. Theoretically, this allows the model to "search" the neighborhood of a suboptimal representation and shift it toward the "true" intended state during the training process.
Hybrid Processing
The model utilizes a dual-scoring mechanism. It calculates a classical score using the inner product of BERT embeddings for the target word, its glosses, and example sentences. Simultaneously, it calculates a quantum score using quantum measurement theory. These scores are fused to produce the final sense prediction.
Experimental Setup
The model was trained on the SemCor corpus and the WordNet Gloss Tagged (WNGT) dataset. Evaluation was conducted across several standard benchmarks, including Senseval-2, Senseval-3, and SemEval-07/13/15. To specifically measure performance on rare senses, the authors created dedicated "Most Frequent Sense" (MFS) and "Long-Tail Sense" (LTS) evaluation subsets.
Results and Discussion
QR-WSD achieved state-of-the-art performance, reaching an F1-score of 80.5% on the unified "ALL" test set, surpassing previous leading models such as BEM and EWISER. When augmented with WNGT data, the performance further improved to 81.0%.
The most significant finding lies in the model's ability to handle rare senses. In ablation studies, the version of the model utilizing the full quantum-inspired framework (QR-WSD+) significantly outperformed a version without it (QR-WSD-) on the LTS dataset. Specifically, the F1-score for rare senses jumped from 66.8% to 70.7%. This confirms that the Hilbert space constraints and superposition mechanism are particularly effective for senses with few training examples.
Furthermore, the model demonstrated cross-lingual robustness. Tested across 18 languages, QR-WSD consistently outperformed the XLMR-Base baseline, indicating that the benefits of quantum-inspired representations are universal across different linguistic structures.
Conclusion
QR-WSD introduces a mathematically principled way to tackle the long-tail problem in WSD. By moving beyond Euclidean space and adopting quantum-inspired concepts like superposition, the model achieves superior generalization for rare word senses. This work opens new avenues for applying quantum-inspired methods to other few-shot learning tasks in natural language processing, such as entity linking or relation extraction. Future research will investigate whether more advanced quantum concepts, such as entanglement, can further capture the complex dependencies between words in a sentence.
Summary updated