Quantum Interference Model for Semantic Biases of Glosses in Word Sense Disambiguation

Junwei Zhang, Ruifang He, Fengyu Guo, Chang LiuAAAI 2024 · DOI

TL;DR

This paper introduces a novel approach to Word Sense Disambiguation (WSD) by leveraging quantum mechanics. Instead of a single representation for word senses, they propose a multi-view representation using quantum superposition to capture semantic biases across different glosses. A quantum interference model then calculates the probability of a word sense, with the interference term acting as a confidence score, leading to improved WSD performance.

What Problem Does This Paper Address?

Word Sense Disambiguation (WSD) is a crucial task in Natural Language Processing (NLP) that aims to identify the correct meaning of a word in a given context. Traditional WSD models often rely on a single representation for each word sense, typically derived from dictionary glosses. However, the authors observe that glosses for the same word sense, even from different sources, exhibit semantic biases, meaning they offer different descriptive perspectives. This inherent bias makes it challenging to accurately capture the full meaning of a word sense using a single, unified representation. Simply averaging or combining these diverse glosses can lead to a loss of information and inaccurate disambiguation.

Methods

The core innovation of this paper lies in its application of quantum mechanics to WSD.

  • Quantum Superposition for Multi-View Representation: The authors propose using a quantum superposition state to represent multiple glosses of a single word sense. This allows each gloss to contribute its unique perspective without being forced into a single representation.
  • Quantum Interference Model: To determine the probability of a target word belonging to a specific word sense (represented by a superposition state), they employ a quantum interference model. This model is inspired by the double-slit experiment in quantum physics.
  • Interference Term as Confidence: A key insight is that the "interference term" within the quantum interference model can be interpreted as a confidence level for the word sense prediction. This term quantifies the interaction between different gloss representations, providing a valuable supervisory signal.
  • BERT Integration: The model leverages pre-trained language models like BERT to obtain initial word and gloss embeddings, which are then transformed into quantum states.

Datasets

The paper evaluates its model on standard WSD benchmarks and cross-lingual datasets.

  • English All-Words WSD Evaluation Framework:
    • Training: SemCor
    • Development: SemEval-2007 (SE07)
    • Test Sets: Senseval-2 (SE2), Senseval-3 (SE3), SemEval-2013 (SE13), SemEval-2015 (SE15), and a combined ALL set. Nouns, verbs, adjectives, and adverbs are also tested separately.
  • Cross-Lingual Datasets: Datasets proposed by Pasini et al. (2021), with English glosses used as a compromise for languages where direct glosses are unavailable. The XLMR-Base model is used for comparison.

Key Insights and Ideas

  • Semantic Biases in Glosses are Real and Problematic: The paper empirically demonstrates that glosses for the same word sense are not monolithic and carry distinct perspectives, which traditional methods struggle to reconcile.
  • Quantum Superposition Naturally Handles Multi-View Data: The quantum superposition state provides a mathematically elegant way to represent and combine diverse information sources (glosses) without losing their individual characteristics.
  • Interference as a Confidence Score is Powerful: The analogy to quantum interference provides a novel way to quantify the certainty of a word sense prediction, acting as a direct supervisory signal for training.
  • Quantum Models Offer a Promising Alternative: The success of this quantum-inspired approach suggests that quantum mechanics can offer unique computational advantages for NLP tasks beyond traditional probabilistic models.

Key Results

  • Competitive Performance: The proposed model, "Our-Base" and "Our-Large" (using BERT-base and BERT-large respectively), achieves competitive performance on the standard English all-words WSD evaluation framework, outperforming several baseline systems and previous work. For instance, "Our-Large" achieved an F1-score of 81.6% on the ALL test set.
  • Ablation Study Confirms Contributions: Ablation experiments show that both the multi-view representation (compared to "Our-single") and the quantum interference term (compared to "Our-inter") significantly contribute to the performance improvement, with the interference term yielding a larger gain (around 3 percentage points).
  • Cross-Lingual Robustness: The model demonstrates good performance on cross-lingual datasets, outperforming the XLMR-Base comparison model in most languages, though performance is better for languages with character types similar to English.

Limitations and Future Work

  • Dataset Specificity: The paper notes that for datasets with a high proportion of high-frequency word senses (like SE15), a simpler, reliable single representation might be more effective than the multi-view approach, as the complexity can sometimes hinder performance on easier tasks.
  • Cross-Lingual Gloss Limitation: The use of English glosses for cross-lingual evaluation is a compromise. Future work could explore methods for obtaining or translating glosses in native languages for better cross-lingual performance.
  • Niche Languages: Performance on niche or non-alphabetic languages in cross-lingual settings was weaker, suggesting further research is needed to adapt the model to a wider linguistic diversity.
  • Generalization: The paper aims to explore the generalization performance of the model to other languages, which was partially addressed but could be further investigated.

Summary updated

BibTeX
@inproceedings{zhang2024quantum,
  author = {Junwei Zhang and Ruifang He and Fengyu Guo and Chang Liu},
  title = {{Quantum Interference Model for Semantic Biases of Glosses in Word Sense Disambiguation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i17.29927},
}
Figure