TL;DR
This survey provides a comprehensive overview of recent advancements in Word Sense Disambiguation (WSD), a crucial NLP task for understanding word meaning in context. It highlights how deep learning, particularly Transformer models, has significantly improved WSD performance, even surpassing human agreement on some benchmarks, while also discussing the limitations of current approaches and future research directions.
What Problem Does This Paper Address?
Word Sense Disambiguation (WSD) is the task of identifying the correct meaning of a word in a given context from a predefined set of senses. This problem has been a long-standing challenge in Natural Language Processing (NLP) since the early days of machine translation. Polysemy, where a single word can have multiple meanings (e.g., "bass" as a fish or a sound), is a pervasive linguistic phenomenon that hinders accurate text understanding. While traditional WSD methods existed, the field has seen a surge in performance due to recent breakthroughs in representation learning, particularly with the advent of large pre-trained language models. This survey aims to consolidate and analyze these recent trends, providing a clear picture of the state-of-the-art, the resources used, and the challenges that remain.
Methods
The paper categorizes WSD approaches into two main types:
- Knowledge-Based WSD: These methods leverage external lexical resources like WordNet or BabelNet. They often employ graph algorithms (e.g., random walks, clique approximation) to navigate the semantic relationships between word senses. Recent knowledge-based methods are increasingly integrating contextualized word representations.
- Supervised WSD: These methods learn to disambiguate senses from annotated data. The survey focuses on neural supervised systems, which have achieved the best results. Key techniques include:
- Purely Data-Driven Models: These frame WSD as a classification problem, using token taggers that map contextualized word representations to sense probabilities.
- Models Exploiting Glosses: These methods incorporate definitions (glosses) of word senses, often by encoding them into vectors and using them to enhance sense representations or as part of the classification process.
- Models Exploiting Relations: These integrate information from the semantic network structure of knowledge bases (e.g., WordNet edges) into supervised models.
- Models Exploiting Other Knowledge: This category includes approaches that leverage additional information sources like translations, images, or web search contexts.
A significant trend across many of these methods is the heavy reliance on pretrained Transformer models (like BERT) for generating contextualized word embeddings, which are then fed into various downstream WSD architectures.
Datasets
The survey discusses several key datasets:
- Sense Inventories:
- Princeton WordNet: The de facto standard, a large, manually curated lexicographic database of English.
- BabelNet: A multilingual dictionary and semantic network covering many languages, built by mapping various resources like WordNet and Wikipedia.
- Wiktionary: A collaborative dictionary project.
- Annotated Corpora:
- SemCor: The largest manually annotated dataset for English WSD, using WordNet senses.
- WordNet Gloss Corpus (WNG): Contains sense definitions and examples from WordNet, used to supplement SemCor.
- Evaluation Benchmarks:
- Senseval and SemEval datasets: Standard benchmarks for English WSD evaluation.
- FEWS: An English benchmark designed to address weaknesses in WSD approaches, particularly for infrequent senses, by using Wiktionary examples and definitions.
- XL-WSD: A recent, comprehensive multilingual benchmark that extends the English evaluation framework to 18 languages, using BabelNet senses. This is a significant step towards standardizing multilingual WSD evaluation.
Key Insights and Ideas
- Transformer Dominance: Pretrained language models (Transformers) have become indispensable, significantly boosting performance across both knowledge-based and supervised WSD approaches.
- The "Glass Ceiling" is Broken: Recent WSD models, particularly supervised ones, have achieved performance levels that meet or even exceed human inter-annotator agreement on some standard English benchmarks. This raises questions about the interpretation of these high scores and the true difficulty of the task.
- Knowledge-Based Methods Still Relevant (but different): While purely knowledge-based methods are outperformed by supervised ones on English WSD, the information within knowledge bases remains crucial. Many successful supervised methods now hybridize with knowledge-based techniques, leveraging their structure and content.
- Glosses are Gold: Incorporating word sense definitions (glosses) has proven to be highly effective in improving WSD performance, often leading to state-of-the-art results.
- Multilingual WSD is Advancing: The development of resources like BabelNet and benchmarks like XL-WSD is paving the way for more robust multilingual WSD, with cross-lingual transfer learning showing promising results.
- Beyond Discrete Senses: The inherent limitations of fixed sense inventories (granularity, inability to handle new word uses) are driving research towards tasks like Word-in-Context (WiC) and lexical substitution, which do not rely on predefined senses.
Key Results
- The best supervised WSD models, like ESCHER, achieve F1 scores around 80-83% on standard English benchmarks (e.g., ALL, S15).
- The use of pretrained language models can lead to 2-point improvements over models without them.
- Concatenating the WNG corpus to SemCor can increase system performance by 1.8 to 2.6 F1 points.
- XL-WSD results show that cross-lingual zero-shot models trained on English can outperform language-specific models trained on noisy silver data for many languages, though performance is still lower than for English.
Limitations and Future Work
- Benchmark Inadequacy: Current benchmarks may not fully capture the complexity of WSD, especially concerning domain shift and infrequent senses. New, challenging test sets are needed.
- Robustness to Domain Shift: Models need to be more robust to domain shifts and perform reliably on real-world web text (e.g., social media).
- Multilingual WSD Investigation: More comprehensive research is needed for non-English languages, including creating more training and testing data.
- Integration with Entity Linking: Combining WSD with Entity Linking (associating mentions with entities in knowledge bases) is a promising avenue, leveraging unified resources like BabelNet.
- Application of WSD: The paper suggests applying well-performing WSD systems to downstream tasks like Machine Translation, Semantic Role Labeling, and Question Answering.
- Grounding Language Models: WSD can help ground pretrained language models onto knowledge bases, providing them with richer semantics and enabling connections to other modalities like vision.
- Handling Evolving Lexicons: Future work should continue to explore methods that can handle evolving lexicons and new word uses, potentially by moving beyond fixed sense inventories.
Summary updated