The Knowledge Acquisition Bottleneck Problem in Multilingual Word Sense Disambiguation

Tommaso PasiniIJCAI 2020 · DOI

TL;DR

This paper surveys the challenges and advancements in Word Sense Disambiguation (WSD), particularly focusing on the "knowledge acquisition bottleneck" – the difficulty of creating large, manually annotated datasets for different languages. It reviews various approaches to overcome this, from manual annotation to sophisticated automatic methods, and discusses the evolution of supervised WSD models, highlighting the impact of pre-trained language models.

What Problem Does This Paper Address?

Word Sense Disambiguation (WSD) is a fundamental task in Natural Language Processing (NLP) that aims to identify the correct meaning of a word in a given context. For example, the word "bank" can refer to a financial institution or the side of a river. Accurately disambiguating word senses is crucial for many downstream NLP applications. However, the most effective WSD models, known as supervised models, require vast amounts of sense-annotated data (text where words are manually labeled with their correct meaning). Creating these datasets is a labor-intensive and expensive process, especially across multiple languages, leading to the "knowledge acquisition bottleneck." This paper addresses this challenge by surveying existing methods to alleviate this bottleneck and providing an overview of supervised WSD models that benefit from these efforts.

Methods

The paper categorizes WSD approaches into three main types:

  • Knowledge-based: These methods leverage external knowledge resources like semantic networks (e.g., WordNet, BabelNet) without requiring sense-annotated data.
  • Unsupervised (Word Sense Induction): These approaches do not rely on pre-defined sense inventories or annotated data, instead using clustering techniques to group words with similar meanings.
  • Supervised: These models require sense-annotated corpora for training. The paper details advancements in supervised WSD, moving from traditional feature-based models (like Support Vector Machines) to neural-based models. A significant development has been the integration of pre-trained language models (like BERT) and contextualized word embeddings, which have dramatically improved performance. The paper also discusses how knowledge bases are increasingly being integrated into neural models.

Datasets

The paper discusses several datasets and resources crucial for WSD:

  • SemCor: A foundational sense-annotated corpus for English, derived from the Brown corpus and annotated with WordNet senses.
  • WordNet: A widely used lexical database that provides synsets (sets of synonyms representing a concept) and glosses (definitions).
  • BabelNet: A large multilingual lexicalized semantic network that integrates various resources like WordNet and Wikipedia.
  • Parallel Corpora: Datasets containing texts in multiple languages that are translations of each other. These are leveraged for semi-automatic and automatic creation of sense-annotated data.
  • Wikipedia: Used as a source for creating large-scale, though sometimes biased, sense-annotated corpora through hyperlink analysis.
  • Newer Datasets/Approaches: The paper highlights efforts to create multilingual sense-annotated corpora through automatic methods, such as OMSTI, EuroSense, SenseDefs, Train-O-Matic, and OneSeC, which aim to mitigate the knowledge acquisition bottleneck.

Key Insights and Ideas

  • The "Knowledge Acquisition Bottleneck" is Central: The paper emphasizes that the primary hurdle in WSD, especially for multilingual applications, is the immense cost and effort required to create sense-annotated data.
  • Evolution of WSD Models: There's a clear progression from early dictionary-based and feature-based methods to sophisticated neural networks.
  • Pre-trained Language Models Revolutionized WSD: Models like BERT have significantly boosted WSD performance by effectively capturing contextual word meanings, leading to state-of-the-art results.
  • Multilingual WSD is Gaining Traction: While English has been the primary focus, there's a growing effort to develop WSD systems for other languages, driven by the creation of multilingual resources and models.
  • Knowledge Integration is Key: Combining external knowledge (from knowledge bases) with learned representations from neural models often yields superior results.
  • Sense Granularity is a Persistent Challenge: WordNet's fine-grained senses can be problematic, and the optimal level of granularity for senses is still an open question, potentially depending on the downstream application.

Key Results

  • Supervised WSD models, particularly those leveraging pre-trained language models and knowledge graphs, have surpassed the 80% accuracy ceiling on English all-words WSD benchmarks.
  • Models like GlossBERT and EWISER have shown strong performance by incorporating gloss knowledge and knowledge base relations, respectively.
  • Zero-shot cross-lingual WSD models, enabled by multilingual pre-trained language models, can achieve performance comparable to language-specific models, opening new avenues for multilingual NLP.
  • Automatic corpus creation methods like Train-O-Matic and OneSeC have demonstrated the ability to generate large-scale, high-quality sense-annotated data for multiple languages, though often limited to nominal instances.

Limitations and Future Work

Limitations:

  • Data Scarcity for Low-Resource Languages: Despite advancements, a significant lack of large-scale, high-quality sense-annotated datasets exists for many languages.
  • Bias in Automatically Generated Corpora: Approaches relying on sources like Wikipedia may introduce biases towards certain topics or lack coverage of less common senses.
  • Limited Scope of Some Automatic Methods: Some automatic annotation methods are restricted to specific word types (e.g., nouns) or specific senses present in existing corpora.

Future Work:

  • Active Learning: Coupling knowledge-based approaches with active learning to efficiently gather human annotations and reduce costs.
  • Multilingual Gold Standards: Developing large-scale, multilingual gold standard datasets to better evaluate and advance WSD systems across diverse languages.
  • Sense-Annotating as a Game: Exploring reinforcement learning paradigms for WSD, potentially formalizing the problem as an agent-environment interaction.

Summary updated

BibTeX
@inproceedings{pasini2020knowledge,
  author = {Tommaso Pasini},
  title = {{The Knowledge Acquisition Bottleneck Problem in Multilingual Word Sense Disambiguation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/687},
}
Figure