AMuSE-WSD: An All-in-one Multilingual System for Easy Word Sense Disambiguation

Riccardo Orlando, Simone Conia, Fabrizio Brignone, Francesco Cecconi, Roberto NavigliEMNLP 2021

TL;DR

This paper introduces AMuSE-WSD, a novel end-to-end system that simplifies Word Sense Disambiguation (WSD) by offering high-quality sense annotations for 40 languages through a state-of-the-art neural model. It addresses the gap of complex, research-oriented WSD systems by providing a user-friendly, ready-to-use package with a RESTful API and a web interface, making advanced WSD accessible for broader applications.

What Problem Does This Paper Address?

Word Sense Disambiguation (WSD) is a crucial task in Natural Language Processing (NLP) that aims to identify the correct meaning of a word in its given context. For instance, the word "bank" can refer to a financial institution or the side of a river. While recent deep learning models have significantly advanced WSD performance, these systems are often not readily available as easy-to-use packages. Researchers and developers face challenges due to complex implementation requirements, the need for extensive preprocessing (like sentence splitting, tokenization, lemmatization, and part-of-speech tagging), and the lack of integrated, multilingual solutions. Existing ready-to-use systems are often limited to English or rely on older, less performant methods. This paper aims to bridge this gap by providing a comprehensive, multilingual, and easy-to-integrate WSD system.

Methods

AMuSE-WSD is built upon a reimplementation of a state-of-the-art neural WSD model, specifically the one proposed by Conia and Navigli (2021). This model leverages a Transformer encoder architecture. For a given word in context, it first generates a contextualized representation using the average of hidden states from the last four layers of a pretrained Transformer. This representation is then transformed into a sense-specific hidden representation, which is finally used to compute a probability distribution over all possible senses in a given sense inventory.

A key aspect of AMuSE-WSD is its integrated preprocessing pipeline. It utilizes popular toolkits like spaCy and Stanza to handle document splitting, tokenization, lemmatization, and part-of-speech tagging. The system intelligently selects the appropriate preprocessing model based on the language, prioritizing spaCy for speed and falling back to Stanza for languages where spaCy's performance is suboptimal.

The system is made accessible through a RESTful API, allowing users to query it online or host it offline. It also features a user-friendly web interface for quick disambiguation without any coding. For offline use, AMuSE-WSD is distributed as Docker images, optimized for various configurations and hardware (CPU/GPU), with inference speedups achieved through ONNX.

Datasets

The paper evaluates AMuSE-WSD on several standard English WSD datasets: Senseval-2 (SE2), Senseval-3 (SE3), SemEval-2007 (SE07), SemEval-2013 (SE13), and SemEval-2015 (SE15). For multilingual WSD, it uses datasets from SemEval-2013 (French, German, Italian, Spanish) and SemEval-2015 (Italian, Spanish). Additionally, it is evaluated on the XL-WSD dataset, a new multilingual benchmark comprising 17 languages.

For training, the system uses the union of SemCor and the WordNet Gloss Corpus.

Key Insights and Ideas

  • Bridging the Research-Implementation Gap: The most significant insight is the successful creation of a high-quality, end-to-end WSD system that is genuinely easy to use. This democratizes access to advanced WSD capabilities, moving beyond research labs.
  • Multilingualism as a Core Feature: Unlike many previous systems, AMuSE-WSD is designed from the ground up for multilingual support, offering a significant advantage for global NLP applications.
  • Integrated Preprocessing is Crucial: The paper highlights that robust WSD requires more than just a sophisticated model; a seamless preprocessing pipeline is essential for practical usability, especially when dealing with raw text.
  • Performance and Accessibility Trade-offs: The availability of different model configurations (large, medium, small) and deployment options (API, Docker, ONNX optimization) allows users to balance performance needs with hardware constraints and inference speed requirements.
  • Multi-label Classification for Ambiguity: Framing WSD as a multi-label classification problem acknowledges the inherent ambiguity in language and allows models to assign multiple valid senses when boundaries are unclear.

Key Results

AMuSE-WSD demonstrates state-of-the-art performance, significantly outperforming previous end-to-end WSD systems. For example, the AMuSE-WSD (XLM-R large) configuration achieved a +7.6 F1 score improvement on the concatenation of English test sets compared to the previous best end-to-end system, SyntagRank. Multilingually, it showed improvements of +6.8, +6.8, and +9.6 on SemEval-2013, SemEval-2015, and XL-WSD, respectively.

Even the lighter "amuse-small-multilingual" configuration, based on MiniLM, showed substantial gains over SyntagRank. The system also offers significant inference speedups, with ONNX optimization on CPU making it up to 2.5 times faster and 1.2 times smaller than its vanilla PyTorch implementation.

Limitations and Future Work

The paper acknowledges that while AMuSE-WSD offers a significant step forward, it is still based on the underlying pretrained language models and sense inventories. Future work could involve exploring more advanced neural architectures, incorporating richer knowledge sources, or further optimizing inference for even more demanding real-time applications. The authors also express hope that AMuSE-WSD will encourage further research in lexical semantics and its integration into real-world applications.

Summary updated

BibTeX
@inproceedings{orlando2021amuse,
  author = {Riccardo Orlando and Simone Conia and Fabrizio Brignone and Francesco Cecconi and Roberto Navigli},
  title = {{AMuSE-WSD: An All-in-one Multilingual System for Easy Word Sense Disambiguation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
}
Figure