Personalized PageRank with Syntagmatic Information for Multilingual Word Sense Disambiguation

Federico Scozzafava, Marco Maru, Fabrizio Brignone, Giovanni Torrisi, Roberto NavigliACL 2020

TL;DR

This paper introduces SyntagRank, a novel knowledge-based system for multilingual Word Sense Disambiguation (WSD) that leverages syntagmatic (co-occurrence) information alongside traditional paradigmatic (hierarchical) relations. By integrating these richer semantic connections into a graph-based approach, SyntagRank achieves state-of-the-art performance, outperforming existing systems on both English and multilingual WSD tasks.

What Problem Does This Paper Address?

Word Sense Disambiguation (WSD) is a fundamental challenge in Natural Language Processing (NLP) where the goal is to identify the correct meaning of a word in a given context. Traditional approaches fall into two main categories: supervised methods, which require vast amounts of hand-labeled data (expensive and time-consuming, especially for multiple languages), and knowledge-based methods, which rely on lexical knowledge bases (LKBs) like WordNet. While knowledge-based methods are more easily adaptable to multilingual settings, their performance is often limited by the structure of the LKBs they use. LKBs typically emphasize paradigmatic relations (e.g., "dog" is a type of "animal"), but often underrepresent syntagmatic relations (e.g., "dog" often appears with "bark" or "leash"). This paper addresses the gap in performance between supervised and knowledge-based WSD by proposing a method to enrich LKBs with syntagmatic information, thereby improving the accuracy of knowledge-based WSD systems, particularly in a multilingual context.

Methods

SyntagRank employs a knowledge-based approach that builds upon the Personalized PageRank (PPR) algorithm. The core innovation lies in the construction of a comprehensive LKB that integrates three sources:

  • WordNet: The foundational lexical-semantic database for English, providing synsets (sets of cognitive synonyms) and paradigmatic relations.
  • Princeton WordNet Gloss Corpus (PWNG): This corpus provides definitions (glosses) for WordNet synsets. By linking synsets to words within their own glosses, new, contextually relevant relations are induced.
  • SyntagNet: This is the key novel resource. SyntagNet is a database of manually disambiguated lexical collocations and word associations, specifically designed to capture syntagmatic relations. It provides pairs of co-occurring words that are linked to WordNet synsets.

The system constructs a unified graph from these sources. For a given input sentence, it first performs a multilingual NLP pipeline (tokenization, lemmatization, PoS tagging) and then retrieves candidate senses for each content word. The PPR algorithm is then applied, with the initial probability mass personalized to the candidate senses of the target words. This allows the system to propagate relevance through the graph, with syntagmatic links providing direct pathways between semantically related words that frequently co-occur. The system also incorporates a frequency-based weighting mechanism (freqws) to mitigate the bias towards Most Frequent Senses (MFS) often seen in unsupervised WSD.

Datasets

The paper evaluates SyntagRank on several standard WSD datasets:

  • English: Senseval-2, Senseval-3, SemEval-2007, SemEval-2013, and SemEval-2015.
  • Multilingual: Amended versions of SemEval-2013 and SemEval-2015 evaluation datasets for German, Spanish, French, and Italian.

The paper does not introduce new benchmark datasets but rather leverages existing ones to demonstrate the effectiveness of its approach.

Key Insights and Ideas

  • Syntagmatic information is crucial for WSD: The paper strongly argues and demonstrates that incorporating syntagmatic relations, which capture how words co-occur in context, significantly enhances WSD performance beyond what paradigmatic relations alone can achieve.
  • SyntagNet as a valuable resource: The creation and utilization of SyntagNet, a resource specifically curated for syntagmatic lexical-semantic combinations, is a key contribution. It bridges the gap left by traditional LKBs.
  • Knowledge-based WSD can rival supervised methods: By enriching LKBs with syntagmatic data, knowledge-based systems like SyntagRank can achieve performance comparable to, and in this case, superior to, supervised systems, while retaining the advantage of easier multilingual adaptation.
  • Personalized PageRank is effective for graph-based WSD: The application of PPR, with personalization to target word senses, proves to be a robust mechanism for propagating relevance through the enriched semantic graph.

Key Results

SyntagRank achieves state-of-the-art results on both English and multilingual WSD tasks. In Table 1, SyntagRank consistently outperforms other graph-based systems like Babelfy and UKB across various English and multilingual evaluation settings, often by a considerable margin. For instance, in the multilingual SemEval-2015 task, SyntagRank achieves an F1 score of 71.2, outperforming Babelfy (69.4) and UKB (63.4). The paper also highlights that these results are statistically significant.

Limitations and Future Work

The authors acknowledge a few limitations:

  • Quality of BabelNet lexicalizations: For multilingual candidate retrieval, the system relies on BabelNet, whose lexicalizations are induced from automatically linked resources, potentially leading to sub-optimal quality.
  • Lack of SemCor equivalents for other languages: The absence of a direct equivalent to the English SemCor corpus for other languages means that accessible MFS information is not readily available for computing weighted averages.

To address these, they devised a strategy to mimic MFS ranking by assigning confidence scores to different lexical resources that contribute to BabelNet. Future work could involve further exploration of how to improve the quality of multilingual lexicalizations and MFS estimation. The paper also introduces a Web interface and a RESTful API for SyntagRank, making the system accessible to the research community and facilitating further research and applications.

Summary updated

BibTeX
@inproceedings{scozzafava2020personalized,
  author = {Federico Scozzafava and Marco Maru and Fabrizio Brignone and Giovanni Torrisi and Roberto Navigli},
  title = {{Personalized PageRank with Syntagmatic Information for Multilingual Word Sense Disambiguation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
}
Figure