ReSyf: a French lexicon with ranked synonyms

Mokhtar B. Billami, Thomas François, Núria GalaCOLING 2018

TL;DR

This paper introduces ReSyf, a novel French lexical resource that ranks synonyms by their reading difficulty for native speakers. By combining a large lexical network with a sophisticated ranking algorithm validated by human annotators, ReSyf offers a valuable tool for improving text accessibility and enabling automatic lexical simplification.

What Problem Does This Paper Address?

While numerous lexical resources exist, there's a scarcity of those that provide a quantitative measure of word difficulty, particularly for synonyms. This gap is significant in educational contexts and for applications like automatic text simplification, where identifying and replacing complex words with simpler alternatives is crucial. Existing resources often lack the granularity needed to distinguish subtle differences in synonym difficulty, or they treat words without considering their specific senses, leading to imprecise difficulty estimations. The authors aim to fill this gap by creating a resource that not only identifies synonyms but also ranks them based on their perceived difficulty for L1 French readers.

Methods

The creation of ReSyf involved a multi-step methodology:

  • Lexical Resource: The authors leveraged the JeuxDeMots lexical network, a crowdsourced resource rich in semantic relationships, including synonymy.
  • Word Sense Disambiguation (WSD): To address the polysemy of words, a WSD algorithm was developed. This algorithm uses "semantic signatures" derived from the JeuxDeMots network's "ASSOCIATED IDEA" relation to compare synonym signatures with candidate word sense signatures. The goal is to accurately link a synonym to its correct sense before assessing its difficulty.
  • Ranking Algorithm: A supervised machine learning approach, specifically an SVM model, was employed for ranking synonyms by difficulty. This involved:
  • Training Dataset Creation: Pairs of words were created using the Manulex graded lexicon, where one word was designated as more complex than the other. Two methods for deriving a difficulty score from Manulex were explored (Manulex-3N and Manulex-Cont), with Manulex-3N proving more effective.
  • Feature Engineering: A comprehensive set of 69 linguistic and psycholinguistic features were extracted for each word sense, covering spelling, frequency, semantic, and morphological aspects.
  • Model Training and Optimization: An SVM model was trained on the feature-rich pairs. A grid-search was used for hyperparameter tuning, and a 10-fold cross-validation was performed. Notably, a model using a reduced set of 21 highly correlated features achieved comparable or slightly better accuracy than a full model.

Datasets

  • JeuxDeMots: Used as the primary source for lexical entries and synonymy relations.
  • Manulex: A graded lexicon for French, used to create the training dataset for the ranking algorithm by providing word difficulty annotations.
  • Human Annotations: A crucial dataset for evaluating the ranking algorithm, consisting of manually ranked synonym lists provided by 40 human annotators.

Key Insights and Ideas

  • Synonym Difficulty is Sense-Dependent: The paper strongly emphasizes that the difficulty of a synonym is not inherent to the word form itself but is tied to its specific meaning (word sense). This necessitates robust Word Sense Disambiguation before ranking.
  • Semantic Signatures for WSD: The use of "semantic signatures" derived from network structural properties (rather than just corpus co-occurrence) offers a novel way to represent words and senses for WSD.
  • Ranking Outperforms Coarse-Grained Difficulty Levels: The authors argue that ranking synonyms by difficulty is more effective than assigning discrete difficulty levels, especially for fine-grained distinctions.
  • Feature Importance for Difficulty Prediction: The study identified key features (e.g., log-frequency, number of senses, morphological family features) that are highly predictive of word difficulty.
  • Human Validation is Key: The high correlation between the automatic ranks and human judgments underscores the effectiveness of the developed ranking algorithm and the validity of the ReSyf resource.

Key Results

  • High Agreement with Human Judgments: The ReSyf ranking algorithm achieved an impressive 91.04% agreement with human annotators, with synonyms correctly sorted or inversed by at most one rank.
  • Effective WSD: The WSD algorithm demonstrated high precision (99.28% for Algorithm 1) in correctly assigning synonyms to their target word senses.
  • Robust Ranking Model: The SVM model, particularly when trained on the Manulex-3N dataset with 21 selected features, achieved an accuracy of 77.8% in predicting the relative difficulty of word pairs.
  • Comprehensive Lexicon: ReSyf contains a substantial number of entries, with a distribution across different parts of speech and a mean of 4.95 synonyms per polysemic entry sense.

Limitations and Future Work

  • Multiword Expressions (MWEs): The current ranking of MWEs is an approximation based on the average difficulty of their content words. Handling MWE complexity more precisely is a significant challenge and a key area for future work.
  • Tailoring to Specific Audiences: The authors envision tailoring ReSyf for specific user groups (e.g., dyslexic readers, individuals with vision impairments) by adapting the simplification strategies. This could involve favoring certain types of synonyms based on specific needs.
  • Continued Lexicon Refinement: JeuxDeMots is constantly evolving, suggesting that ReSyf will also benefit from ongoing updates and expansions.

Summary updated

BibTeX
@inproceedings{billami2018resyf,
  author = {Mokhtar B. Billami and Thomas François and Núria Gala},
  title = {{ReSyf: a French lexicon with ranked synonyms}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}
Figure