TL;DR
This paper introduces CompNA, a system for predicting lexical complexity in text by combining a wide array of linguistic features. The model leverages Gradient Boosting to achieve competitive results on the SemEval-2021 Task 1, demonstrating that morphological and distributional features are strong indicators of word complexity.
What Problem Does This Paper Address?
Making text understandable for everyone is a significant challenge in natural language processing. This includes simplifying content for individuals with lower literacy levels or non-native speakers. Predicting "lexical complexity" – identifying words or phrases that are difficult to understand – is a crucial step towards this goal. While previous work often focused on binary classification (complex vs. simple), SemEval-2021 Task 1 introduced a more nuanced approach: predicting a continuous complexity score. This task is split into two sub-tasks: predicting the complexity of single words and predicting the complexity of multi-word expressions within their context.
Methods
The CompNA model tackles both sub-tasks using a unified approach: aggregating a large set of diverse linguistic features and feeding them into a Gradient Boosting regressor. The core of their method lies in the extensive feature engineering, which captures various linguistic aspects:
- Morphological Features: This includes part-of-speech tags, syntactic dependencies, syllable counts, word length, and specific word endings (desinences).
- Distributional Features: Word embeddings (GloVe) and word frequencies are used to capture how words are used in context.
- Lexical Features: The model checks for the presence of words in specialized lexicons like "obscure words," "medical words," and "simple English words."
For multi-word expressions (sub-task 2), the same features are applied to each word within the expression, with an additional feature for the frequency of the entire expression. The regressor itself is an ensemble of 120 Decision Trees with a maximum depth of 5, trained using Gradient Boosting with optimized hyperparameters.
Datasets
The CompNA model was trained and evaluated on the dataset provided for SemEval-2021 Task 1. This dataset comprises sentences annotated with complexity scores for single words and multi-word expressions. The data was sourced from three diverse corpora: the World English Bible, European Parliament proceedings, and biomedical articles. This diversity is noted as a unique aspect of the task.
Key Insights and Ideas
- Morphology Matters: The study strongly confirms that morphological features (like syllables and word endings) are highly predictive of lexical complexity, contributing significantly to the model's performance.
- Distributional Clues: Word embeddings and frequencies also play a vital role, indicating that how a word is used in text is a strong indicator of its difficulty.
- Contextual Blind Spot: While the model excels with morphological and distributional features, it appears to overlook the broader context of the word within the sentence, which is identified as a key limitation.
- Acronyms as Outliers: Acronyms are highlighted as outliers in the frequency-complexity relationship, often being annotated with higher complexity scores than their frequency might suggest.
Key Results
The CompNA model achieved competitive results on SemEval-2021 Task 1.
- Sub-task 1 (Single Words): The model ranked 26th, achieving above-average performance. Its R² score was comparable to the winning model, though with a notable difference of almost 18%.
- Sub-task 2 (Multi-word Expressions): The model ranked 24th, achieving average performance. It outperformed the baseline but lagged behind the winning model.
- Feature Importance: The most influential features were identified as "All uppercase letters" and "Word frequency," followed by syllable-related features and specific word endings like "ess" and "ium."
Limitations and Future Work
The authors explicitly acknowledge that their model overlooks features related to the context of the word. They suggest that improving the model's ability to incorporate contextual information is a key area for future work. This could involve exploring more sophisticated contextual embeddings or attention mechanisms.
Summary updated