Abstract
Distinguishing between abstract concepts (e.g., "justice") and concrete ones (e.g., "apple") is a fundamental task in psycholinguistics, yet scaling this to large vocabularies is limited by the cost of human annotation. This paper introduces a weak supervision framework that utilizes morphological rules and contextual patterns to learn abstractness. By training on "noisy" labels derived from word suffixes and Wikipedia context, an RNN-based model achieves near-human correlation with expert ratings, demonstrating high data efficiency and generalizability across various datasets.
Introduction
In psycholinguistics, "abstractness" refers to the degree to which a concept cannot be directly perceived through the five senses. While intuitive for humans, this distinction presents a significant challenge for AI systems. Existing resources for word abstractness are often small due to the reliance on manual labeling, which creates a bottleneck for downstream Natural Language Processing (NLP) tasks such as metaphor detection, lexical simplification, and readability assessment.
This research addresses the data scarcity problem by demonstrating that abstractness can be inferred from linguistic structure and context without massive human-labeled datasets. The authors propose a "weak supervision" approach that bootstraps learning from simple morphological cues.
Methodology
The proposed approach integrates two primary linguistic signals: Morphology and Context.
Weak Labeling and Seed Rules
Instead of human-provided labels, the researchers utilized a "seed" rule based on morphology. Words with specific suffixes, such as -ism (e.g., marxism) and -ness (e.g., happiness), were automatically labeled as abstract. These were paired with a set of random words assumed to be concrete, forming a noisy training set.
Contextual Learning
The models were trained on 800,000 sentences extracted from Wikipedia. To prevent the models from simply memorizing specific words, the target concepts were masked during training. This forced the models to identify abstractness based solely on the surrounding linguistic environment—the "company a word keeps."
Model Architectures
Three distinct modeling approaches were evaluated:
- Naive Bayes (NB): A baseline probabilistic model utilizing word counts.
- Nearest Neighbor (NN-RAD): A model using GloVe word embeddings to identify similar concepts in vector space.
- RNN with Attention: A Bidirectional LSTM equipped with an attention mechanism to prioritize contextual words that best indicate abstractness.
Experimental Setup
The models were trained on 1,040 unigrams from Wikipedia and evaluated against several "gold standard" benchmarks:
- MRC Database: A standard psycholinguistic resource.
- BWK Dataset: A large-scale manual annotation containing 40,000 concepts.
- New Evaluation Set: A custom dataset of 1,500 Wikipedia concepts (unigrams, bigrams, and trigrams) annotated by seven human experts.
Results and Discussion
The RNN model emerged as the superior approach, particularly for unigrams, achieving a 0.74 Pearson correlation with human labels. This performance is notable as it falls within the range of typical human-to-human agreement (0.65–0.72).
Data Efficiency and Generalization
A key finding was the model's remarkable data efficiency. As shown in the performance analysis below, the RNN model achieves high correlation with human intuition using as few as 10 sentences per concept.
Beyond performance, the model demonstrated an ability to "discover" new morphological markers. After training, it successfully identified other suffixes associated with abstractness, such as -sis (e.g., analysis) and -ics (e.g., politics), which were not part of the initial seed rules.
Complexity and Limitations
While the model excels at unigrams, performance decreases as concept complexity increases. Trigrams (e.g., "absolute monarchy") achieved a correlation of 0.49, likely due to the higher subjectivity and lower human agreement associated with complex phrases. Additionally, the use of static embeddings limits the model's ability to handle polysemy, where a word like "bank" may have both concrete and abstract senses depending on context.
Conclusion
This work demonstrates that weak supervision is a powerful tool for capturing nuanced psychological properties like abstractness. By combining simple morphological heuristics with contextual deep learning, the authors provide a scalable "recipe" for building psycholinguistic resources in a data-efficient manner. This approach holds significant promise for adaptation to other languages where labeled data is currently unavailable.
Summary updated