TL;DR
This study investigates whether linguistic features can predict the quality of figurative language (metaphors and sarcasm), moving beyond simple identification. The authors found that while linguistic sophistication plays a role, particularly in predicting creativity (novelty and mirth), it explains only a small portion of the variance in human ratings, suggesting other factors are at play.
What Problem Does This Paper Address?
While computational approaches have become adept at identifying figurative language, they often overlook the quality of that language. This paper aims to bridge that gap by exploring whether specific linguistic features can explain why some metaphors and sarcastic statements are perceived as higher quality than others. The authors focus on two key aspects of quality: theoretical components (conceptual distance for metaphors, incongruity for sarcasm) and general creativity (novelty and mirth). Understanding these predictors is crucial for developing more nuanced NLP models that can not only detect figurative language but also assess its effectiveness and originality.
Methods
The researchers employed a multi-faceted approach:
- Data Collection: They collected original data by having participants produce metaphors and sarcastic responses to various prompts. This allowed them to control the experimental setup and gather specific linguistic measures.
- Human Ratings: Participants' generated metaphors and sarcastic statements were rated by human judges using an analytic rubric. This rubric assessed quality based on theoretical definitions of metaphor (conceptual distance) and sarcasm (incongruity), as well as general creativity (novelty and mirth).
- Linguistic Feature Extraction: Two sophisticated NLP tools were used to extract linguistic features:
- TAALES (Tool for the Automatic Analysis of LExical Sophistication): This tool measured lexical sophistication through various indices, including word familiarity, concreteness, imageability, meaning-fulness, age of acquisition (AoA), spoken word frequency, and semantic diversity.
- TAACO (Tool for the Automatic Analysis of Cohesion): This tool was used to calculate semantic overlap between the prompt and the generated metaphor using the word2vec measure, aiming to capture semantic distance.
- Statistical Analysis: Linear Mixed Effects (LME) models were used to predict the human ratings of figurative language quality using the extracted linguistic features. This approach accounts for individual differences among participants and the specific prompts used.
Datasets
The study introduced a newly collected corpus of:
- 1304 metaphors: Generated by participants in response to conventional and novel metaphor prompts.
- 716 sarcastic responses: Generated by participants in response to cartoon prompts (adapted from existing studies and original creations).
These datasets were specifically designed to allow for the collection of detailed linguistic features and human quality ratings, differing from existing corpora that might focus solely on identification.
Key Insights and Ideas
- Linguistic Sophistication Matters, But Only So Much: The study found that linguistic features, particularly those related to lexical sophistication (like Age of Acquisition and spoken word frequency), do predict aspects of figurative language quality. However, the effect size is generally small.
- Creativity is More Predictable Than Core Concepts: Linguistic features were better at predicting novelty and mirth (creativity) than conceptual distance (metaphor) or incongruity (sarcasm). This suggests that while we can identify some linguistic markers of creativity, the core theoretical underpinnings of metaphor and sarcasm are less directly captured by these features alone.
- Context is King for Sarcasm: The paper highlights that sarcasm's quality is heavily influenced by social and pragmatic context, making it harder to predict using purely linguistic features compared to metaphors. A simple phrase can be sarcastic depending on the situation, which is difficult for text-based analysis to capture.
- Prompt Type Influences Sarcasm Quality: For sarcasm, the type of prompt (e.g., simple comic vs. multi-panel) interacted with linguistic features, suggesting that less context-rich prompts might rely more on lexically sophisticated language for effective sarcasm.
Key Results
- Metaphor Conceptual Distance: Higher Age of Acquisition (AoA) scores and higher Semantic Diversity were associated with lower conceptual distance ratings. Conversely, higher Word Concreteness was linked to higher conceptual distance. These features explained only 4.1% of the variance.
- Metaphor Novelty/Mirth: Higher Imageability, lower Source Similarity (word2vec), and lower COCA Spoken Word Frequency were significant predictors of higher novelty/mirth ratings. This suggests that more sophisticated language and less direct referencing of the prompt lead to higher perceived creativity. These features explained 7.5% of the variance.
- Sarcasm Incongruity: Higher MRC Meaningfulness was a significant, negative predictor of incongruity ratings, meaning more associated words led to lower incongruity. This model explained only 2% of the variance.
- Sarcasm Novelty/Mirth: Higher Age of Acquisition (AoA) received significantly higher novelty/mirth ratings. Interactions showed that for desert island prompts, higher MRC Familiarity and Brysbaert Concreteness increased novelty/mirth perceptions. These features explained 6.8% of the variance.
Limitations and Future Work
- Short Responses: The authors acknowledge that the brevity of participant responses might bias lexical measurements. Future work could explore longer, more complex examples.
- Small Variance Explained: The linguistic features explained a relatively small amount of variance in human ratings, indicating that other factors (e.g., individual differences, broader contextual knowledge, pragmatic understanding) are crucial for judging figurative language quality.
- Future Directions: The study suggests that future work on classifiers for figurative language should consider quality, not just identification. Linguistic features related to creativity are promising avenues for this. The authors also point out the challenge of capturing sarcasm's context-dependent nature through purely linguistic analysis.
Summary updated