Verifying Claims About Metaphors with Large-Scale Automatic Metaphor Identification

Kotaro Aono, Ryohei Sasano, Koichi TakedaNAACL 2024 · DOI

TL;DR

This paper uses a large-scale corpus analysis to empirically verify linguistic claims about metaphors. They found that metaphorical language indeed tends to involve less concrete, imageable, and familiar direct objects, and is more prevalent in emotional and subjective contexts, providing robust evidence for long-standing theories.

What Problem Does This Paper Address?

For decades, linguists have proposed theories about when and why we use metaphors. A prominent idea, stemming from Lakoff and Johnson's work, is that metaphors help us understand abstract concepts (like arguments) by mapping them onto more concrete ones (like war). This leads to specific predictions: for instance, that the "target" concept in a metaphor (e.g., "argument" in "winning an argument") will be less concrete, less imageable, and less familiar than its literal counterpart. While these claims are influential, they've often been supported by small, anecdotal examples. The challenge this paper addresses is the lack of large-scale, data-driven verification for these linguistic hypotheses. Without such evidence, it's hard to know how broadly applicable these theories are or how they manifest in real-world language use.

Methods

The core of this research lies in applying a sophisticated automatic metaphor identification model to a massive dataset. They employed a model called MisNet, which combines two established techniques: Metaphor Identification Procedure (MIP) and Selectional Preference Violation (SPV). MIP works by detecting differences in word embeddings between metaphorical and literal uses, while SPV focuses on semantic incongruity between a word and its surrounding context.

To verify their claims, they collected sentences from the Common Crawl corpus and used their metaphor identification model to label verb-object pairs as either metaphorical or literal. They established a threshold: if over 70% of a verb-object pair's occurrences were metaphorical, they treated it as a metaphorical example; if less than 30% were metaphorical, it was considered a literal example. For claims related to concreteness, imageability, and familiarity, they used external datasets to score the direct objects of these verb-object pairs. For claims about emotion and subjectivity, they used a sentiment analysis model to categorize sentences and then analyzed metaphor usage rates across these categories.

Datasets

  • VU Amsterdam Metaphor Corpus: Used to train the metaphor identification model (MisNet). This corpus contains 190,000 manually annotated lexical units.
  • LFP Dataset: Used to evaluate the concreteness and imageability of words. This dataset contains approximately 100,000 English words with assigned scores.
  • Word-Complexity Lexicon (WCL): Used to approximate word familiarity. This dataset contains 15,000 English words with a 6-point word complexity score.
  • Common Crawl: The primary source for the large-scale corpus of 1 billion English sentences used for analysis.
  • Stanza's sentiment analysis model: Used to classify sentences by emotional polarity (positive, neutral, negative).

Key Insights and Ideas

  • Metaphor as a cognitive tool: The study provides strong empirical backing for the idea that metaphors aren't just stylistic flourishes but deeply embedded cognitive mechanisms that help us grasp abstract concepts through more tangible ones.
  • The "less is more" principle in metaphor: The findings suggest that when we use metaphors, we often choose target concepts that are less defined by concrete attributes (concreteness, imageability, familiarity). This might be because these less "fixed" concepts are more malleable and better suited for mapping abstract ideas onto.
  • Emotion and subjectivity as metaphor drivers: The research highlights that emotional and subjective language is a fertile ground for metaphors. This suggests that metaphors play a crucial role in expressing nuanced internal states and feelings, making them more than just descriptive tools.
  • The power of large-scale analysis: This work demonstrates the value of moving beyond small-scale linguistic examples to large-scale corpus analysis for validating linguistic theories.

Key Results

  • Concreteness, Imageability, Familiarity: For all 49 verbs analyzed, direct objects in metaphorical examples consistently showed lower scores for concreteness, imageability, and familiarity compared to their literal counterparts. This finding was statistically significant (p < 0.0001 for concreteness and imageability, and for 45 out of 49 verbs for familiarity).
  • Emotion and Subjectivity:
    • Sentences with emotional polarity (positive or negative) showed significantly higher metaphor usage rates than neutral sentences.
  • Positive sentences exhibited the highest metaphor usage rates.
  • Sentences with a first-person subject (indicating likely subjectivity) had significantly higher metaphor usage rates than those with a third-person subject.

Limitations and Future Work

  • Language Specificity: The analysis was limited to English. The authors acknowledge that it's unclear if these findings generalize to other languages.
  • Automatic Determination: The metaphor identification system is automatic, and while its performance is generally good, potential biases in its errors could influence the results.
  • Text-Based Corpus: The analysis primarily used written language from Common Crawl. The applicability of findings to spoken language is unknown.
  • Familiarity Approximation: Familiarity was approximated using word complexity scores due to the lack of a direct dataset. If this approximation is flawed, it could impact conclusions.

Future work could involve cross-lingual studies, more in-depth analysis of the metaphor identification model's error patterns, and investigations using spoken language corpora.

Summary updated

BibTeX
@inproceedings{aono2024verifying,
  author = {Kotaro Aono and Ryohei Sasano and Koichi Takeda},
  title = {{Verifying Claims About Metaphors with Large-Scale Automatic Metaphor Identification}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-short.62},
}
Figure