Phrase-Level Metaphor Identification Using Distributed Representations of Word Meaning

Omnia Zayed, John P. McCrae, Paul BuitelaarNAACL 2018

TL;DR

This paper introduces a semi-supervised approach for identifying metaphorical verb-noun phrases using pre-trained word embeddings. By measuring semantic similarity between candidate phrases and a small set of known metaphors, the system aims to detect metaphors without requiring large annotated datasets or complex feature engineering, offering a more flexible and resource-efficient solution.

What Problem Does This Paper Address?

Metaphors are fundamental to human language, allowing us to express abstract concepts by drawing parallels with more concrete ones. However, computationally identifying metaphors, especially at the phrase level (e.g., "eradicate poverty"), is a significant challenge for Natural Language Processing (NLP) tasks like machine translation and text simplification. Existing methods often rely on extensive lexical resources or require large, manually annotated datasets, which are costly and time-consuming to create. This paper aims to bridge this gap by developing a more efficient, semi-supervised method that leverages the power of distributed word representations to capture metaphorical meaning.

Methods

The core of the proposed approach lies in using distributed representations of word meaning, specifically pre-trained word embeddings. The system works as follows:

  1. Candidate Extraction: A dependency parser (Stanford parser) is used to extract verb-noun pairs from sentences, identifying potential metaphorical expressions.
  2. Similarity Measurement: For each candidate verb-noun pair, the system calculates its semantic similarity to a predefined seed set of known metaphorical verb-noun pairs. This similarity is measured using cosine distance between word embedding vectors.
  3. Hypothesis-Driven Classification: The authors hypothesize that a metaphorical candidate will share semantic features with known metaphors. However, they acknowledge that simply comparing the entire phrase isn't enough. They propose that by looking at the similarity of the verb and noun components individually, and specifically focusing on the similarity of the nouns when verbs are similar, they can better distinguish metaphors from literal expressions.
  4. Semi-Supervised Learning: The system uses a small set of manually identified metaphorical verb-noun pairs (the "seed set") as positive examples. A similarity threshold, optimized on a development dataset, is then used to classify new candidates. This avoids the need for large amounts of labeled data.

Datasets

The paper utilizes several datasets for training and evaluation:

  • Seed Set: A set of 62 metaphorical verb-noun pairs originally obtained from the British National Corpus (BNC) and annotated by Shutova et al. (2010).
  • VUA Test Set: A subset of the VU Amsterdam Metaphor Corpus (VUA), comprising approximately 300 verb-noun pairs (160 positive, 145 negative) extracted from original sentences. This dataset is annotated at the token level, but the authors adapted it for phrase-level analysis.
  • MOH Dataset: A manually annotated dataset of 647 verb-subject and verb-object pairs, with 316 metaphorical and 331 literal instances, originally derived from WordNet senses.

Key Insights and Ideas

  • Leveraging Word Embeddings for Metaphor: The paper demonstrates that pre-trained word embeddings (like Word2Vec and GloVe) can effectively capture semantic relationships relevant to metaphor identification, even without explicit linguistic feature engineering.
  • The Importance of Individual Word Similarity: The hypothesis that focusing on the similarity of individual components (verb and noun) and their relationship, especially when verbs are shared, is crucial for accurate metaphor detection. This moves beyond simple holistic phrase similarity.
  • Semi-Supervised Approach Efficiency: The success of a semi-supervised approach using a small seed set highlights the potential for building metaphor detection systems with significantly less manual annotation effort compared to fully supervised methods.
  • Context Matters, But Can Be Tricky: The authors note that metaphoricity can be context-dependent, and their system's performance can be affected by the limitations of the seed set and the inherent ambiguity in some annotations.

Key Results

The paper reports performance comparisons against a baseline system (Shutova et al., 2010).

  • GloVe Outperforms Word2Vec: The proposed approach generally performed better when using GloVe word embeddings compared to Word2Vec, especially when combined with the cosine distance metric.
  • VUA Test Set Performance: On the VUA test set, the proposed system achieved an F-score of 0.7354 with GloVe and cosine distance, outperforming the baseline's F-score of 0.0385 (for WMD) and 0.5763 (for cosine distance with Word2Vec).
  • MOH Dataset Performance: On the MOH dataset, the system achieved an F-score of 0.6841 with GloVe and cosine distance, again showing improvement over the baseline's F-score of 0.0189 (for WMD) and 0.5739 (for cosine distance with Word2Vec).
  • Recall Challenges: The system exhibited lower recall when using Word2Vec, potentially due to limited coverage of the seed set.
  • Baseline Limitations: The baseline system showed very low recall on the test sets, largely because its clustering approach could not accommodate many of the test set candidates.

Limitations and Future Work

  • Seed Set Size: The authors acknowledge that the size of the seed set is a limitation. Increasing its size could improve the system's coverage and performance.
  • Multi-Word Expressions: The current approach primarily focuses on verb-noun pairs and does not explicitly handle multi-word expressions like phrasal verbs.
  • Contextual Ambiguity: The inherent context-dependency of metaphor and potential inconsistencies in dataset annotations pose challenges.
  • Future Work: The authors plan to investigate the impact of larger seed sets, extend the approach to other syntactic constructions (like phrasal verbs), and explore different word embedding models and similarity measures. They also aim to address the issue of multi-word expressions.

Summary updated

BibTeX
@inproceedings{zayed2018phrase,
  author = {Omnia Zayed and John P. McCrae and Paul Buitelaar},
  title = {{Phrase-Level Metaphor Identification Using Distributed Representations of Word Meaning}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
}
Figure