Unsupervised Detection of Metaphorical Adjective-Noun Pairs

Malay Pramanick, Pabitra MitraNAACL 2018

TL;DR

This paper introduces a novel unsupervised framework for detecting metaphorical adjective-noun pairs, a common linguistic phenomenon. By leveraging abstractness ratings, cosine similarity, and edit distance, the method achieves competitive accuracy without relying on hand-coded knowledge, making it adaptable to evolving language.

What Problem Does This Paper Address?

Metaphors are pervasive in human language, enriching communication but posing a challenge for computational analysis. Automatically identifying and interpreting metaphors is crucial for advancing Natural Language Processing (NLP) tasks like machine translation and summarization. While many methods exist, supervised approaches require extensive labeled data, and existing unsupervised methods often depend on brittle, hand-coded knowledge bases that struggle to keep pace with language evolution. This paper aims to bridge this gap by proposing an unsupervised metaphor detection framework that is robust, scalable, and adaptive.

Methods

The core of this paper is an unsupervised framework for metaphor detection that relies on a set of carefully selected features. The authors employ a k-means clustering algorithm to partition adjective-noun (AN) pairs into two distinct clusters, which are then interpreted as metaphorical and literal. The key innovation lies in the features used for this clustering:

  • Abstractness Ratings: This feature quantifies how abstract or concrete a word is, based on the idea that metaphors often involve transferring properties from concrete domains to abstract ones. The authors use pre-computed abstractness ratings for words.
  • Cosine Similarity: This measures the semantic similarity between the adjective and the noun in a pair. The hypothesis is that metaphorical pairs often consist of words that are semantically distant. Word embeddings (from Google word2Vec trained on Wikipedia) are used to compute these similarities.
  • Edit Distance: This feature captures the phonetic or orthographic similarity between the adjective and the noun. The authors calculate the edit distance (normalized by word length) in both directions (adjective to noun, and noun to adjective) to account for potential sound or letter repetitions that might be indicative of figurative language, especially in poetic contexts.

The framework is designed to be entirely unsupervised, meaning it learns from the data without explicit human-defined rules or domain knowledge.

Datasets

The primary dataset used in this research is the TSV dataset, created by Tsvetkov et al. (2014). This dataset is specifically annotated for adjective-noun pairs, categorizing them as either metaphorical or literal.

  • TSV-Train: Consists of 884 metaphorical AN pairs and 884 literal AN pairs.
  • TSV-Test: Contains 100 metaphorical AN pairs and 100 literal AN pairs.

The authors note that this dataset was curated by annotators using public resources and then refined to remove duplicates and ambiguous cases.

Key Insights and Ideas

  • Unsupervised Adaptability: The most significant insight is the successful development of an unsupervised method that doesn't require hand-coded knowledge. This makes the system inherently more adaptable to the dynamic nature of language, where new words and metaphorical usages emerge constantly.
  • Feature Synergy: The combination of abstractness ratings, cosine similarity, and edit distance proves effective. Abstractness captures the conceptual shift, cosine similarity identifies semantic divergence, and edit distance accounts for phonetic/orthographic patterns, suggesting that different linguistic cues contribute to metaphor detection.
  • Clustering for Classification: Using k-means clustering to directly partition AN pairs into metaphorical and literal categories is a straightforward yet effective approach for an unsupervised setting. The challenge of labeling clusters is addressed by comparing accuracy against a 50% baseline.

Key Results

The paper reports the accuracy of their k-means clustering approach using different combinations of features:

  • Entire Dataset (Table 3):
    • Abstractness Ratings only: 72.31% accuracy.
  • Abstractness Ratings + Cosine Similarity: 72.56% accuracy.
  • Abstractness Ratings + Cosine Similarity + Edit Distance: 72.87% accuracy.
  • Training and Test Data (Table 4):
    • Abstractness Ratings (Training/Test): 71.21% / 82.5% accuracy.
  • Abstractness Ratings + Cosine Similarity (Training/Test): 71.44% / 82.5% accuracy.
  • Abstractness Ratings + Cosine Similarity + Edit Distance (Training/Test): 71.55% / 84% accuracy.

The results show a consistent improvement in accuracy as more features are added, with the combination of all three features yielding the best performance. The test set accuracy of 84% is particularly noteworthy, demonstrating the model's ability to generalize to unseen data.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the abstractness ratings were obtained from a pre-trained neural network and that obtaining these ratings for words not present in the original training set can be challenging, requiring heuristics like hyphen removal or averaging.
  • The unsupervised nature means that the initial labeling of the two clusters as "metaphorical" or "literal" is arbitrary and requires a post-hoc check against a baseline.
  • The authors mention that their method is not directly comparable to supervised methods in terms of Precision, Recall, and F-score, as their focus is on unsupervised accuracy.
  • Future Work:
    • The authors suggest that the proposed framework's features are language-independent and could be applied to other languages.
  • Dependency parsers could be used to extract AN metaphors from running texts, extending the application beyond pre-defined pairs.
  • Further exploration of feature combinations and clustering algorithms could potentially yield even better results.

Summary updated

BibTeX
@inproceedings{pramanick2018unsupervised,
  author = {Malay Pramanick and Pabitra Mitra},
  title = {{Unsupervised Detection of Metaphorical Adjective-Noun Pairs}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
}
Figure