Token-Level Metaphor Detection using Neural Networks

Erik-Lân Do Dinh, Iryna Gurevych2016 · DOI

Abstract

Automatic metaphor detection usually relies on various features, incorporating e.g. selectional preference violations or concreteness ratings to detect metaphors in text. These features rely on background corpora, hand-coded rules or additional, manually created resources, all specific to the language the system is being used on. We present a novel approach to metaphor detection using a neural network in combination with word embeddings, a method that has already proven to yield promising results for other natural language processing tasks. We show that foregoing manual feature engineering by solely relying on word embeddings trained on large corpora produces comparable results to other systems, while removing the need for additional resources.

Introduction

Identifying metaphorical language—where words are used non-literally, such as "the fabric of society"—is a fundamental challenge in Natural Language Processing (NLP). Traditional approaches have heavily favored "feature engineering," requiring researchers to manually curate linguistic indicators like selectional preference violations (e.g., checking if a verb's object is semantically compatible) or concreteness ratings. However, these resources are often language-specific and expensive to produce. This paper investigates whether a neural network can detect metaphors using only word embeddings, which are mathematical representations of words learned automatically from raw text.

Methodology

The core of the proposed system is a Multilayer Perceptron (MLP), a feedforward neural network designed to classify individual tokens as metaphorical or literal based on their context.

The Multilayer Perceptron architecture used for metaphor detection, showing the input, hidden, and output layers.

As illustrated in Figure 1, the model processes a target word within a context window of five words on either side ($w=5$). The input layer concatenates 300-dimensional word2vec embeddings for these 11 words into a single vector. This vector passes through a hidden layer with a hyperbolic tangent (tanh) activation function. To prevent overfitting, the authors employed dropout during training. The final output layer uses a softmax function to produce a probability distribution over the two classes: metaphorical or literal.

Evaluation and Results

The model was evaluated using the VU Amsterdam Metaphor Corpus (VUAMC), a subset of the British National Corpus containing approximately 100,000 content tokens across four genres: academic, conversation, fiction, and news. This dataset is notably imbalanced, with only 15% of tokens labeled as metaphorical.

Performance metrics (Precision, Recall, and F1-score) across different genres and feature combinations.

The experimental results (Figure 3) demonstrate that word embeddings alone provide a strong baseline, achieving an overall F1-score of 0.5518. Interestingly, adding traditional features like Part-of-Speech (POS) tags and concreteness ratings (Conc) only yielded marginal improvements, bringing the F1-score to 0.5614. This suggests that high-dimensional word embeddings already implicitly capture the semantic "clashes" and abstract-concrete distinctions that manual features were designed to represent.

Discussion

Several key insights emerged from the study:

  • Domain Sensitivity: The model performed best on the News subcorpus (F1 of 0.6352). This is likely because the word2vec embeddings were pre-trained on Google News data, highlighting the importance of domain alignment between pre-training and the target task.
  • Linguistic Complexity: Error analysis revealed that the model struggled more with verbs (19.5% error rate) than nouns (9.75%). Verbs often exhibit higher polysemy and more varied metaphorical usage, making them harder to classify within a fixed context window.
  • Contextual Limitations: A fixed window of five words is occasionally insufficient for complex metaphors. For instance, identifying "Berlin Wall" as a metaphor in certain sentences requires broader sentential or even paragraph-level context.

Conclusion

This research demonstrates that neural networks using only word embeddings can achieve performance comparable to state-of-the-art systems that rely on complex, hand-crafted features. By removing the dependency on specialized linguistic databases, this approach offers a more scalable and language-independent path for metaphor detection in diverse NLP applications.

Summary updated

Abstract

Automatic metaphor detection usually relies on various features, incorporating e.g. selectional preference violations or concreteness ratings to detect metaphors in text. These features rely on background corpora, hand-coded rules or additional, manually created resources, all specific to the language the system is being used on. We present a novel approach to metaphor detection using a neural network in combination with word embeddings, a method that has already proven to yield promising results for other natural language processing tasks. We show that foregoing manual feature engineering by solely relying on word embeddings trained on large corpora produces comparable results to other systems, while removing the need for additional resources.
BibTeX
@misc{dinh2016token,
  author = {Erik-Lân Do Dinh and Iryna Gurevych},
  title = {{Token-Level Metaphor Detection using Neural Networks}},
  year = {2016},
  doi = {10.18653/v1/w16-1104},
}
Figure