Bigrams and BiLSTMs Two Neural Networks for Sequential Metaphor Detection

Yuri Bizzoni, Mehdi GhanimifardNAACL 2018

TL;DR

This paper introduces and compares two neural network architectures for automatically detecting metaphors in text: a Bi-LSTM model and a novel bigram composition model. They demonstrate that incorporating explicit semantic features like concreteness scores and breaking down long sentences can significantly improve metaphor detection performance, even with simpler models.

What Problem Does This Paper Address?

Automatically identifying metaphors in text is a challenging task. Metaphors are deeply embedded in language, often relying on subtle shifts in meaning and context. Traditional approaches have involved manually crafted rules or machine learning models that rely on linguistic features like abstractness, imageability, and word embeddings. However, these methods can be complex and may not fully capture the nuanced nature of metaphor. This paper aims to improve metaphor detection by exploring deep learning architectures and investigating the impact of input manipulation, specifically the addition of explicit semantic features and sentence segmentation.

Methods

The authors propose and compare two main neural network architectures:

  • Bi-LSTM Model: This is a standard bidirectional Long Short-Term Memory (Bi-LSTM) network, a type of recurrent neural network well-suited for processing sequential data like text. They experimented with different depths and widths of this architecture.
  • Bigram Composition Model: This novel architecture is based on recursive feed-forward concatenation of input word embeddings. It's designed to capture local compositions, specifically bigrams (pairs of words), and can be made deeper to allow for wider window sizes. This model is presented as a generalization of architectures used for bigram phrase composition.

They also investigate two key input manipulations:

  • Word Embeddings: They use pre-trained GloVe and Word2Vec embeddings.
  • Concreteness Scores: They incorporate explicit concreteness scores for words, sourced from a large dataset of human annotations. This feature aims to capture the abstract-concrete polarity often present in metaphors.
  • Sentence Breaking: They experiment with breaking long sentences into shorter segments to mitigate the negative effects of padding.

Datasets

The primary dataset used in this study is the VU Amsterdam Metaphor Corpus (VUAMC). This corpus is described as the largest publicly available annotated corpus for metaphor detection. It's divided into four genres: news texts, fiction, academic texts, and conversations. The authors note that the corpus has fine-grained annotations, sometimes leading to ambiguity or questionable classifications.

Key Insights and Ideas

  • Concreteness Matters: Explicitly adding word concreteness scores as a feature significantly boosted performance, particularly for the compositional architecture. This suggests that the abstract-concrete gap is a crucial signal for metaphor detection.
  • Simplicity Can Be Effective: The bigram composition model, despite being simpler than the Bi-LSTM, achieved comparable results, especially when enriched with concreteness scores. This highlights that complex architectures aren't always necessary for good performance.
  • Sentence Length is a Factor: Breaking down long sentences into shorter sequences improved model performance. This indicates that long-distance dependencies might not be as critical for metaphor detection as the noise introduced by extensive padding.
  • Complementary Models: Combining the outputs of the Bi-LSTM and the compositional models yielded the best overall results, suggesting they learn to detect slightly different types of metaphors.

Key Results

  • The best performing model was a deep neural network composed of a Bi-LSTM preceded and followed by fully connected layers, augmented with concreteness scores and operating on relatively short sequences. This model achieved an F1 score of 0.636 on the evaluation set from the NAACL 2018 shared task.
  • The bigram composition model, when combined with concreteness scores and sentence breaking, achieved an F1 score of 0.580.
  • Adding concreteness scores alone improved the F1 score by up to 4 points for the compositional model.
  • Sentence breaking improved the F1 score by up to 6 points for the Bi-LSTM model.

Limitations and Future Work

The authors acknowledge the difficult nature of the original annotations in the VUAMC, noting that the fine-grained nature can lead to ambiguity and questionable classifications. They suggest that adding more explicit features could potentially help reduce inconsistencies.

Future work directions include:

  • In-depth error analysis: A systematic analysis of model errors to better understand what neural networks learn about metaphor detection.
  • Comparison with simpler algorithms: Extending comparisons to traditional machine learning algorithms to better assess the contribution of deep learning.
  • Exploring more explicit features: Investigating how a consistent body of work on metaphor detection using traditional machine learning can be directly compared with their deep learning approaches.

Summary updated

BibTeX
@inproceedings{bizzoni2018bigrams,
  author = {Yuri Bizzoni and Mehdi Ghanimifard},
  title = {{Bigrams and BiLSTMs Two Neural Networks for Sequential Metaphor Detection}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
}
Figure