A Generalized Idiom Usage Recognition Model Based on Semantic Compatibility

Changsheng Liu, Rebecca HwaAAAI 2019 · DOI

Abstract

Idioms present a significant challenge for Natural Language Processing due to their dual literal and figurative interpretations. This paper introduces a generalized model for idiom usage recognition that leverages semantic compatibility. By adapting the Continuous Bag-of-Words (CBOW) architecture with sequential modeling, attention mechanisms, and non-linear evaluation layers, the model identifies figurative usage by detecting semantic "clashes" between an idiom's literal meaning and its context. Trained on raw Wikipedia text, the model achieves state-of-the-art performance on standard benchmarks without requiring idiom-specific training data.

Introduction

Idiomatic expressions like "playing with fire" or "kicking the bucket" are ubiquitous in natural language but notoriously difficult for computational models. The primary hurdle is ambiguity: an idiom can be used literally (e.g., "the firefighter was playing with fire during the drill") or figuratively (e.g., "investing in that volatile stock is playing with fire").

Traditional approaches rely on "per-idiom" classification, requiring thousands of human-annotated examples for every unique phrase. This paper addresses the need for a generalized model—one that can handle any idiom it encounters by understanding the underlying relationship between a phrase and its context, rather than memorizing specific idiomatic patterns.

Methodology: From CBOW to Semantic Compatibility

The researchers developed a transfer learning framework based on an adapted Continuous Bag-of-Words (CBOW) architecture. The core intuition is that a model trained to predict word-context compatibility on general text can be repurposed to detect the "absurdity" of literal idiom meanings in figurative contexts.

Architectural Upgrades

To transform standard CBOW into a robust compatibility evaluator, three key modifications were implemented:

  1. Sequential Modeling with Bi-LSTM: Unlike standard CBOW, which averages context word embeddings, this model uses a Bidirectional LSTM to capture word order and long-range dependencies.
  2. Attention Mechanisms: To identify "critical clues" in the context (e.g., specific prepositions or nouns that trigger idiomatic readings), the authors integrated Global and Local Attention layers.
The attention mechanism utilizes a relevance matrix to weight context words based on their importance to the target word.
  1. Non-linear Semantic Evaluation: Standard models use dot products to measure similarity, which can lead to a "transitivity paradox." The authors replaced this with a Multilayer Perceptron (MLP) layer to model complex, non-linear relationships between context and target representations.

The Workflow

The model is first trained on a 100-million-token subset of Wikipedia to learn general semantic compatibility. During inference, the "Literal Representation" of a target idiom is compared against the "Context Representation" generated from the surrounding sentence. The Semantic Compatibility Evaluation Layers then output a score indicating whether the usage is literal or figurative.

The transfer learning workflow: an Adapted CBOW model is trained on general text, and its layers are transferred to the Idiom Usage Recognition Model.

Experimental Results

The model was evaluated on two major benchmarks: the SemEval 2013 Task 5B and the Verb-Noun Combination (VNC) Dataset.

The generalized model demonstrated remarkable efficacy, achieving an F-score of 0.76 on SemEval and 0.75 on VNC. Crucially, it significantly outperformed previous unsupervised generalized models (which typically scored between 0.58 and 0.61) and remained competitive with supervised models that require idiom-specific training data. Ablation studies confirmed that the MLP-based Semantic Evaluation Layer was the most critical component for performance.

Conclusion

By shifting the focus from pattern matching to semantic compatibility, this research provides a scalable path forward for idiom recognition. The model's ability to generalize to unseen idioms without manual annotation addresses a major bottleneck in NLP. Future work may extend this "clash detection" logic to other figurative forms, such as metaphors and sarcasm, where literal meanings also tend to clash with their surroundings.

Summary updated

BibTeX
@inproceedings{liu2019generalized,
  author = {Changsheng Liu and Rebecca Hwa},
  title = {{A Generalized Idiom Usage Recognition Model Based on Semantic Compatibility}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33016738},
}
Figure