Abstract
Hessel Haagsma, Malvina Nissim, Johan Bos. Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018).
Introduction
Determining whether an expression like "hit the wall" is used literally or figuratively remains a significant challenge in Natural Language Processing (NLP). These Potentially Idiomatic Expressions (PIEs) are crucial for tasks like machine translation, where a literal translation of a figurative phrase can lead to nonsensical output. This paper explores a novel unsupervised approach to idiom disambiguation by contrasting the semantic "fit" of an idiom’s literal words against a literalized version of its figurative meaning.
The Problem: Lexical Cohesion
Most unsupervised disambiguation methods rely on lexical cohesion. This principle suggests that if an expression is used literally, its component words (e.g., hay and sun in "make hay while the sun shines") will be semantically related to the surrounding context. Conversely, if the words seem out of place, the usage is likely figurative. However, traditional methods typically only test the literal sense; they fail to explicitly verify if the figurative sense fits the context. The authors propose providing the model with "the other side of the coin"—a literal representation of the idiom's figurative meaning.
Methodology
The authors extend the graph-based lexical cohesion classifier developed by Sporleder and Li (2009). The process involves:
- Graph Representation: The PIE and its context words are represented as nodes in a graph, with edges weighted by the cosine similarity of their GloVe word embeddings.
- Idiom Literalisation: For each PIE, dictionary definitions were manually refined into concise "literalisations" (e.g., etched in stone becomes "permanently fixed").
- Contrastive Analysis: The system constructs two graphs:
- Graph A (Literal Signal): Contains the original PIE words.
- Graph B (Figurative Signal): Replaces the PIE words with the literalisation.
- Classification: If Graph B exhibits higher average semantic cohesion than Graph A, the expression is classified as idiomatic.
Datasets
To ensure a robust evaluation, the authors aggregated four major corpora into a "Combined" benchmark, normalizing all labels to a binary "literal vs. figurative" classification.
The combined dataset was split into a development set (8,235 instances) and a test set (3,073 instances).
Key Results and Insights
On the combined test set, the results were as follows:
- Original Cohesion Method: 67.61% (Harmonic Mean of macro/micro accuracy).
- Literalisation Method: 69.49% (Harmonic Mean).
- Baseline (Most Frequent Sense): 61.98%.
While the literalisation method slightly outperformed the original, both remained behind state-of-the-art supervised systems. However, several key insights emerged:
- Complementary Signals: The original and literalisation methods agreed on only 70% of cases. In 16% of instances, only the literalisation method was correct, suggesting the two approaches capture different semantic nuances.
- Potential for Hybridization: A hypothetical system capable of perfectly selecting between the two methods could achieve ~79% accuracy.
- Semantic Transparency: For idioms where literal words are related to the figurative meaning (e.g., laid waste), the literalisation method can struggle as the original words already possess high context cohesion.
- Context Sensitivity: The literalisation method performed best with a narrow 5-word window, whereas the original method benefited from broader sentence-level context.
Limitations and Future Work
The authors identify several areas for improvement:
- Feature Expansion: The current model focuses on nouns, potentially missing critical context provided by verbs.
- Representation: Moving from word-to-word comparisons to compositional representations (e.g., sentence embeddings) could better capture figurative nuances.
- Automation: Future work could explore automatically extracting literalisations from dictionaries to reduce manual effort.
While the literalisation approach currently falls short of supervised performance, it introduces a valuable contrastive signal that highlights the complexity of capturing figurative context in unsupervised NLP.
Summary updated