Rodrigo Agerri, John Barnden, Mark Lee, Alan Wallington
STEP 2008 Conference Proceedings
Abstract
This paper addresses the historical isolation of metaphor research from mainstream applied Natural Language Processing (NLP). The authors propose the Recognizing Textual Entailment (RTE) task as a standardized evaluation framework to bridge this gap. By demonstrating a statistically significant performance deficit in current NLP systems when encountering metaphorical language, the authors argue for a hybrid processing approach—combining shallow linguistic analysis with deep, commonsense reasoning—and the development of large-scale, annotated metaphorical datasets.
Introduction: The Ubiquity of Metaphor
Metaphor is often perceived as a poetic flourish, yet linguistic analysis reveals it is ubiquitous in everyday communication, appearing approximately once every 17 words. Despite its prevalence, applied NLP has largely marginalized metaphor resolution. The authors identify a "chicken-and-egg" problem: applied NLP avoids metaphor because it introduces context-sensitivity and indeterminacy that current systems cannot handle, while metaphor research remains stalled at a small scale because it lacks the shared, large-scale datasets and standardized evaluation metrics necessary for modern machine learning.
To move the field forward, this paper proposes a "RoadMap" that integrates metaphor resolution into the Recognizing Textual Entailment (RTE) framework, providing a clear metric for semantic understanding.
The RTE Framework for Metaphor Resolution
The core of the proposal is to shift the focus from mere metaphor detection to metaphor understanding. In the RTE framework, a system must determine if a metaphorical Text (T) entails a literal Hypothesis (H).
For example:
- T: "The idea was incubated in his mind."
- H: "He developed the idea."
If a system can correctly identify this entailment, it demonstrates a functional understanding of the metaphor's meaning. This approach treats metaphor as a matter of "commonsense reasoning" rather than formal logic, allowing for the plausible, non-certain inferences that characterize human communication.
A Multi-Tiered Processing Spectrum
The authors suggest that metaphor resolution requires an integrated spectrum of algorithms, ranging from shallow linguistic cues to deep cognitive modeling:
1. Shallow Processing and Affect
Using robust parsing (such as the Rasp system) and lexical resources like WordNet, systems can identify syntactic signals of metaphor. Furthermore, the authors utilize tools like E-Drama to detect "affect" or emotional value. For instance, identifying that the word "pig" applied to a person implies an unpleasant attribute, even if the specific semantic mapping is not fully resolved.
2. Deep Processing (ATT-Meta)
For complex metaphors, the authors propose the ATT-Meta approach, which involves "source-domain reasoning." The system temporarily treats the metaphor literally—reasoning about the biological requirements of incubation (heat, protection, time)—and then uses View-Neutral Mapping Adjuncts (VNMAs) to translate those literal conclusions back into the target domain of ideas and mental processes.
3. Knowledge Integration
To scale these methods, the authors advocate for supplementing hand-coded rules with common-sense resources like ConceptNet 3.0 and web-mined knowledge to provide the necessary breadth for real-world applications.
Empirical Evidence: The "Metaphor Deficit"
The necessity of this framework is underscored by the authors' analysis of existing NLP benchmarks. By examining the performance of top-tier systems on the RTE-1 and RTE-2 datasets, they uncovered a "metaphor deficit."
In an analysis of RTE-2 across ten different system runs, seven systems showed a statistically significant drop in performance (p < 0.05) when handling metaphorical examples compared to literal ones. In one specific test case from RTE-1, only one of the four top-performing systems correctly handled the metaphorical entailment. These results provide quantitative proof that metaphors are a significant blind spot in current semantic processing.
Conclusion and the RoadMap
The paper concludes by outlining a path forward for the NLP community. The authors are currently developing a 242,000-word metaphor-oriented corpus derived from the British National Corpus (BNC). The ultimate goal is the creation of a large-scale, public, annotated dataset specifically designed for RTE evaluation that accounts for various dimensions of metaphor, including target/source subject matter, idiomaticity, and productive variation (e.g., understanding that "the foulest recesses of the mind" is a variation of a common metaphor).
By adopting the RTE framework, metaphor research can move from "toy" examples to a robust, empirical discipline that enhances the overall depth and accuracy of natural language understanding.
Summary updated