Consider the phrase "spilling the beans." If a chef drops a container of legumes in a kitchen, the phrase is entirely literal and compositional—its meaning is derived directly from its parts. If a politician reveals a classified secret to the press, the phrase is figurative and non-compositional.
To a human reader, distinguishing between these two meanings is trivial because we instantly evaluate the surrounding environment. For natural language processing systems, however, this task represents a profound challenge known as token-level literal-figurative disambiguation. As Diab et al. point out, it is not enough for a model to possess type-level knowledge—knowing that a phrase can be an idiom. A language model must perform token-level analysis to determine whether a specific instance of an expression is meant literally or figuratively.
The Challenge of Neutral Contexts
Early computational approaches to figurative language relied on detecting semantic anomalies or selectional restriction violations. For example, in the sentence "The walls have ears," a model might flag the expression as figurative because inanimate objects cannot possess biological organs.
However, Markert et al. demonstrated that a significant portion of figurative language occurs in entirely neutral contexts. In these sentences, both the literal and figurative interpretations are grammatically and semantically valid. If someone says, "They broke the ice," they could be describing an Arctic research vessel or a social interaction. Because there is no explicit logical conflict or error to trigger a non-literal interpretation, models cannot rely on simple anomaly detection. They must instead reason about how the components of the phrase weave into the broader discourse.
The Intuition of Lexical Cohesion
How can an algorithm determine the correct sense of an ambiguous phrase without human labels? The breakthrough came from leveraging the principle of lexical cohesion.
If a phrase is used literally, its individual component words will naturally share strong semantic relationships with the surrounding text. In an unsupervised framework developed by Li et al., the system measures how well the literal definitions of the component words link to the rest of the sentence.
Literal Cohesion:
[The chef] ---> cooked ---> [spilled] ---> the ---> [beans] ---> on the ---> [floor]
|___________________________^______________________| |
|___________________________________________________________________________|
(Strong semantic ties between "chef", "spilled", "beans", and "floor")
Figurative Disconnect:
[The politician] ---> confessed ---> [spilled] ---> the ---> [beans] ---> to the ---> [press]
|_______________| ? ? |_______________|
|________________________________________________________|
("Beans" and "spilled" have no semantic cohesion with "politician" or "press")
When the phrase is figurative, words like "beans" or "ice" become semantic outliers to the immediate physical context. By modeling these relationships using distributional semantics, Liu et al. showed that you can construct a "literal usage profile" for expressions. If a sentence deviates sharply from this profile, it serves as a strong mathematical signal that the phrase is operating non-compositionally.
Neural Representation and Attention Routing
With the advent of deep learning, the focus shifted from explicit lexical cohesion metrics to dense, contextualized vector spaces. Instead of calculating word-to-word similarity scores manually, sequence labeling models utilize architectures like BiLSTMs and Transformers to capture full sentential context implicitly. Choi et al. demonstrated that leveraging these contextualized layers allows models to effectively map whether a word represents a concrete physical action or an abstract concept.
To handle the stark binary choice between literal and figurative tracks, modern architectures build explicit routing mechanisms into the network. For instance, the PIER model introduced by Zeng et al. utilizes an attention fusion mechanism.
[ Input Context ]
|
(Attention Fusion)
/ \
v v
[Literal Representation] [Idiomatic Representation]
\ /
v v
[ Dynamic Contextual Routing ]
|
v
[ Final Token Meaning ]
Instead of forcing a single vector to represent both possibilities, the network computes parallel representations for the literal composition and the idiomatic meaning. The attention mechanism then dynamically weights and routes these representations based on the surrounding environment, allowing the model to smoothly transition between literal and figurative interpretations without degrading its general language understanding.
To make these boundaries even sharper, researchers use specialized optimization objectives. He et al. introduced an adaptive contrastive triplet loss during fine-tuning. This loss function mathematically forces the model to push the vector space representation of a figurative phrase far away from its literal counterpart, ensuring that the model learns the subtle, contextual boundaries that separate an action from a metaphor.
Beyond Surface Shortcuts
While large language models (LLMs) perform exceptionally well on standard benchmarks, recent evaluations reveal that they often rely on superficial pattern matching rather than deep pragmatic reasoning. Mi et al. introduced the DICE (Dataset for Idiomatic Contrastive Evaluation) benchmark to expose these "surface shortcuts." They found that if an LLM frequently encounters a phrase like "kick the bucket" in idiomatic contexts during pre-training, it develops a strong prior bias to always interpret it figuratively, failing when confronted with a balanced, literal context.
True figurative understanding—encompassing idioms, metonymy, and hyperbole—is fundamentally pragmatic. As noted in a study on Probing for Hyperbole in Pre-Trained Language Models, identifying rhetorical exaggeration requires integrating world knowledge and speaker intent, which is encoded primarily in the final layers of language models at the sentence level rather than the word level.
To anchor this reasoning, researchers have begun integrating structured external knowledge. By pairing language models with commonsense knowledge graphs like IEKG, developed by Zeng et al., models gain access to explicit "if-then" pragmatic assertions (e.g., If Person X "breaks the ice", then they are "starting a social interaction"). This explicit grounding ensures that the model can navigate token-level ambiguity reliably, knowing exactly when a ship is breaking physical ice versus when a person is breaking a social one.
Key Foundational Work
Construction of an Idiom Corpus and its Application to Idiom Identification based on WSD Incorporating Idiom-Specific Features (2008) — contributed 1 concept elements to this direction.
Metonymy Resolution as a Classification Task (2002) — contributed 1 concept elements to this direction.