Most of the time, language is compositional: if you know what "red" means and what "apple" means, you know what a "red apple" is. Neural networks excel at this additive logic. However, human language is famously peppered with "pains in the neck"—multiword expressions (MWEs) like "cut corners" or "on the rocks" where the meaning of the whole is entirely detached from the sum of its parts.
Recent research reveals that even the most advanced Large Language Models (LLMs) suffer from a persistent literal bias. When faced with non-compositional language, these models default to the dictionary definitions of individual words, leading to "semantic hallucinations" and catastrophic failures in reasoning.
The Compositionality Trap
The core of the problem lies in how models are trained. Because they learn from massive corpora where words like "kick" and "bucket" usually appear in their literal senses, their internal vector representations are heavily skewed toward these common meanings. Vered Shwartz argues in A Long Hard Look at MWEs in the Age of Language Models that this "compositionality bias" makes it difficult for models to decouple literal word meanings from figurative ones. Even when a model correctly identifies a phrase as an idiom, its internal "thought process" remains anchored to the literal components.
This isn't just a minor glitch; it acts as contextual poison. Harsh Jhamtani and colleagues found in Investigating Robustness of Dialog Models to Popular Figurative Language Constructs that a single misunderstood idiom can degrade a conversation for multiple turns. If a user says a relationship is "on the rocks," a biased model might suddenly start generating responses about geology, losing the thread of the social interaction entirely.
Why Models Struggle to "Turn Off" Literalism
One might assume that simply telling a model to "be literal" or "be figurative" would solve the issue. However, research by Akhila Yerukola in Is the Pope Catholic? Yes, the Pope is Catholic. Generative Evaluation of Intent Resolution in LLMs shows that models often "know" the underlying meaning of a trope but still respond to the surface-level words. This suggests a gap between linguistic decoding and pragmatic application.
Furthermore, metaphorical structures are so deeply embedded in training data that models struggle to avoid them. A study on ChatGPT-Written Speeches found that even when explicitly instructed to produce purely literal text, LLMs continue to use conceptual metaphors, suggesting that figurative language is not an optional "layer" but a fundamental part of how these models organize information.
Bridging the Gap: Literalization and Knowledge Bases
To fix this, researchers are moving away from pure statistical pattern matching toward more "figurative-aware" architectures and strategies:
- Literalization Strategies: Harsh Jhamtani demonstrated that automatically "translating" figurative speech into literal terms before the model processes it can significantly improve response quality without any retraining.
- Specialized Knowledge Bases: Shuang Li introduced IDIOMKB, a framework that uses a "Knowledge Base Chain-of-Thought" (KB-CoT) prompting strategy. This allows smaller models to achieve the idiomatic accuracy of much larger ones by explicitly looking up non-compositional meanings.
- Synthetic Augmentation: Since idioms are relatively rare in standard datasets, researchers like Doğukan Arslan are using LLMs to generate massive synthetic idiom corpora. This data helps train smaller models to recognize the tension between literal sentiment and figurative intent, preventing the "catastrophic failures" seen in tasks like sentiment analysis.
From Word-by-Word to Phrase-Based Understanding
The challenge of non-compositionality is not new. As far back as 1980, Robert Wilensky and Yigal Arens proposed PHRAN, an analyzer that treated phrases as holistic units rather than sequences of words.
Modern research is essentially rediscovering this need. Whether through PIE (a parallel idiomatic corpus by Suma Bhat and others) or ID10M (a multilingual idiom identification framework), the goal is the same: to move neural models past their literal-mindedness and toward a truly human-like grasp of the "unspoken" meanings in our speech.