Overcoming the Literal Bias in Generative Models

20 concepts14 papers

When a human hears the phrase "couch potato," they do not picture a literal tuber resting on a sofa. Instead, they activate a network of abstract and cultural associations—laziness, television, and physical inactivity. However, state-of-the-art generative models frequently suffer from a profound literal bias. When tasked with processing or visualizing figurative language, these models default to surface-level word meanings, rendering a physical potato on a couch rather than the intended idiomatic concept.

This challenge spans across both textual and multimodal artificial intelligence. To build models that truly understand human expression, researchers are developing frameworks to move beyond surface-level patterns and capture the symbolic essence of language.

Why Models Fail: The Alt-Text Trap

To understand why text-to-image diffusion models fail at figurative language, we must look at how they are trained. Standard text-to-image pipelines rely heavily on massive datasets of images paired with descriptive alt-text (e.g., "a red apple on a wooden table"). This training paradigm creates a rigid, direct mapping between specific tokens and concrete physical objects.

As noted by Hassan Shahmohammadi et al. in their work on ViPE: Visualise Pretty-much Everything, this direct mapping leaves models unequipped to handle abstract or metaphorical language. Because the training data lacks examples of symbolic representation, the model assumes every word corresponds to a literal object.

This architectural limitation manifests as over-literalization. For instance, when an abstract noun is paired with a concrete verb (such as "time flies"), diffusion models struggle because they attempt to render the physical actions of the words rather than the underlying concept, as shown by Mohammed Khaliq et al. in Comparison of Image Generation Models for Abstract and Concrete Event Descriptions.

The problem is equally severe in vision-language models (VLMs) evaluated on understanding tasks. In the IRFL benchmark introduced by Ron Yosef et al., state-of-the-art models failed spectacularly at matching figurative phrases like "a sinking ship" to their contextual meanings, achieving a mere 22% accuracy compared to 97% for humans. The models consistently prioritize surface-level text over abstract intent.

[Figurative Input] ---> "Couch Potato"
                             |
                             v
[Standard Model]  ---> Maps tokens directly to physical objects
                             |
                             v
[Literal Output]  ---> A literal potato sitting on a sofa

The Solution: Unpacking and Conceptual Elaboration

To bypass this literal bias, researchers have introduced a powerful paradigm shift: conceptual elaboration. Instead of feeding a figurative phrase directly into a generative model, frameworks insert an intermediate translation layer that "unpacks" the abstract metaphor into a concrete, visually renderable scene.

[Figurative Input] ---> "Couch Potato"
                             |
                             v
[Intermediate LLM] ---> Unpacks abstract meaning into a concrete scenario:
                        "A tired person sitting on a sofa watching television"
                             |
                             v
[Diffusion Model]  ---> Renders the intended symbolic scene accurately

Chain-of-Thought Visual Metaphors

In I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphors, Tuhin Chakrabarty et al. demonstrated that Large Language Models (LLMs) can act as an ideal bridge. By using Chain-of-Thought prompting, the LLM explicitly deconstructs a metaphor into its implicit properties before generation. Instead of letting a diffusion model guess how to draw a abstract concept, the LLM generates a detailed description of a physical scene that symbolically evokes the target emotion or idea.

Visual Elaborations

This is the core mechanism behind ViPE, developed by Hassan Shahmohammadi et al.. By training a lightweight model specifically to translate abstract lyrics and poetic text into detailed visual descriptions, the authors enabled standard diffusion models to produce meaningful imagery for complex concepts that would otherwise result in chaotic, literal nonsense.

Scenario Generation

A similar approach proved effective for understanding noun compounds. Sinan Kurtyigit et al. showed that prompting models to generate descriptive scenarios rather than relying on keyword matching significantly improves how AI handles idiomatic expressions like "couch potato" or "climate change," mitigating the false positives that plague traditional image retrieval systems.

Structural and Attentional Alignment

Unpacking the text into a descriptive prompt is only the first step. The downstream generation process must also be guided to ensure that the relationship between the metaphorical elements is preserved.

In GOME: Grounding-based Metaphor Binding With Conceptual Elaboration For Figurative Language Illustration, Linhao Zhang et al. tackled this by analyzing the rhetorical structure of metaphors (identifying the tenor and the vehicle). GOME uses an attentional binding technique during the image diffusion process to force the model to merge these concepts structurally, preventing the model from simply rendering a disjointed collection of literal objects.

When dealing with highly abstract creative texts like poetry, the PoemToPixel framework, designed by Koustava Goswami et al., combines LLM summarization with a specialized key-element extraction algorithm called PoeKey. This ensures that the generated image captures the overarching emotional "soul" of the poem rather than getting bogged down by individual literal words within the verses.

The Pervasiveness of Literal Bias

The struggle with non-literal reasoning is not exclusive to image generation; it represents a fundamental bottleneck in pure text processing as well.

  • Poetry Summarization: In Unveiling the Essence of Poetry: Introducing a Comprehensive Dataset and Benchmark for Poem Summarization, Ridwan Mahbub et al. demonstrated that standard language models frequently fail to summarize verse. Instead of capturing symbolic meaning, they interpret metaphors as factual events or resort to copy-pasting lines.
  • Content Moderation: This limitation has real-world consequences for safety. Adarsh Kumar et al. found that hostility detection models in low-resource languages like Hindi frequently misclassify sarcastic or idiomatic social media posts because they cannot separate literal word choices from non-literal, cultural intent.
  • Creative Synthesis: Whether generating creative similes (Tuhin Chakrabarty et al.) or interpreting internet memes (Yingyi Xue et al.), models must be explicitly trained or prompted to look past surface-level accuracy.

As benchmarks expand to include low-resource languages and complex rhetorical devices (Paria Khoshtab et al.), overcoming the literal bias remains a critical frontier. True linguistic competence requires models to stop looking at the world through a purely literal lens and begin understanding the abstract relationships that define human communication.

Key Foundational Work

Generating similes effortlessly like a Pro: A Style Transfer Approach for Simile Generation (2020) — contributed 1 concept elements to this direction.
Deep-speare: A joint neural model of poetic language, meter and rhyme (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_figurative_language_interpretation_gener_91ad5999,
  title = {figurative language interpretation (generative literalism)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/figurative-language-interpretation-generative-literalism/rdr_91ad599950d34332816f0b115485c1b1},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_figurative_language_interpretation_gener_91ad5999_v1,
  title = {figurative language interpretation (generative literalism)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/figurative-language-interpretation-generative-literalism/rdr_91ad599950d34332816f0b115485c1b1/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake