Mitigating Narrative Homogeneity in Text Generation

40 concepts35 papers1 proposal

When you ask a modern language model to write a story, a poem, or even a product description, you often get a sense of déjà vu. The text is grammatically flawless and highly coherent, yet it feels profoundly boring. Characters follow predictable paths, jokes rely on a limited rotation of tropes, and data descriptions fall back on rigid, repetitive templates.

This is the "diversity crisis" of generative text. Standard autoregressive models are trained to maximize the likelihood of the next token. While inference-time techniques like top-$p$ or temperature sampling inject randomness, they do not fundamentally alter the model's underlying bias toward the "path of least resistance"—the safe, high-frequency statistical averages of the training data. To build systems capable of genuine creativity, researchers are developing methods to break this probability trap.

Quantifying the Homogeneity Trap

To fix narrative monotony, we first need to measure it. As Weijia Xu et al. demonstrate, large language models frequently suffer from a severe lack of plot diversity, repeating identical narrative arcs and predictable tropes across entirely different prompts. To quantify this, the authors introduced the Sui Generis score, an automatic metric for narrative uniqueness. Their evaluation highlights a stark truth: human-written stories possess a level of structural originality that state-of-the-art models fail to replicate.

This issue spans across diverse text generation domains:

  • Educational Content: Zichao Wang et al. identify a "memorization trap" in math word problem generation, where models replicate existing training examples rather than synthesizing novel problems.
  • Interactive Environments: In generating text-based fantasy worlds, Angela Fan et al. observe that models naturally default to formulaic, templated descriptions that break user immersion.
  • Specialized Annotations: Even in scientific domains, such as generating natural language functional descriptions for proteins, Yaoyao Xu et al. note that insufficient architectural constraints lead to repetitive and uninformative outputs.
  • Writing Assistants: In automated sentence completion tasks, Shuming Shi et al. show that standard language models produce dull, repetitive prose, limiting their utility for creative human collaboration.

Disentangling Content from Form

Why do models get stuck in these repetitive loops? A primary reason is that standard architectures conflate what is being said (the semantic content) with how it is being said (the stylistic template). When these two dimensions are tangled, the model defaults to the most common syntactic structure available for that topic.

One prominent solution is latent space disentanglement, which forces the model to treat structure and content as independent variables.

Standard Generation:
[Prompt/Data] ───> [Tangled Latent Space] ───> [Defaults to Safe/Generic Output]

Disentangled Generation (e.g., VTM / MixPoet):
                   ┌───> [Content Subspace] ───┐
[Prompt/Data] ───┤                           ├───> [Diverse & Varied Output]
                   └───> [Style/Template Space]┘

For instance, Rong Ye et al. developed the Variational Template Machine (VTM) for data-to-text generation. VTM explicitly separates content from templates within a Variational Autoencoder (VAE) latent space. By doing so, the model can map a single structured data table to a wide variety of sentence structures without requiring explicit human-annotated templates.

In creative writing, Maosong Sun et al. applied a similar intuition to poetry generation with MixPoet. They split a VAE's latent space into controllable subspaces representing different historical contexts or life experiences. By "mixing" these latent factors, the model can generate stylistically distinct poems from a single thematic prompt, overcoming intra-topic monotony.

Similarly, in translating structured database queries into natural language questions, Xiaojing Yu et al. utilize flexible templates and a joint semantic space to prevent models from generating rigid, grammatically awkward repetitions.

Modular and Hierarchical Synthesis

Another powerful intuition for driving diversity is to prevent the model from generating an entire narrative in a single, unconstrained pass. Left to its own devices, an LLM generating text token-by-token will often choose the most generic next step. However, if the model is forced to construct a structural "blueprint" first, diversity can be enforced at the conceptual level before sentences are ever formed.

Yan Ma et al. implemented this strategy via Modular Story Premise Synthesis (MoPS). Instead of generating a story directly from a prompt, MoPS deconstructs the premise into a nested dictionary of hierarchical modules: theme, background, persona, and plot. Systematically synthesizing and recombining these modules bypasses repetitive tropes and ensures open-ended narrative depth.

A parallel structural decomposition is used by Xinran Zhao et al. for automated scientific ideation. Inspired by historical mechanical thinking machines, the authors decompose research concepts into three explicit axes: Theme, Domain, and Method. Recombining these distinct axes ensures that generated ideas span a wide, diverse space rather than clustering around generic machine learning trends.

To inject life into narrative characters, Kyeongman Park et al. propose a character-centric imagination framework. By forcing the model to build deep, non-standard character profiles before generating the text, the downstream plot naturally veers away from the path of least resistance.

Altering the Decoding and Learning Landscape

Finally, researchers are addressing homogeneity directly at the token-sampling and optimization levels.

Syntactic Guidance

Standard sampling methods look at token probabilities in isolation, which often leads to choosing common adjectives or nouns. To force linguistic variety, Zhixian Yang et al. proposed a Part-of-Speech (POS) guided softmax and sampling method. The model first predicts a syntactic category (e.g., verb, adjective) and then samples a word from within that specific category, forcing the model to explore diverse grammatical structures it would otherwise skip.

In-Context Diversification

At the prompting level, Tianhui Zhang et al. introduce In-Context Diversification (ICD) for commonsense reasoning. This approach uses a two-step prompting strategy combined with a diversity-aware sampling algorithm, allowing LLMs to generate multiple valid, creative perspectives without requiring any model fine-tuning.

Curriculum Learning

When training models for highly complex creative tasks like humor, standard fine-tuning often fails because the model defaults to a limited rotation of memorized jokes. To solve this, Yang Chen et al. utilize a multi-stage curriculum learning framework. The model first masters underlying linguistic structures before learning to layer on humor, using a triplet preference optimization to ensure the model learns context-aware incongruity rather than static joke templates.

Contextual Conditioning

Homogeneity also stems from an information deficit; if a model only has a generic prompt to go on, it produces a generic response. By grounding generation in external knowledge or explicit stylistic parameters, the output space expands.

  • Qibin Chen et al. combat monotony in e-commerce descriptions by fusing raw product attributes with external knowledge bases and specific user personas.
  • Jiayi Zhang et al. introduce a "Locate&Gen" framework that explicitly identifies where a plain text passage can be vivified and injects novel, contextually relevant similes, breaking up safe prose with aesthetic variety.

Key Foundational Work

Generating Interactive Worlds with Text (2020) — contributed 1 concept elements to this direction.
The Amazing World of Neural Language Generation (2020) — contributed 1 concept elements to this direction.
Towards Knowledge-Based Personalized Product Description Generation in E-commerce (2019) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_narrative_homogeneity_70c2059b,
  title = {narrative homogeneity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/narrative-homogeneity/rdr_70c2059bbe9b4c3d9554477f587fe867},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_narrative_homogeneity_70c2059b_v1,
  title = {narrative homogeneity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/narrative-homogeneity/rdr_70c2059bbe9b4c3d9554477f587fe867/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake