Introduction
While large pre-trained language models (LLMs) have achieved remarkable success in constrained tasks such as machine translation and summarization, they often struggle in open-domain scenarios like storytelling or poetry. These tasks are characterized by high conditional entropy, where a single prompt can lead to an infinite variety of valid continuations. In these settings, standard auto-regressive models—which predict the next token based on surface-level patterns—frequently suffer from a lack of long-term coherence and the amplification of social biases present in their training data. This paper outlines a research program dedicated to moving beyond simple next-token prediction toward a framework of controllable text generation that prioritizes discourse structure, literary aesthetics, and ethical alignment.
Methodology: From Surface Patterns to Structural Control
The core of the proposed approach is the disentanglement of high-level planning from low-level surface realization. By treating creativity as a structural challenge rather than a purely linguistic one, the research introduces several key strategies:
1. Hierarchical Plan-and-Write Framework
Instead of generating text word-by-word from scratch, the system first generates a "structured plan" consisting of entities, events, and relations. A separate surface realization model then converts this outline into natural language. This hierarchical approach allows for human-in-the-loop collaboration at the planning stage and ensures the model maintains a consistent "roadmap" throughout long-form generation.
2. Temporal Modeling and Non-linear Narratives
To move beyond repetitive or boring linear plots, the research utilizes temporal prompts (e.g., tags like <before> and <after>) and rescoring models. These tools allow the system to re-rank naive plans based on their temporal logic and relevance, enabling the generation of sophisticated narrative devices such as flashbacks.
3. Insertion-based Generation
Traditional models are restricted to left-to-right generation. The authors propose an insertion-based framework that allows models to "fill in the blanks" or expand upon a plan non-monotonically. This flexibility is particularly effective for satisfying specific lexical constraints and refining text iteratively.
4. Controllable Decoding for Fairness
Rather than relying on the expensive fine-tuning of massive models, this research emphasizes post-processing techniques. By using small auxiliary models or specialized decoding algorithms, the system can guide the LLM toward fairer outputs during inference. This approach treats bias mitigation as a controllable generation problem, effectively "guardrailing" the model against toxic language and social stereotypes without sacrificing its creative potential.
Evaluation and Datasets
The research draws on a diverse array of data sources and benchmarks to validate these methods:
- Silver Standard Plans: Extracted from large corpora using information extraction (IE) tools to identify underlying event structures.
- Creative Benchmarks: Evaluation across domains including sonnets, poetry, and figurative language (puns, similes, and metaphors).
- Fairness Metrics: The development of automated metrics to detect social stereotypes and ad hominem attacks in open-domain dialogue systems.
Key Insights and Results
The research demonstrates that explicitly modeling the structure of a story solves the "drifting" problem common in long-form LLM outputs. Key findings include:
- Enhanced Coherence: Hierarchical models significantly outperform flat, auto-regressive models in maintaining plot consistency over long distances.
- Constraint Fidelity: Specialized attention mechanisms and insertion-based models show much higher fidelity to user-provided keywords, whereas traditional models often fail to include up to 20% of required plan elements.
- Zero-Shot Creativity: The framework successfully generates sonnets with discourse-level coherence and poetic features in zero-shot settings.
- Effective Bias Reduction: Controllable decoding methods successfully reduced the presence of harmful stereotypes in "chit-chat" dialogue without degrading the fluency of the conversation.
Limitations and Future Directions
Despite these advancements, several challenges remain. Insertion-based models currently require scaling to larger datasets to match the raw performance of standard auto-regressive LLMs. Furthermore, there is a persistent need for better integration of external common sense knowledge bases to ensure that creative outputs remain logically "sensible." Future work aims to create a virtuous cycle by using generation models to improve information extraction tools, which in turn will provide better structured data for future planning models.
Summary updated