TL;DR
Researchers have developed a new framework for interactive storytelling that prevents Large Language Models (LLMs) from wandering aimlessly or ending stories too abruptly. By introducing a hidden "story goal" and a two-stage prompting pipeline ("low" for challenges and "rise" for resolution), the system generates cohesive fairy tales that balance structured narrative arcs with player agency.
What Problem Does This Paper Address?
While LLMs are excellent at generating local text, they struggle with long-term narrative structure. In interactive "mixed-initiative" storytelling—where a user and an AI take turns—plots often become "emergent" but incoherent. Without a clear ending in sight, stories either loop indefinitely (the "AI Dungeon" problem) or fail to provide a satisfying emotional payoff.
The authors aim to "tame the plotline" by ensuring stories are:
- Structured: They have a beginning, middle, and a definitive resolution.
- Goal-Driven: The protagonist works toward a specific objective.
- Interactive: The user can influence the plot through actions without breaking the narrative arc.
Methods
The authors propose a pipeline that divides the story generation into two distinct phases to enforce a classic "rising" emotional arc:
- The "Low" Stage: For the first $n$ passages, the system prompt explicitly instructs the LLM to describe challenges and ensures the protagonist never reaches their goal. This builds tension and prevents "premature" happy endings.
- The "Rise" Stage: After a set number of turns, the prompt switches. The LLM is now encouraged to bring the protagonist closer to their goal and check for completion.
- Prompt Ensemble: The pipeline uses a mix of zero-shot and few-shot prompts to generate four elements: the Setting, the Passage (story chunk), Suggested Actions (for the user to choose from), and the Epilogue.
- Mixed-Initiative Loop: After each passage, the model suggests three actions. The user can pick one or type their own, ensuring they have agency over the direction of the plot.
Datasets
The study did not involve training on a new massive corpus but rather used a hand-curated evaluation set:
- Input Pairs: 100 distinct combinations of 10 protagonists (e.g., a unicorn, a wizard, a cat) and 10 goals (e.g., "defeat an evil dragon," "return the stolen sun").
- Evaluation Data: 100 full "playthroughs" were generated for each of the four models tested (Saiga-Llama3-8B, a proprietary 29B model, Mixtral-8x7B-Instruct, and GPT-4o) in both English and Russian.
Key Insights and Ideas
- System Prompts as "Directors": The most important finding is that narrative structure can be effectively managed simply by swapping system prompts mid-story. You don't need complex architectural changes; you just need to tell the model when it's allowed to let the hero win.
- The "Modern Fairy Tale" Struggle: LLMs struggled to incorporate "modern reality" (e.g., setting a fairy tale in a city with skyscrapers) unless specifically prompted. They naturally gravitate toward "once upon a time" tropes.
- Goal Achievement as a Stopping Condition: By treating the "goal reached" state as a trigger for the epilogue, the authors solved the problem of stories that never end.
- Sentiment as a Proxy for Arc: The researchers used sentiment analysis (via RuBERT) to prove their method works. They successfully visualized a "rising" arc where the beginning of the story has lower (more challenging) sentiment and the end has high (triumphant) sentiment.
Key Results
- Model Performance: GPT-4o and the proprietary 29B model were the top performers, achieving Coherence scores of 0.94–1.00.
- Consistency: GPT-4o showed the highest logical consistency (0.96 in Russian, 0.94 in English), meaning it rarely forgot character names or contradicted previous events.
- Human vs. AI Evaluation: The authors found high agreement between human annotators and GPT-4o acting as an evaluator, suggesting that LLMs are becoming reliable tools for measuring creative writing quality.
- Emotional Arcs: Quantitative sentiment tracking confirmed that the two-stage prompting successfully created a "rising" emotional trajectory across all tested models.
Limitations and Future Work
- Fixed Length: The current experiments capped stories at 8 passages. The authors want to explore "rise-fall" or "rise-fall-rise" arcs for longer, more complex narratives.
- Hyperparameters: The study used default model settings; the authors suggest that a dedicated hyperparameter search might further improve story "creativity" vs. "logic."
- Safety vs. Engagement: The authors noted a "cautious" bias in models—strict safety prompts can sometimes make stories less engaging by avoiding all conflict.
- Secondary Goals: Future versions might introduce secondary "side quests" to make the interactive experience feel more like a full-fledged RPG.
Summary updated