Introduction
Large Language Models (LLMs) have revolutionized natural language generation, producing fluent and stylistically diverse prose. However, they frequently struggle with the "common sense" logic required to maintain long-term narrative coherence. Without a grounded world model, LLMs are prone to hallucinations—characters might teleport, items appear without cause, or the plot may devolve into repetitive, illogical sequences. This research addresses the challenge of narrative coherence by introducing a hybrid framework that separates logical structure from linguistic expression.
Methodology: The Logic-Prose Pipeline
The proposed approach utilizes Automated Planning to provide a "logical skeleton" for the LLM. By constraining the generative process within a validated sequence of actions, the system ensures that the story adheres to the rules of its world.
1. Symbolic Planning with PDDL
The story's world is defined using the Planning Domain Definition Language (PDDL). This involves two primary components:
- Domain File: Defines the "physics" of the story world, including possible actions, their preconditions (what must be true for an action to occur), and their effects (how the world state changes).
- Problem File: Specifies the initial state of the world and the desired goal state.
An automated planner processes these files to generate a valid plan—a sequence of logical steps (e.g., walk_to_forest, pick_up_bow) that transitions the world from the start to the goal.
2. LLM as Narrator
The validated plan is then fed to the LLM step-by-step. Instead of a vague prompt, the LLM receives specific logical actions to translate into descriptive narrative. To maintain consistency, the system employs "story-agnostic" prompts—style guidelines and background information that remain constant across different stories, helping the model recognize the pattern of turning discrete actions into a cohesive story.
Experimental Setup and Datasets
The study evaluated this pipeline by adapting three classic children's stories into planning domains: The Way Home for Wolf (Bright and Field 2020), Robin Hood and the Golden Arrow (San Souci and Lewis 2010), and The Paper Bag Princess (Munsch and Martchenko 1980). These stories were selected for their clear, linear structures and simple vocabulary, providing a controlled environment for testing the planning-to-prose transition.
Key Insights
- Scaffolding for Coherence: The plan acts as essential scaffolding. By forcing the LLM to follow a validated sequence, the model is prevented from violating the story's internal logic.
- State-Space Reasoning: Storytelling is framed as a state-space problem where actions are governed by preconditions and effects ($PRE(a) \to s'$). While LLMs struggle with this mathematical reasoning, symbolic planners excel at it.
- Few-Shot Learning: The research found that providing the LLM with at least two examples of "action-to-story" pairs significantly improved its ability to follow the plan's structure effectively.
Results and Evaluation
The hybrid approach was evaluated through both quantitative and qualitative measures:
- Quantitative Alignment: Using Part of Speech (PoS) tagging, the author confirmed that nearly all nouns (characters/locations) and verbs (actions) defined in the logical plan were successfully reflected in the LLM-generated text.
- Human Evaluation: In blind studies, participants consistently rated the Plan + LLM stories higher than those generated by an LLM alone. Evaluators specifically noted that the plan-guided stories were better at avoiding repetition and that events occurred in a more plausible order.
Limitations and Future Directions
While this work establishes a robust foundation for logical storytelling, several avenues for future research remain:
- Temporal Constraints: Integrating Linear Temporal Logic (LTL) could allow for more complex requirements, such as "safety" (e.g., "The Batmobile must stay connected to Alfred while Batman is away") and "liveness" (e.g., "Batman must eventually return to the Batcave").
- Non-Determinism: Moving beyond linear stories to "Choose-Your-Own-Adventure" styles where actions have uncertain outcomes.
- Scaling Complexity: Adapting this framework for multi-threaded, complex novels remains a significant challenge for future symbolic-generative hybrids.
Summary updated