SWAG: Storytelling With Action Guidance

Jonathan Pei, Zeeshan Patel, Karim El-Refai, Tianle Li

Generating long-form stories that are both coherent and engaging remains a significant challenge for Large Language Models (LLMs). While models like GPT-4 excel at local logic, they often produce "one-shot" narratives that feel flat or aimless over extended lengths. A new research paper from UC Berkeley, SWAG: Storytelling With Action Guidance, proposes a solution by treating storytelling as a search problem guided by a specialized "director" model.

The SWAG Framework

SWAG introduces a dual-model feedback loop that separates prose generation from narrative direction. The system consists of a "Story Model" that writes the text and an "Action Discriminator" (AD LLM) that selects high-level narrative beats—such as "add a plot twist" or "increase suspense"—to steer the writer.

The process operates in three iterative steps:

  1. Paragraph Generation: The Story Model writes a paragraph based on the initial prompt.
  2. Action Selection: The AD LLM analyzes the context and chooses the best "action" from a predefined list to continue the story.
  3. Iterative Writing: The Story Model is prompted to write the next paragraph while specifically incorporating the chosen action.

Training the "Director"

To develop an effective AD LLM, the researchers utilized a multi-stage alignment pipeline designed to handle long-context windows and complex narrative decision-making.

The SWAG training pipeline, illustrating the transition from preference data curation to DPO alignment.

The pipeline involves Supervised Fine-Tuning (SFT) on a massive dataset of long stories, followed by the collection of preference data where GPT-4 ranks the best narrative actions for specific snippets. Finally, Direct Preference Optimization (DPO) is used to align the model's choices with these high-quality rankings.

Narrative Steering and the "Suspense Bias"

By controlling the model at the intent level rather than the word level, SWAG provides a high-level roadmap that keeps stories dynamic. However, the researchers discovered a significant hurdle during data curation: the "Suspense Bias."

A bar chart showing GPT-4's overwhelming preference for selecting 'suspense' as a narrative action compared to other options.

As illustrated in the frequency data, GPT-4 has a massive bias toward choosing "suspense" regardless of context. To create a well-rounded storyteller, the authors had to rebalance their training data, forcing the model to explore other narrative tools like humor, mystery, and character development.

Key Results

The authors evaluated SWAG using GPT-4-Turbo as a judge and human annotators from Surge AI. The findings highlight the power of guided generation:

  • Outperforming Scale: Guided 7B parameter models (like Mistral-7B) significantly outperformed GPT-3.5-Turbo in "interest" and "surprise" categories.
  • Win Rates: In head-to-head matchups against standard end-to-end generation, SWAG Mistral-7B achieved a 68% win rate.
  • Human Preference: Human judges overwhelmingly preferred SWAG-generated stories, noting they felt more "alive" and less predictable than standard AI outputs.

Conclusion

SWAG provides a promising blueprint for AI content generation by framing storytelling as a collaboration between a "writer" and a "director." This modular approach demonstrates that smaller, open-source models can achieve state-of-the-art results when provided with a narrative compass, moving AI closer to truly crafted storytelling.

Summary updated

Abstract

Automated long-form story generation typically employs long-context large language models (LLMs) for one-shot creation, which can produce cohesive but not necessarily engaging content. We introduce Storytelling With Action Guidance (SWAG), a novel approach to storytelling with LLMs. Our approach reduces story writing to a search problem through a two-model feedback loop: one LLM generates story content, and another auxiliary LLM is used to choose the next best "action" to steer the story's future direction. Our results show that SWAG can substantially outperform previous end-to-end story generation techniques when evaluated by GPT-4 and through human evaluation, and our SWAG pipeline using only open-source models surpasses GPT-3.5-Turbo.
BibTeX
@inproceedings{pei2024swag,
  author = {Jonathan Pei and Zeeshan Patel and Karim El-Refai and Tianle Li},
  title = {{SWAG: Storytelling With Action Guidance}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.824},
}
Figure