Narrative Generation in the Wild: Methods from NaNoGenMo

Judith van Stegeren, Mariët TheuneACL 2019

TL;DR

This paper analyzes the methods used by participants in NaNoGenMo, an online event challenging people to generate 50,000-word novels, to achieve narrative coherence. It categorizes these methods, from templating and Markov chains to simulation and deep learning, and discusses their effectiveness in creating long, coherent stories.

What Problem Does This Paper Address?

Generating long, coherent narratives is a significant challenge in text generation. As story length increases, maintaining narrative coherence—where all story elements are interrelated in a meaningful way—becomes increasingly difficult, especially for neural models. This paper investigates how a community of amateur and hobbyist developers tackles this problem in a real-world, large-scale event, NaNoGenMo, which pushes the boundaries of generated text length beyond typical academic research.

Methods

The authors analyzed 61 completed NaNoGenMo 2018 projects, focusing on the 35 that generated novel-like text. They categorized the text generation methods used, including:

  • Templating: Using pre-defined structures and filling in blanks, offering high control but potentially leading to repetitive output. Some projects automated template creation using POS tagging.
  • Markov Chains: Statistical models trained on corpora to predict the next word, offering a more creative but often less coherent output.
  • Remixing: Combining or modifying existing texts, ranging from simple word substitution to more complex transformations.
  • Machine Translation: Using translation techniques, including cross-modal translation (image to text), to generate novel content.
  • Deep Learning: Employing LSTMs and other neural networks, which often produced grammatically incorrect but stylistically evocative text.

For narrative coherence, the paper identified four main approaches:

  • High-level Specification: Hard-coding a narrative structure using domain-specific languages (like Samovar) to define the story's plot and events.
  • Hard-coded Narrative Elements: Adding a coherent beginning and/or ending to otherwise incoherent generated text to frame the narrative.
  • Simulation: Creating a world with rules and characters whose interactions generate events and drive the narrative, often resulting in journal-like formats.
  • Evoking a Narrative: Using techniques like limiting the text's subject matter or employing recognizable styles to encourage readers to infer coherence, leveraging "charity of interpretation."

Datasets

The paper mentions that participants primarily used publicly available text data, such as:

  • Project Gutenberg
  • The Internet Archive
  • Darius Kazemi's Corpora (a collection of word lists)
  • User-created corpora from online resources (subtitles, reports, etc.)
  • Specific datasets for deep learning models, like crowdsourced Dungeons & Dragons character biographies and CreepyPasta stories.

No new datasets were introduced in this paper.

Key Insights and Ideas

  • NaNoGenMo as a "Wild" Lab: The event provides a unique, large-scale, and diverse dataset of practical text generation approaches outside of academic constraints, offering insights into what works "in the wild."
  • Coherence is Hard, but Achievable in Many Ways: While neural models struggle with long-term coherence, participants employed a variety of creative strategies, from explicit structural specification to subtle reader-guided interpretation, to achieve it.
  • Form Over (Sometimes) Content: Techniques like hard-coded beginnings/endings or evoking narrative through style demonstrate that the perception of coherence can be as important as strict logical flow, especially for long texts.
  • Community Influence: The availability of tools like Tracery and Markovify, and the community's own surveys, highlight the role of accessible libraries and shared knowledge in driving specific technical choices.
  • Trade-offs are Inherent: Most methods involve a trade-off between control and creativity, or between coherence and novelty. For instance, templating offers control but can be repetitive, while Markov chains offer novelty but can lack coherence.

Key Results

  • Out of 61 completed NaNoGenMo 2018 projects, 35 generated novel-like text.
  • Only 14 of these projects exhibited explicit narrative structure and coherence.
  • Python and JavaScript were the most popular programming languages, likely due to the availability of NLP/NLG libraries.
  • The paper provides qualitative analysis and examples of creative projects for each method discussed, illustrating their strengths and weaknesses.

Limitations and Future Work

  • Manual Analysis: The analysis of narrative coherence was informal and manual, performed by the first author.
  • Scope: The paper provides a high-level overview and brief discussion of example projects, suggesting that readers refer to the NaNoGenMo repository for more detailed examples.
  • Future Work: The authors implicitly suggest that further research could explore how to combine different approaches to leverage their respective strengths. They also note that the NaNoGenMo data is available for further analysis.

Summary updated

BibTeX
@inproceedings{stegeren2019narrative,
  author = {Judith van Stegeren and Mariët Theune},
  title = {{Narrative Generation in the Wild: Methods from NaNoGenMo}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure