Bringing Stories Alive: Generating Interactive Fiction Worlds

Prithviraj Ammanabrolu, Wesley Cheung, Dan Tu, William Broniec, Mark O. Riedl

TL;DR

This paper tackles the challenge of automatically generating interactive fiction (IF) worlds by extracting knowledge graphs from existing stories and then using these graphs to generate descriptive text. Their novel approach, AskBERT, leverages question-answering models to build these knowledge graphs, leading to more coherent and engaging IF experiences, especially in genres with distinct thematic commonsense.

What Problem Does This Paper Address?

Creating compelling interactive fiction worlds requires more than just generating text; it demands "world building" that is semantically consistent, interesting, and coherent. This involves incorporating both everyday commonsense knowledge (e.g., kitchens are in houses) and thematic knowledge specific to a genre (e.g., light-speed travel in sci-fi). Existing methods often struggle to balance these requirements, leading to worlds that are either too generic or lack narrative depth. This paper aims to bridge this gap by procedurally generating IF worlds that are rich in detail and adhere to genre conventions, drawing inspiration from existing story plots.

Methods

The authors propose a two-phase approach:

  1. Knowledge Graph Construction:
    • Neural Approach (AskBERT): This method uses a pre-trained question-answering model (ALBERT) to extract entities (locations, characters, objects) and their relations from a story. It iteratively asks questions to the QA model, masking out previous answers to refine extraction. The extracted entities form the vertices of a knowledge graph, and relations are added by querying the QA model about connections between entities.
  2. Rule-Based Baseline: This approach uses OpenIE5 for information extraction, followed by named-entity recognition and part-of-speech tagging. Triples are generated, and relations are inferred based on sentence order and explicit location mentions.
  3. Description Generation:
    • Neural Approach: A conditional transformer-based language model (GPT-2 fine-tuned on plot summaries) generates descriptive "flavortext" for entities in the knowledge graph. The model is prompted with the story context up to the entity's mention and a descriptive question.
  4. Rule-Based Baseline: This method utilizes pre-defined templates from the TextWorld library to generate descriptions for locations and objects.

Datasets

The authors compiled a dataset of 695 fairy tales and 536 mystery stories by scraping plot summaries from Wikipedia. This dataset was used for both training and evaluation. They did not introduce a new benchmark dataset but rather leveraged existing stories to create their IF worlds.

Key Insights and Ideas

  • Thematic Commonsense is Crucial: The paper strongly suggests that effectively incorporating thematic commonsense knowledge is vital for generating coherent and engaging IF worlds, especially in genres like fairy tales where thematic elements diverge significantly from everyday commonsense.
  • Question-Answering for Knowledge Extraction: Using QA models like ALBERT for knowledge graph construction offers a powerful way to extract structured information from text without requiring task-specific labeled data.
  • Balancing Flavor and Coherence: The neural description generation approach strikes a balance between knowledge graph verbalization (which can be dry) and open-ended generation (which can lack coherence), resulting in more engaging narratives.
  • Genre Matters: The performance of different generation approaches varies significantly across genres, highlighting the importance of genre-specific knowledge and expectations.

Key Results

  • Knowledge Graph Construction: Human evaluations showed that the neural AskBERT model produced more coherent knowledge graphs and better genre resemblance, particularly for fairy tales, compared to the rule-based and random approaches.
  • Full Game Evaluation: In mystery genres, participants generally preferred games generated by the neural approach over rule-based and random baselines. For fairy tales, the neural approach significantly outperformed the rule-based and random methods in terms of coherence and genre resemblance. The human-authored games consistently performed best across all metrics.

Limitations and Future Work

  • Human-Authored Baseline: While the human-authored games served as a strong benchmark, the paper acknowledges that their neural model did not outperform them.
  • Rule-Based Limitations: The rule-based description generation, while consistent, lacked the "flavor" and detail of the neural approach.
  • Future Work: The authors suggest that their work can be seen as complementary to other procedural content generation systems, particularly those focusing on quest generation. Further research could explore more sophisticated ways to integrate thematic commonsense and improve the generation of more complex narrative structures.

Summary updated

Abstract

World building forms the foundation of any task that requires narrative intelligence. In this work, we focus on procedurally generating interactive fiction worlds---text-based worlds that players "see" and "talk to" using natural language. Generating these worlds requires referencing everyday and thematic commonsense priors in addition to being semantically consistent, interesting, and coherent throughout. Using existing story plots as inspiration, we present a method that first extracts a partial knowledge graph encoding basic information regarding world structure such as locations and objects. This knowledge graph is then automatically completed utilizing thematic knowledge and used to guide a neural language generation model that fleshes out the rest of the world. We perform human participant-based evaluations, testing our neural model's ability to extract and fill-in a knowledge graph and to generate language conditioned on it against rule-based and human-made baselines. Our code is available at https://github.com/rajammanabrolu/WorldGeneration.
BibTeX
@inproceedings{ammanabrolu2020bringing,
  author = {Prithviraj Ammanabrolu and Wesley Cheung and Dan Tu and William Broniec and Mark O. Riedl},
  title = {{Bringing Stories Alive: Generating Interactive Fiction Worlds}},
  booktitle = {AIIDE},
  year = {2020},
}
Figure