GPT-2-based Human-in-the-loop Theatre Play Script Generation

Rudolf Rosa, Patrícia Schmidtová, Ondřej Dušek, Tomáš Musil, David Mareček, Saad Obaid, Marie Nováková, Klára Vosecká, Josef DoležalNAACL 2022

TL;DR

This paper introduces THEaiTRE, an interactive tool that uses GPT-2 to generate theatre play scripts, allowing human users to guide the process. They explore two generation strategies: a flat approach with summarization and a hierarchical approach that first generates a synopsis and then the script. While improvements over vanilla models are shown, challenges remain in generating consistent character personalities and strictly adhering to synopses.

What Problem Does This Paper Address?

Generating long, coherent narratives like theatre play scripts is a significant challenge for current language models. Standard models like GPT-3 excel at shorter texts but struggle with the extended context and complex structure required for a full play. Theatre scripts, in particular, present unique difficulties: they are lengthy, have a pseudo-multi-author nature (different characters speaking in distinct styles), and require intricate formatting (dialogue, stage directions, character names). Fully automatic generation of a stageable play is not yet feasible. This paper aims to bridge this gap by developing an interactive system that leverages large language models while incorporating human guidance to steer the generation process, making it more manageable and potentially useful for playwrights.

Methods

The core of the THEaiTRE project lies in adapting pre-trained language models, specifically GPT-2, for theatre script generation. They investigate two primary generation strategies:

  • Flat Generation with Summarization: This approach builds upon previous work by using a standard generative model but incorporates extractive text summarization to handle the limited context window of GPT-2. This allows for the generation of longer scripts by summarizing preceding content to maintain global context. The models are fine-tuned on a dataset of theatre and movie scripts.
  • Hierarchical Two-Stage Generation: This novel approach involves two phases:
  • Synopsis Generation: A user provides a play title, and the model generates a synopsis. This phase uses models fine-tuned on a dataset of synopses from theatre plays, movies, and TV series.
  • Script Generation: The generated synopsis is then used to condition the generation of the play script. The synopsis is split into smaller chunks, and each chunk is used as input for generating corresponding script segments.

Both approaches incorporate a human-in-the-loop mechanism via a custom-built web interface. Users can regenerate lines, choose the next character to speak, or insert/delete lines, allowing for minimal but precise intervention.

Datasets

The paper utilizes several datasets:

  • Theatre and Movie Scripts: A collection of approximately 12,000 theatre and movie scripts scraped from various websites and a previously published dataset (Lison and Meena, 2016) for movie scripts. This dataset is used for fine-tuning the generation models.
  • Title-Synopsis Pairs: A dataset of over 50,000 title-synopsis pairs, compiled from Wikipedia synopses of theatre plays, movies (Robischon, 2018; Kar et al., 2018), TV series fan wikis, and books (Bamman and Smith, 2017). This is used for training the synopsis generation models.
  • ScriptBase Corpus: A near-domain corpus containing movie scripts and their synopses, used for aligning script scenes to synopsis scenes in the hierarchical approach.

Key Insights and Ideas

  • Human-in-the-loop is Crucial: The authors emphasize that fully automatic generation of a stageable play is not yet achievable. Interactive tools that allow human guidance are essential for steering the generation towards coherent and meaningful narratives.
  • Hierarchical Generation Shows Promise but Faces Challenges: The two-stage hierarchical approach (title -> synopsis -> script) offers a structured way to generate longer content. However, a key finding is that models struggle to strictly follow the pre-generated synopsis, indicating a need for better mechanisms to enforce adherence.
  • Character Consistency is a Major Hurdle: A significant limitation identified by theatre professionals is the difficulty in generating characters with consistent personalities and motivations. The generated lines often lack distinctiveness between characters, making them feel interchangeable.
  • Fine-tuning on Domain-Specific Data is Beneficial: Fine-tuning GPT-2 models on theatre and movie scripts significantly improves their performance compared to vanilla models.
  • Alignment of Script and Synopsis Scenes is Complex: The hierarchical approach requires aligning script scenes to synopsis scenes, a non-trivial task that the authors address using SBERT embeddings and dynamic programming.

Key Results

  • The THEaiTRE tool, with its interactive interface, allows for more controlled generation compared to vanilla models.
  • Preliminary results and discussions with theatre professionals suggest improvements in generated output, though specific quantitative metrics for the overall play generation are not detailed in the main text.
  • The hierarchical approach, while conceptually promising, showed difficulties in consistently following the generated synopsis.
  • A human evaluation of synopsis generation models indicated that GPT2-medium performed best due to its highest overall impression score.
  • Table 5 provides basic statistics comparing different model variants for script generation, showing variations in output length and complexity. For instance, the Hierarchical variants tend to produce shorter scripts but with higher entropy and perplexity, suggesting more variability.

Limitations and Future Work

The authors acknowledge several limitations:

  • Character Consistency: The inability to differentiate and maintain consistent personalities for individual characters is a major weakness. They plan to address this by exploring NLG personalization techniques.
  • Synopsis Adherence: The hierarchical approach struggles to follow the synopsis accurately.
  • Length Limitations: While improvements are made, generating full-length scripts remains a challenge. They suggest exploring abstractive summarization trained specifically for theatre plays.
  • Data Scarcity: Corpora of theatrical texts are scarce, and copyright issues limit the release of their dataset.
  • Theatrical Abstractions: The current approach relies on flat textual representations. Future work could incorporate more abstract theatrical concepts (e.g., dramatic situations) for better generation.

Future work also includes incorporating natural language inference checks and dialogue act semantic representations to improve coherence, and exploring character personalization techniques.

Summary updated

BibTeX
@inproceedings{rosa2022gpt,
  author = {Rudolf Rosa and Patrícia Schmidtová and Ondřej Dušek and Tomáš Musil and David Mareček and Saad Obaid and Marie Nováková and Klára Vosecká and Josef Doležal},
  title = {{GPT-2-based Human-in-the-loop Theatre Play Script Generation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure