Plug-and-Blend: A Framework for Controllable Story Generation with Blended Control Codes

Zhiyu Lin, Mark RiedlNAACL 2021

TL;DR

This paper introduces Plug-and-Blend, a novel framework for controllable story generation that allows users to specify multiple, potentially overlapping, topics with continuous weights. This enables fine-grained control over generated narratives, moving beyond simple prompts to facilitate co-creative writing with large language models.

What Problem Does This Paper Address?

Large pre-trained language models (LLMs) like GPT-3 are incredibly powerful for text generation, but controlling their output beyond a basic prompt is challenging. This lack of control makes them ill-suited for co-creative tasks where human users need to guide the generation with specific constraints, such as style, context, or topic. Existing conditional generation methods often require modifying the base model or are limited in their ability to handle multiple, blended control signals. The authors aim to bridge this gap by providing a "plug-and-play" framework that can work with existing LLMs without requiring access to their internal parameters, allowing for nuanced control over generated content.

Methods

The Plug-and-Blend framework consists of two main components:

  1. Blending Generative Model: This component takes the logit outputs from a base LLM and biases them towards specific control codes (topics). It uses a technique inspired by adversarial networks, treating the output of a control model (e.g., a discriminator trained to predict topic presence) as a heuristic to steer the generation. Crucially, it supports blending multiple control codes by combining their influence, allowing for continuous weighting. This means a single sentence can be influenced by multiple topics simultaneously, with varying degrees of emphasis.
  2. Planner: This component translates high-level "control sketches" provided by a human user into concrete control configurations for the generative model. A control sketch defines the number of sentences, a set of topics, and the range of lines within the story where each topic should appear. The planner uses a Gaussian distribution to assign control strengths, giving higher emphasis to the midpoint of a topic's designated range and lower emphasis towards the start and end. This allows for smooth transitions and overlapping topic applications.

The framework is "plug-and-play" because it only interacts with the logit outputs of the LLM, meaning it can be used with various pre-trained models without needing to access or modify their internal weights or hidden states.

Datasets

  • ROCStories: Used for fine-tuning the GPT2-large base language model. This dataset consists of five-sentence stories.
  • AG-news: Used for training the guiding model (a pre-trained GeDi model).

The paper does not introduce a new benchmark dataset but uses existing datasets for training and evaluation.

Key Insights and Ideas

  • Blended Control is Key: The ability to blend multiple topics with continuous weights is a significant advancement. This allows for more nuanced and realistic control, moving beyond binary topic inclusion.
  • Plug-and-Play Efficiency: The framework's ability to work with LLM logits without internal access is crucial for its adaptability and ease of use with state-of-the-art models, especially those with restricted access (like GPT-3).
  • Heuristic Planner for Smooth Transitions: The Gaussian-based planner effectively translates user intent into smooth topic transitions within a generated narrative, creating a more natural flow.
  • Trade-off Between Fluency and Control: The experiments reveal a clear trade-off: increasing control strength generally improves control fidelity (how well topics are represented) but can decrease fluency (measured by perplexity).

Key Results

  • Control Fidelity: The framework demonstrates strong control fidelity, as measured by Kendall's τ-a score. The system successfully generates sentences that increasingly reflect the intended topics as control strength increases. For instance, with a combined control strength of 4x, the τ-a score significantly improved compared to the baseline, indicating better adherence to topic blending.
  • Fluency: Perplexity generally increases with higher control strength, suggesting a potential decrease in fluency. However, the paper notes that the increase is more linear up to a certain point (around 2x strength), after which it becomes more exponential. This suggests there's a usable range for control before fluency is severely impacted.
  • Qualitative Examples: Table 2 showcases generated paragraphs based on different control sketches, demonstrating the framework's ability to blend topics like "Sports" and "Science" and transition between them within a narrative. The bolded sentences in the generated paragraphs highlight these topic shifts.

Limitations and Future Work

  • Limited Topics: The current control model is optimized for only four topics (Business, Science, Sports, World). For full co-creativity, it would need to support a wider range of topics and potentially finer-grained control.
  • Heuristic Planner: While effective, the planner is currently heuristic. The authors suggest exploring reinforcement learning approaches where the planner learns from user feedback and demonstrations to optimize control configurations.
  • Acquisition of Control Sketches: The paper doesn't detail how control sketches are acquired from users. Future work could involve developing more intuitive interfaces for users to express their desired narrative structure and topics.
  • Trade-off Management: The observed trade-off between fluency and control fidelity suggests the need for strategies to balance these aspects, such as iterative deepening or selecting the best sentence from multiple generation attempts with varying control strengths.

Summary updated

BibTeX
@inproceedings{lin2021plug,
  author = {Zhiyu Lin and Mark Riedl},
  title = {{Plug-and-Blend: A Framework for Controllable Story Generation with Blended Control Codes}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure