Plug-and-Play Controller for Story Completion: A Pilot Study toward Emotion-aware Story Writing Assistance

Yusuke Mori, Hiroaki Yamane, Ryohei Shimizu, Tatsuya HaradaACL 2022

TL;DR

This paper introduces a novel approach to story completion that incorporates emotional control, allowing AI to generate narrative continuations that align with specific emotional trajectories. By adapting the Plug-and-Play Language Model (PPLM) for sequence-to-sequence models and leveraging a dimensional emotion lexicon, the authors demonstrate a step towards more emotionally resonant AI-generated stories.

What Problem Does This Paper Address?

Storytelling is deeply intertwined with human emotion, and a compelling narrative often follows an emotional arc. While AI has made strides in generating coherent text, controlling the emotional tone of these generated stories remains a significant challenge. Existing work often focuses on simple positive-negative sentiment or categorical emotions, lacking the nuance required for complex emotional storytelling. This paper tackles the problem of story completion (SC), where the AI needs to fill in a missing sentence in a narrative, with the added constraint of controlling the emotional valence and arousal of the generated text. This is crucial for creative writing assistance, where writers might need help maintaining or guiding the emotional flow of their stories.

Methods

The core of the proposed method involves two main components:

  1. Fine-tuning Unsupervised Pre-trained Large Neural Models for SC: The authors leverage existing large language models (LLMs) like BART, T5, PEGASUS, and ProphetNet, which are pre-trained on vast amounts of text. These models are fine-tuned for the specific task of story completion. The SC task is framed by inserting a special token, <missing_position>, into the narrative context, which the model then learns to replace with a coherent sentence.
  2. Emotion-aware Controlling of Fine-tuned Models using PPLM: To introduce emotional control, the researchers adapt the Plug-and-Play Language Model (PPLM) framework. PPLM, originally designed for decoder-only models like GPT-2, is modified to work with sequence-to-sequence models. Instead of relying on simple bag-of-words or sentiment discriminators, they utilize the NRC Valence, Arousal, and Dominance (VAD) lexicon. This allows for the control of emotions along two continuous dimensions (valence and arousal), offering more nuanced control than discrete categories. The PPLM acts as an additional module that guides the generation process to align with specified emotional targets.

Datasets

The study uses the ROCStories dataset for both training and evaluation. This dataset consists of five-sentence stories. For the SC task, one sentence is removed, and the model is tasked with generating it. The dataset was split into training, development, and test sets in an 8:1:1 ratio.

Key Insights and Ideas

  • Dimensional Emotion Control is Key: Moving beyond simple positive/negative or categorical emotions to continuous dimensions like valence and arousal (based on Russell's circumplex model) is crucial for more sophisticated emotional control in storytelling.
  • Summarization Parameters Help SC: The study found that using training parameters optimized for summarization tasks surprisingly improved performance on story completion tasks. This suggests that the ability to condense information and understand context, vital for summarization, is also beneficial for generating coherent and relevant story continuations.
  • PPLM Adaptation is Feasible but Challenging: Adapting PPLM, originally for decoder-only models, to sequence-to-sequence models is demonstrated to be possible. However, the authors highlight that fine-tuning and coordinating the parameters for precise emotional control can be very difficult.

Key Results

  • BART Large Performs Best (No-Emotion Baseline): Among the no-emotion-aware baselines, the BART large model achieved the highest scores across various automatic evaluation metrics (BLEU, ROUGE, METEOR, BERTScore, BLEURT).
  • Task-Specific Parameters Matter: For some models (T5, XLM-ProphetNet), using task-specific parameters (especially those related to summarization) led to better results compared to common parameters.
  • Emotion Control is Achieved, but with Difficulty: The emotion-controlled generation examples show that the model can produce text aligning with specified valence and arousal ranges. For instance, a "negative and excited" emotion target resulted in a relevant story continuation. However, the authors note that adjusting these parameters for precise control was "very severe" and difficult.
  • Shorter Sentences with PPLM: When PPLM was used, the generated sentences tended to be shorter compared to those generated without PPLM.

Limitations and Future Work

  • Parameter Adjustment Difficulty: The primary limitation acknowledged is the significant difficulty in adjusting PPLM parameters for precise emotional control. This makes it challenging for users to achieve desired emotional outcomes.
  • User Familiarity with Parameters: The authors note that users might be unfamiliar with terms like "valence" and "arousal," potentially leading to confusion and a negative impression of the system.
  • Preliminary Study: This is presented as a pilot study, and the results require improvement before practical system integration.
  • Future Directions:
    • Investigating more intuitive ways to control emotions, potentially by treating dimensions rather than just positive-negative.
  • Exploring alternative methods for parameter adjustment or entirely different approaches to emotion control.
  • Conducting human evaluations to assess the perceived quality and emotional resonance of the generated stories.
  • Applying the methods to other datasets, such as the WritingPrompts dataset, which offers more varied story lengths.

Summary updated

BibTeX
@inproceedings{mori2022plug,
  author = {Yusuke Mori and Hiroaki Yamane and Ryohei Shimizu and Tatsuya Harada},
  title = {{Plug-and-Play Controller for Story Completion: A Pilot Study toward Emotion-aware Story Writing Assistance}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure