Generating Narrative Text in a Switching Dynamical System

Noah Weber, Leena Shekhar, Heeyoung Kwon, Niranjan Balasubramanian, Nathanael Chambers

Abstract

This paper introduces a novel framework for narrative generation that bridges the gap between high-level structural control and low-level linguistic fluency. By modeling stories as trajectories within a Switching Linear Dynamical System (SLDS), the authors demonstrate a method to guide neural text generation using discrete "narrative scaffolds," such as sentiment. The proposed model outperforms standard language models in coherence and human preference, particularly in goal-directed tasks like story interpolation.

Introduction

Generating coherent, long-form narrative text remains a significant challenge in artificial intelligence. While classical planning-based systems offered high controllability, they often lacked the natural fluency of modern neural networks. Conversely, modern Large Language Models (LLMs) excel at local fluency but frequently struggle with global consistency, leading to aimless or illogical plot developments.

The authors identify a lack of explicit representation for narrative scaffolding—the high-level plan—and narrative dynamics—the evolution of the story state—in current neural models. To address this, they propose a probabilistic framework that treats story generation as "filling in the blanks" along a structured path.

Methodology: The Switching Linear Dynamical System

The core of the approach is the application of a Switching Linear Dynamical System (SLDS) to represent text. This model decomposes the generation process into three distinct layers:

  1. Discrete States ($S_i$): These represent the narrative scaffold, such as sentiment switches (Positive, Negative, Neutral).
  2. Vector Context ($Z_i$): A continuous latent state that captures the evolving context of the story.
  3. Discrete Words ($X_i$): The actual generated text, produced by a neural decoder (GRU-based RNN) conditioned on the latent state.
The SLDS architecture showing the hierarchical relationship between discrete narrative states, continuous vector context, and the generated word sequence.

Unlike standard dynamical systems, the SLDS utilizes multiple transition matrices. The specific "switch" ($S_i$) active at any given step determines which matrix transforms the previous latent state into the next, allowing the model to steer the narrative trajectory based on desired structural goals. Training is performed using Amortized Variational Inference, while generation utilizes a Gibbs sampler for tasks like story interpolation (filling in middle sentences between a given start and end).

Experimental Setup and Results

The model was evaluated using the ROCStories corpus, consisting of five-sentence commonsense stories. Sentiment labels were automatically generated using the VADER tool to provide the necessary scaffolding for training. The SLDS was compared against standard Language Models (LM), Sequence-to-Sequence (S2S) models, and a simple Linear Dynamical System (LDS) without switching.

Human Evaluation and Coherence

In head-to-head human evaluations, the SLDS showed a clear advantage over baseline models. Judges preferred stories generated by the SLDS significantly more often than those from standard language models or sequence-to-sequence models.

Human preference and sentence coherence scores comparing the SLDS against Language Model (LM) and Sequence-to-Sequence (S2S) baselines.

As shown in the results, the SLDS achieved a human preference rate of 51.3% against the LM and 51.9% against the S2S model. Furthermore, the "Sentence Coherence" scores were consistently higher for the SLDS, indicating that the transitions between sentences felt more logical and grounded in a global plan.

Controllability and Semi-Supervision

The model demonstrated exceptional controllability, achieving a Macro F1 score of 96.1 in sentiment classification, confirming that the generated text accurately reflected the intended "switch" state. Additionally, the authors found that the SLDS is highly data-efficient; it learned meaningful dynamics even when only 1% to 10% of the training data was labeled with sentiment tags. In interpolation tasks, the SLDS outperformed baselines on automatic metrics like ROUGE and METEOR, proving its effectiveness at goal-directed generation.

Conclusion

By treating narrative as a trajectory through a latent space guided by discrete switches, the SLDS provides a powerful mechanism for controllable and coherent story generation. While current work focuses on sentiment as a scaffold, the framework is generalizable to more complex structures like character goals or plot points. Future research may explore non-linear transitions and more sophisticated decoding strategies to further enhance the richness of the generated narratives.

Summary updated

BibTeX
@inproceedings{weber2020generating,
  author = {Noah Weber and Leena Shekhar and Heeyoung Kwon and Niranjan Balasubramanian and Nathanael Chambers},
  title = {{Generating Narrative Text in a Switching Dynamical System}},
  booktitle = {CoNLL},
  year = {2020},
  doi = {10.18653/v1/2020.conll-1.42},
}
Figure