Hierarchical Encoders for Modeling and Interpreting Screenplays

Gayatri Bhat, Avneesh Saluja, Melody Dye, Jan FlorjanczykNAACL 2021

TL;DR

This paper introduces a novel hierarchical neural network architecture designed to understand the complex structure of movie screenplays. By modeling screenplays as sequences of scenes, and scenes as sequences of dialogue and action, their approach significantly improves performance on tasks like genre classification and offers a new way to visualize narrative trajectories.

What Problem Does This Paper Address?

Understanding long-form documents like movie scripts presents a significant challenge for natural language processing (NLP). While NLP has made great strides with shorter texts, the sheer length and intricate structure of documents like screenplays make it difficult for standard end-to-end models to capture all the relevant information. Existing approaches often rely on complex pipelines of feature engineering or manual selection of text segments, which can be labor-intensive and prone to errors. The authors aim to bridge this gap by proposing a neural architecture that inherently leverages the document's structure, making it more efficient and generalizable.

Methods

The core of this work is a hierarchical encoder that mirrors the natural structure of screenplays: scripts are composed of scenes, and scenes are composed of dialogue and action statements.

  • Hierarchical Structure: The model processes text at multiple levels:
  • Statement-level encoders: Separate encoders for action and dialogue statements process words within each statement.
  • Scene-level encoders: These encoders aggregate statement embeddings to create representations for entire scenes.
  • Script-level encoder: Finally, scene embeddings are combined to form a representation for the whole screenplay.
  • Attention Mechanisms: The authors explore different encoder types, including GRUs (Gated Recurrent Units) with and without attention, and a "bag-of-embeddings" approach. Attention is crucial for filtering out less important information and focusing on salient parts of the text.
  • Unsupervised Interpretability Module: To understand what the model has learned, they introduce a scene descriptor model. This module learns to represent scene embeddings as a mixture of "topics" or descriptors, allowing for visualization of narrative trajectories (e.g., how themes like "violence" or "romance" evolve throughout a script). This is achieved through a reconstruction task, minimizing the difference between the descriptor-based representation and an attention-weighted bag-of-words representation of the scene.

Datasets

The primary dataset used for evaluation is ScriptBase-J, a corpus of movie scripts with expert-curated tags. The authors also use an internal dataset with similar tag attributes. These datasets are used for multi-label tag prediction tasks.

Key Insights and Ideas

  • Structure is Key: The paper strongly suggests that explicitly modeling the hierarchical structure of documents (like scenes within a script) is crucial for improving performance on downstream tasks, outperforming models that treat the document as a flat sequence.
  • Dialogue vs. Action Matters: Distinguishing between dialogue and action statements within a scene and processing them separately leads to better results, indicating that these different discourse functions carry distinct information.
  • Interpretability Through Descriptors: The unsupervised scene descriptor model provides a powerful tool for visualizing narrative arcs and understanding how themes or topics evolve across a screenplay, offering insights beyond simple classification accuracy.
  • Similarity-Based Evaluation: The authors highlight that standard F1 scores might not fully capture model performance on complex, overlapping tag sets. A similarity-based scoring procedure, which accounts for relationships between tags, can provide a more nuanced and potentially more accurate evaluation.

Key Results

  • The hierarchical encoder significantly improves performance on two multi-label tag prediction tasks compared to baselines. For instance, their best model (GRU+Attn) shows substantial gains over a logline-based baseline.
  • The paper demonstrates that incorporating structural elements like scene-level processing and distinguishing between action/dialogue statements leads to robust improvements.
  • The unsupervised interpretability module successfully visualizes narrative trajectories for screenplays, showing how themes evolve and aligning with critical plot points. For example, they show how "Military" and "Violence" themes spike during wartime events in "Pearl Harbor."

Limitations and Future Work

  • Character Information: While the model incorporates some character information, it acknowledges that a more sophisticated representation of character identities could further enhance performance and allow for richer analysis (e.g., comparing character archetypes across movies).
  • Tag Taxonomy Complexity: The authors note that the complexity and overlap within tag taxonomies can make standard evaluation metrics less informative. They propose similarity-based scoring as a more representative evaluation method.
  • Generalizability: While the approach is discussed as generalizable to other structured documents (novels, plays, etc.), the primary evaluation is on screenplays.
  • Future Directions: The authors suggest exploring richer character representations, persona-based characterization, and applying the framework to other structured document types. They also emphasize that screenplays are an underutilized data source in NLP and encourage more research in this domain.

Summary updated

BibTeX
@inproceedings{bhat2021hierarchical,
  author = {Gayatri Bhat and Avneesh Saluja and Melody Dye and Jan Florjanczyk},
  title = {{Hierarchical Encoders for Modeling and Interpreting Screenplays}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure