From None to Severe: Predicting Severity in Movie Scripts

Yigeng Zhang, Mahsa Shafaei, Fabio Gonzalez, Thamar Solorio

TL;DR

This paper introduces a multitask deep learning framework to predict the fine-grained severity of age-restricted content (Sex, Violence, Profanity, Substance, and Frightening scenes) using only movie dialogue scripts. By combining classification with a pairwise ranking auxiliary task in a Siamese network, the model achieves state-of-the-art performance while providing human-interpretable explanations via movie-to-movie comparisons.

What Problem Does This Paper Address?

When deciding if a movie is appropriate for a specific audience (like children), standard rating systems like the MPAA offer only a blanket age restriction (e.g., PG-13 or R). They don't break down why a movie received that rating or specify the severity of individual objectionable elements.

While computer vision models can analyze video files for violence or nudity, processing full-length movies is computationally heavy and scales poorly. Analyzing text scripts is a much more lightweight and efficient alternative. However, movie scripts average around 10,000 words—far exceeding the token limits of standard Transformer models—and lack visual or audio cues (like screams or explosions), making content classification a significant natural language processing (NLP) challenge.

Methods

The authors frame the problem as an ordinal multi-class classification task across four fine-grained severity levels: None, Mild, Moderate, and Severe.

To handle long scripts and capture context, they propose a hierarchical architecture called RNN-Trans:

  1. Utterance Encoding: Each individual line of dialogue (utterance) is encoded into a dense vector using SentenceTransformers (based on BERT-large).
  2. Contextual Modeling: A bidirectional LSTM (Bi-LSTM) processes the sequence of utterance embeddings to capture the flow and interrelations of the dialogue.
  3. Document Representation: Max-pooling is applied across all time steps of the LSTM hidden states to generate a single vector representing the entire movie script.

The Multitask Ranking-Classification Framework

Because severity is a relative concept, the authors introduce a novel Siamese network-based multitask framework with tied weights.

Figure 1: Multitask pairwise ranking-classification net- work.

During training, the model takes pairs of movies and optimizes two losses simultaneously:

  • Classification Loss ($l_c$): Predicts the absolute severity level of each movie.
  • Pairwise Ranking Loss ($l_r$): Predicts whether Movie A has less, equal, or more severe content than Movie B.

This ranking subtask acts as a regularizer that forces the model to learn the ordinal relationships between categories rather than treating them as independent classes.

Datasets

The authors created a new, publicly available dataset by scraping up-to-date user ratings from the IMDb Parents Guide for over 15,000 movies and matching them with dialogue scripts.

  • Scope: Covers 5 distinct aspects: Sex & Nudity, Violence & Gore, Profanity, Alcohol, Drugs & Smoking, and Frightening & Intense Scenes.
  • Data Filtering: To ensure robust labels, they filtered out movies with fewer than 5 user votes, resulting in a final dataset of roughly 4,400 to 6,600 movies per aspect.
  • Scale: This dataset is approximately five times larger than restricted datasets used in previous related works.

Key Insights and Ideas

  • Overt vs. Latent Signals: The optimal model architecture depends heavily on the type of content being analyzed. For Profanity, word-level models (like TextRCNN) perform best because profanity relies on explicit, overt signals (swear words). Conversely, Violence and Frightening scenes are often latent or implied in dialogue; here, the utterance-level RNN-Trans model excels because it captures broader context.
  • The Power of Comparison: Framing the network as a Siamese ranking tool dramatically improves interpretability. Instead of just outputting a label like "Moderate Violence," the system can justify its choice to a user by showing that the script is textually more violent than Pride & Prejudice but less violent than Deadpool.

Key Results

The proposed multitask models outperformed multiple strong baselines (including standard BERT and TextCNN) across all five content categories on macro F1 scores:

  • RNN-Trans S-MT (the proposed multitask Transformer model) achieved the best results on Sex (45.68), Violence (55.90), and Frightening (51.55).
  • TextRCNN S-MT (the proposed multitask word-level model) performed best on Profanity (67.26) and Substance (43.92).

When benchmarked against previous state-of-the-art methods using a strict 10-fold cross-validation setup, the proposed framework outperformed prior works by a large margin (e.g., boosting the F1 score for Frightening scenes from 48.81 to 51.03).

Limitations and Future Work

The primary limitation stems from the medium itself: dialogue text cannot capture purely visual or auditory information. For example, the authors note that the model struggled with movies like A Clockwork Orange, where intense visual depictions of substance abuse or nudity occur without explicit verbal discussion.

To address this, the authors propose that future work should explore multimodal approaches that integrate video and audio features alongside text scripts, as well as breaking down categories into more fine-grained sub-types (e.g., distinguishing between different types of violence).

Summary updated

Abstract

In this paper, we introduce the task of predicting severity of age-restricted aspects of movie content based solely on the dialogue script. We first investigate categorizing the ordinal severity of movies on 5 aspects: Sex, Violence, Profanity, Substance consumption, and Frightening scenes. The problem is handled using a siamese network-based multitask framework which concurrently improves the interpretability of the predictions. The experimental results show that our method outperforms the previous state-of-the-art model and provides useful information to interpret model predictions. The proposed dataset and source code are publicly available at our GitHub repository.
BibTeX
@inproceedings{zhang2021from,
  author = {Yigeng Zhang and Mahsa Shafaei and Fabio Gonzalez and Thamar Solorio},
  title = {{From None to Severe: Predicting Severity in Movie Scripts}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.332},
}
Figure