TL;DR
This paper introduces a multitask deep learning framework to predict the fine-grained severity of age-restricted content (Sex, Violence, Profanity, Substance, and Frightening scenes) using only movie dialogue scripts. By combining classification with a pairwise ranking auxiliary task in a Siamese network, the model achieves state-of-the-art performance while providing human-interpretable explanations via movie-to-movie comparisons.
What Problem Does This Paper Address?
When deciding if a movie is appropriate for a specific audience (like children), standard rating systems like the MPAA offer only a blanket age restriction (e.g., PG-13 or R). They don't break down why a movie received that rating or specify the severity of individual objectionable elements.
While computer vision models can analyze video files for violence or nudity, processing full-length movies is computationally heavy and scales poorly. Analyzing text scripts is a much more lightweight and efficient alternative. However, movie scripts average around 10,000 words—far exceeding the token limits of standard Transformer models—and lack visual or audio cues (like screams or explosions), making content classification a significant natural language processing (NLP) challenge.
Methods
The authors frame the problem as an ordinal multi-class classification task across four fine-grained severity levels: None, Mild, Moderate, and Severe.
To handle long scripts and capture context, they propose a hierarchical architecture called RNN-Trans:
- Utterance Encoding: Each individual line of dialogue (utterance) is encoded into a dense vector using
SentenceTransformers(based on BERT-large). - Contextual Modeling: A bidirectional LSTM (Bi-LSTM) processes the sequence of utterance embeddings to capture the flow and interrelations of the dialogue.
- Document Representation: Max-pooling is applied across all time steps of the LSTM hidden states to generate a single vector representing the entire movie script.
The Multitask Ranking-Classification Framework
Because severity is a relative concept, the authors introduce a novel Siamese network-based multitask framework with tied weights.
During training, the model takes pairs of movies and optimizes two losses simultaneously:
- Classification Loss ($l_c$): Predicts the absolute severity level of each movie.
- Pairwise Ranking Loss ($l_r$): Predicts whether Movie A has less, equal, or more severe content than Movie B.
This ranking subtask acts as a regularizer that forces the model to learn the ordinal relationships between categories rather than treating them as independent classes.
Datasets
The authors created a new, publicly available dataset by scraping up-to-date user ratings from the IMDb Parents Guide for over 15,000 movies and matching them with dialogue scripts.
- Scope: Covers 5 distinct aspects: Sex & Nudity, Violence & Gore, Profanity, Alcohol, Drugs & Smoking, and Frightening & Intense Scenes.
- Data Filtering: To ensure robust labels, they filtered out movies with fewer than 5 user votes, resulting in a final dataset of roughly 4,400 to 6,600 movies per aspect.
- Scale: This dataset is approximately five times larger than restricted datasets used in previous related works.
Key Insights and Ideas
- Overt vs. Latent Signals: The optimal model architecture depends heavily on the type of content being analyzed. For Profanity, word-level models (like TextRCNN) perform best because profanity relies on explicit, overt signals (swear words). Conversely, Violence and Frightening scenes are often latent or implied in dialogue; here, the utterance-level RNN-Trans model excels because it captures broader context.
- The Power of Comparison: Framing the network as a Siamese ranking tool dramatically improves interpretability. Instead of just outputting a label like "Moderate Violence," the system can justify its choice to a user by showing that the script is textually more violent than Pride & Prejudice but less violent than Deadpool.
Key Results
The proposed multitask models outperformed multiple strong baselines (including standard BERT and TextCNN) across all five content categories on macro F1 scores:
- RNN-Trans S-MT (the proposed multitask Transformer model) achieved the best results on Sex (45.68), Violence (55.90), and Frightening (51.55).
- TextRCNN S-MT (the proposed multitask word-level model) performed best on Profanity (67.26) and Substance (43.92).
When benchmarked against previous state-of-the-art methods using a strict 10-fold cross-validation setup, the proposed framework outperformed prior works by a large margin (e.g., boosting the F1 score for Frightening scenes from 48.81 to 51.03).
Limitations and Future Work
The primary limitation stems from the medium itself: dialogue text cannot capture purely visual or auditory information. For example, the authors note that the model struggled with movies like A Clockwork Orange, where intense visual depictions of substance abuse or nudity occur without explicit verbal discussion.
To address this, the authors propose that future work should explore multimodal approaches that integrate video and audio features alongside text scripts, as well as breaking down categories into more fine-grained sub-types (e.g., distinguishing between different types of violence).
Summary updated