Abstract
Predicting subsequent actions from natural language descriptions is a core challenge in narrative understanding. This paper introduces the Harry Potter Action prediction (HPAC) corpus, leveraging the unique semantic anchors provided by "spells" in the Wizarding World to create a large-scale, self-annotated dataset. We evaluate several baseline architectures, finding that while sequential models like LSTMs benefit significantly from increased context, simpler models remain competitive on infrequent actions.
Introduction
Action prediction involves inferring the most likely next behavior from a scene description. Unlike reading comprehension or cloze tasks, action prediction suffers from labeling ambiguity; actions are often described with varied, nuanced language. To address this, we utilize the Harry Potter universe, where spells like Alohomora or Lumos serve as discrete, unambiguous labels for complex actions (e.g., unlocking a door or turning on a light).
The HPAC Corpus
The primary contribution of this work is the HPAC (Harry Potter Action prediction) Corpus. Because the original seven books contain only 432 spell occurrences—insufficient for deep learning—the authors crawled fanfiction.net for English stories. The resulting corpus contains 82,836 samples across 85 distinct actions (spells). The data follows a "long tail" distribution, where a few spells (like Avada Kedavra) appear thousands of times, while others appear only a handful of times.
Methods
We frame action prediction as a sequence-to-label classification task. Four architectures were evaluated to establish baselines:
- Multinomial Logistic Regression (MLR): A simple model using one-hot encoded word counts.
- MultiLayer Perceptron (MLP): A basic neural network with one hidden layer.
- Convolutional Neural Network (CNN): Designed to capture local patterns and n-gram features.
- Long Short-Term Memory (LSTM): A sequential model designed to handle long-range dependencies using pre-trained GloVe embeddings.
We varied the "snippet length" (32, 64, 96, or 128 tokens) provided to the models to assess the impact of narrative context on prediction accuracy.
Key Results and Insights
The LSTM model achieved the highest overall performance, particularly as context length increased. With a 128-token snippet, the LSTM reached a Recall@1 of 35.1% and a Recall@10 of 76.9%. This suggests that LSTMs successfully "read" the narrative arc rather than just identifying immediate trigger words.
However, a "Simple Model Paradox" emerged: while LSTMs dominate on frequent actions, MLR often performs better on infrequent actions. Neural models tend to over-fit to majority classes, whereas simpler models remain more robust under data scarcity. Additionally, models frequently struggle with "counter-actions" (e.g., confusing Lumos with Nox) because the surrounding context—such as being in a dark hallway—is nearly identical for both.
Conclusion
The HPAC challenge demonstrates that fictional magic systems provide a unique, discrete vocabulary for intent that can serve as a benchmark for action prediction. While LSTMs lead the pack with sufficient context, the massive performance gap between frequent and infrequent actions remains a hurdle. Future work should explore transferring these insights to real-world domains and incorporating external knowledge bases to resolve semantic confusion between related actions.
Summary updated