Abstract
Lifelong experiences and learned knowledge lead to shared expectations about how common situations tend to unfold. Such knowledge of narrative event flow enables people to weave together a story. However, comparable computational tools to evaluate the flow of events in narratives are limited. We quantify the differences between autobiographical and imagined stories by introducing sequentiality, a measure of narrative flow of events, drawing probabilistic inferences from a cutting-edge large language model (GPT-3). Sequentiality captures the flow of a narrative by comparing the probability of a sentence with and without its preceding story context. We applied our measure to study thousands of diary-like stories, collected from crowdworkers about either a recent remembered experience or an imagined story on the same topic. The results show that imagined stories have higher sequentiality than autobiographical stories and that the sequentiality of autobiographical stories increases when the memories are retold several months later. We find that lower sequentiality is associated with higher proportions of major events. The methods and results highlight opportunities to use computational analyses to investigate the influences of memory and reasoning on language generation processes.
Introduction
While humans are natural storytellers, computationally measuring the "flow" of a narrative has historically been difficult. Most existing tools rely on surface-level features like word frequency or sentiment analysis, which fail to capture the underlying cognitive difference between episodic memory (recalling specific, idiosyncratic details) and schematic knowledge (relying on general expectations of how an event should unfold). This research introduces a mathematical framework to distinguish between stories someone actually lived versus those they imagined, and tracks how the structure of memories changes through retelling.
Methodology: Defining Sequentiality
The core innovation of this study is the sequentiality metric. It leverages the probabilistic nature of Large Language Models (LLMs) to measure how much the preceding context of a story helps predict the subsequent sentence.
As shown in the figure above, the researchers compare two generative models:
- Topic-driven model: Calculates the likelihood of a sentence $S_i$ based only on the overall story topic $\mathcal{T}$.
- Contextual model: Calculates the likelihood of a sentence $S_i$ based on the topic $\mathcal{T}$ plus all preceding sentences $S_0...S_{i-1}$.
Sequentiality is defined as the difference between these two probabilities. A high score indicates a sentence was highly predictable given the context (following a standard "schema"), while a low score suggests a "salient event" that deviates from expected narrative arcs.
Experimental Results
The study utilized the Hippocorpus dataset, containing ~7,000 stories categorized as Recalled (real experiences), Retold (the same experiences 3–6 months later), and Imagined (fictionalized versions of the same topics).
The Schematic Nature of Imagination
The primary finding was a clear ranking in narrative flow: Imagined > Retold > Recalled. Imagined stories consistently exhibited the highest sequentiality, suggesting that when people make things up, they rely heavily on mental "scripts" or schemas (e.g., the standard steps of ordering and paying at a restaurant).
Memory "Gist-ification"
One of the most significant insights is that retold stories have higher sequentiality than fresh recalls. This suggests a process of "memory consolidation" or "gist-ification": as time passes, our brains strip away idiosyncratic episodic details and replace them with more abstract, schematic versions. We essentially "narrativize" our own lives, making our pasts more predictable and standard the more we think about them.
Major Events and Narrative Shifts
By comparing LLM scores with human annotations, the researchers found that sentences identified as "major events" or "plot twists" consistently showed lower sequentiality. These are the specific details that break the expected flow, validating that the LLM-based metric aligns with human perception of narrative importance and surprise.
Conclusion and Future Directions
Sequentiality proves to be a robust predictor of story type, outperforming traditional metrics like word count or concrete event detection. While currently dependent on the cultural biases inherent in the LLM's training data, this metric offers promising applications in fields such as lie detection (identifying stories that are "too schematic" to be real), tracking the evolution of misinformation, and serving as a potential marker for cognitive health disorders like Alzheimer’s, where episodic memory diminishes.
Summary updated