Recollection versus Imagination: Exploring Human Memory and Cognition via Neural Language Models

Maarten Sap, Eric Horvitz, Yejin Choi, Noah A. Smith, James W. Pennebaker

Introduction

Distinguishing between truth and fiction is a foundational challenge in both psychology and computer science. Human memory is not a perfect video recording; it relies on episodic memory (specific, personal experiences). In contrast, imagination draws more heavily on semantic memory (general world knowledge and "commonsense"). While psychologists have studied these differences in small-scale lab settings, large-scale quantification of these "cognitive traces" in natural language has remained elusive. This research, presented at ACL 2020, bridges this gap by using modern NLP tools to analyze how the human brain constructs narratives differently when remembering versus inventing.

The Hippocorpus Dataset

To facilitate this study, the researchers introduced HIPPOCORPUS, a dataset of 6,854 diary-like stories. The collection process involved three categories:

  1. Recalled: Authors wrote about a memorable event from the last six months.
  2. Imagined: A different set of authors was given a brief summary of a recalled story and asked to imagine a story based on that prompt, ensuring differences are due to cognitive processes rather than topic.
  3. Retold: Original authors retold their real memories months later to observe temporal changes.

Quantifying Cognitive Traces

The authors developed three primary measures to analyze storytelling:

1. Narrative Flow (Linearity)

Using OpenAI’s GPT, the researchers compared a "bag" model (sentences are independent) against a "chain" model (sentences follow logically). The difference in predictability defines the "linearity" of the story.

2. Episodic vs. Semantic Knowledge

The study distinguished between concrete factual events and general world knowledge:

  • Episodic traces: Identified using a BERT-based "realis" event detector to find concrete events that actually occurred in the narrative.
  • Semantic traces: Measured using the ATOMIC knowledge graph to track reliance on general commonsense inferences (e.g., "getting married causes happiness").
Comparison of Recalled vs. Imagined narratives. Recalled stories feature higher concrete event density, while imagined stories rely more on semantic commonsense links.

3. Lexical Analysis

Standard LIWC (Linguistic Inquiry and Word Count) tools measured emotional tone and the frequency of "I-words."

Key Findings

The analysis revealed several counter-intuitive insights into human cognition:

  • Imagination is more "logical" than reality: Imagined stories flow more linearly ($d = 0.52$). When inventing, humans follow predictable "A leads to B" scripts. Real memories are "holistic" and messy, with less predictable sentence transitions as the brain jumps between fragmented details.
  • The "Narrativization" of Memory: As memories age or are retold, they begin to resemble imagined stories. They become more linear ($d = 0.17$), less concrete, and rely more on general commonsense. We essentially "edit" our memories into better stories over time.
  • Imagination is Self-Centered and Emotional: Imagined stories contained significantly more "I-words" and emotional language than real recollections.

Conclusion and Limitations

While the study provides a powerful framework for detecting cognitive fingerprints, the authors acknowledge a potential "domain bias": GPT is trained on fiction, which may make it naturally better at predicting "imagined" stories. Future work could explore these traces across different cultures or apply these tools to assist in studying memory-related disorders and deceptive text detection.

Summary updated

Abstract

We investigate the use of NLP as a measure of the cognitive processes involved in storytelling, contrasting imagination and recollection of events. To facilitate this, we collect and release Hippocorpus, a dataset of 7,000 stories about imagined and recalled events. We introduce a measure of narrative flow and use this to examine the narratives for imagined and recalled events. Additionally, we measure the differential recruitment of knowledge attributed to semantic memory versus episodic memory (Tulving, 1972) for imagined and recalled storytelling by comparing the frequency of descriptions of general commonsense events with more specific realis events. Our analyses show that imagined stories have a substantially more linear narrative flow, compared to recalled stories in which adjacent sentences are more disconnected. In addition, while recalled stories rely more on autobiographical events based on episodic memory, imagined stories express more commonsense knowledge based on semantic memory. Finally, our measures reveal the effect of narrativization of memories in stories (e.g., stories about frequently recalled memories flow more linearly; Bartlett, 1932). Our findings highlight the potential of using NLP tools to study the traces of human cognition in language.
BibTeX
@inproceedings{sap2020recollection,
  author = {Maarten Sap and Eric Horvitz and Yejin Choi and Noah A. Smith and James W. Pennebaker},
  title = {{Recollection versus Imagination: Exploring Human Memory and Cognition via Neural Language Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.178},
}
Figure