The human brain reactivates context-specific past information at event boundaries of naturalistic experiences

Avital Hahamy, Haim Dubossarsky, Timothy E.j. Behrens

TL;DR

This study reveals that our brains actively reactivate past experiences at the boundaries of ongoing events, not just during rest, but in real-time. This "replay" in the hippocampus and default mode network helps us stitch together fragmented information into a coherent understanding of unfolding narratives, suggesting a crucial mechanism for continuous learning and comprehension.

What Problem Does This Paper Address?

We experience the world as a continuous flow, yet our brains seem to process it in discrete chunks or "events." To make sense of this, we need to connect these events into a coherent narrative. While it's known that rodents replay past experiences during rest to build mental models, it was unclear if and how humans do something similar in real-time to understand ongoing, naturalistic experiences like watching a movie or listening to a story. Specifically, how does the brain integrate information from temporally distant past events with the current moment to maintain a cohesive understanding?

Methods

The researchers employed functional Magnetic Resonance Imaging (fMRI) to track brain activity in participants as they engaged with naturalistic narratives (a TV show episode and an audiobook). They developed a novel "reactivation index" to identify brain regions where neural representations of past scenes were reactivated at event boundaries (the transitions between scenes). This index specifically aimed to distinguish true reactivation from mere similarities between adjacent scenes by comparing past and future scene correlations. They used a searchlight approach to analyze brain activity at a fine-grained, voxel-level, and also explored mesoscopic-scale patterns by examining shared representations across participants. To assess the relevance of reactivated information, they used a Bag-of-Words model to quantify semantic similarity between scenes.

Datasets

The study utilized two naturalistic fMRI datasets:

  • Sherlock dataset: Participants watched the first 48 minutes of the BBC series "Sherlock." This dataset features a continuous narrative structure.
  • 21st year dataset: Participants listened to a 56-minute auditory story with an alternating narrative structure for the first part, which then merged.

These datasets were chosen for their naturalistic, continuous, and unique event structures, which differ from the highly controlled, repetitive tasks often used in previous replay studies.

Key Insights and Ideas

  • Real-time reactivation: Contrary to previous findings focused on offline replay, this study demonstrates that neural reactivation of past events occurs dynamically at event boundaries during ongoing experience.
  • Hippocampus and DMN involvement: The hippocampus and the default mode network (DMN), known for memory and internal thought, are key players in this real-time reactivation process.
  • Selective reactivation: The brain doesn't just replay everything; it selectively reactivates past events that are semantically relevant to the current context, suggesting a mechanism for prioritizing information crucial for understanding.
  • Bridging temporal gaps: This reactivation serves as a mechanism to bind temporally distant information, allowing for a coherent understanding of complex, unfolding narratives.
  • Mesoscopic scale: Reactivated representations are not confined to individual voxels but show structure at a coarser, mesoscopic scale, indicating shared neural patterns across individuals.

Key Results

  • Consistent reactivation: Across both datasets and analysis methods (within-participant and between-participant), reactivation of temporally-remote past events was consistently observed in the bilateral precuneus/retrosplenial cortex (PCUN), the angular gyrus/lateral occipital cortex (Ang/LOC), and the hippocampus (HIP).
  • Specificity to event boundaries: The reactivation effects were specifically tied to event boundaries, not just random time points.
  • Semantic relevance: In the precuneus/retrosplenial cortex, reactivation was modulated by semantic context, meaning past events with similar content to the current event were more likely to be reactivated.
  • Replication: The findings were robustly replicated across the two independent datasets, strengthening the conclusion that this is a general attribute of processing ongoing experiences.

Limitations and Future Work

  • Predictive coding: While the study found evidence for reactivating past information, it did not find regions where representations at event boundaries were more similar to future events, suggesting a lack of clear predictive coding detection in this naturalistic setting. Future work could explore conditions under which predictive coding might be observed.
  • Contextual modulation in other regions: While the precuneus/retrosplenial cortex showed reproducible context-specific reactivation, other brain regions showed context modulation in one dataset but not the other. Further research with larger datasets is needed to explore this in more brain regions.
  • Alternating narratives: The 21st year dataset's alternating narrative structure complicated analyses looking at time X congruency interactions. Future studies could investigate these interaction effects in stories with more unpredictable structures.
  • DMN's online function: While the DMN is implicated, the precise mechanism by which it binds information online remains an area for further investigation. The authors propose reactivation as a key component of this online function.

Summary updated

Abstract

Although we perceive the world in a continuous manner, our experience is partitioned into discrete events. However, to make sense of these events, they must be stitched together into an overarching narrative-a model of unfolding events. It has been proposed that such a stitching process happens in offline neural reactivations when rodents build models of spatial environments. Here we show that, while understanding a natural narrative, humans reactivate neural representations of past events. Similar to offline replay, these reactivations occur in the hippocampus and default mode network, where reactivations are selective to relevant past events. However, these reactivations occur, not during prolonged offline periods, but at the boundaries between ongoing narrative events. These results, replicated across two datasets, suggest reactivations as a candidate mechanism for binding temporally distant information into a coherent understanding of ongoing experience.
BibTeX
@inproceedings{hahamy2023human,
  author = {Avital Hahamy and Haim Dubossarsky and Timothy E.J. Behrens},
  title = {{The human brain reactivates context-specific past information at event boundaries of naturalistic experiences}},
  booktitle = {Nature Neuroscience},
  year = {2023},
  doi = {10.1038/s41593-023-01331-6},
}
Figure