Consolidation of Sequential Planning

Oliver M. Vikbladh, Evan M. Russek, Neil BurgessbioRxiv 2024 · DOI

TL;DR

This paper reveals how our brains consolidate learned task structures to improve flexible planning. They show that after a period of consolidation, the brain becomes better at simulating future outcomes, particularly in the prefrontal cortex, leading to more adaptive decision-making. This work bridges the gap between memory consolidation and real-time planning, suggesting a dynamic interplay between the hippocampus and prefrontal cortex.

What Problem Does This Paper Address?

Humans are remarkably good at navigating complex and changing environments, a skill that relies heavily on our ability to plan ahead. A key hypothesis is that this planning involves mentally simulating future scenarios, often referred to as "rollouts," using internal models of the world (cognitive maps or schemas). However, it's been unclear how and where in the brain this flexible planning occurs, and how it's maintained or improved over time. Simultaneously, memory consolidation is known to transform memories, making them more stable and abstract, but its specific role in enhancing planning capabilities has been elusive. This paper aims to bridge this gap by investigating whether consolidation strengthens the internal representations used for sequential planning, leading to improved future simulations and more adaptive behavior.

Methods

The researchers developed a novel behavioral task designed to specifically measure sequential planning via "rollouts" while controlling for simpler, non-sequential strategies like the "successor representation" (SR). This task involved participants making accept/reject decisions based on sequences of objects, where the rules for rewards and sequence termination could change.

To analyze brain activity, they employed magnetoencephalography (MEG) and developed a new multivariate analysis technique. This method focused on identifying consistent neural dynamics across trials that shared the same starting object, looking for patterns that unfolded sequentially over time. They also analyzed reaction times (RTs) to assess planning depth and used Bayesian model comparison to determine which cognitive strategies (model-based, SR, or model-free) best explained participants' choices.

Datasets

The study used data from 30 participants who performed the novel behavioral task. The data was collected on two separate days, a day after initial training and a week later, to assess the effects of consolidation. No external datasets were used for training or evaluation.

Key Insights and Ideas

  • Consolidation Enhances Planning: The most striking finding is that consolidation significantly improves sequential planning. This isn't just about remembering facts; it's about strengthening the process of simulating future outcomes.
  • Hippocampus and PFC Collaboration: The study suggests a dynamic partnership between the hippocampus (involved in initial learning and online rollouts) and the prefrontal cortex (PFC) (which becomes more involved after consolidation). This aligns with systems consolidation theories.
  • Neural Signatures of Rollouts: The MEG analysis revealed distinct neural patterns associated with sequential planning, specifically a "lag-shifted dynamic similarity" that indicates the brain is simulating step-by-step progression through the task.
  • Beyond Simple Associations: The findings differentiate true sequential planning (which can adapt to changes in task structure) from simpler strategies like SR (which rely on more abstract associations).

Key Results

  • Behavioral Improvement: Participants who were "Sequential Planners" (SPs) showed increased model-based (MB) weights and improved planning depth on RTs after a week, indicating enhanced flexible planning.
  • Neural Dynamics: MEG data showed consistent neural dynamics reflecting sequential rollouts in SPs, particularly in the anterior temporal lobe (including the hippocampus). After consolidation, these dynamics became more pronounced in the right PFC.
  • Speed of Planning: The speed of these neural dynamics correlated with reaction times, suggesting that faster planning is associated with faster simulated rollouts.
  • Model Comparison: The best-fitting models for predicting choices included both model-based (MB) and successor representation (SR) components, with MB being more indicative of sequential planning.

Limitations and Future Work

  • Nature of Consolidation: While the study shows consolidation enhances planning, the precise nature of the transformed representations (e.g., cognitive maps vs. generative models) still requires clarification.
  • Reactivation Mechanisms: Future work is needed to distinguish whether consolidation is driven by reactivation during task performance or during rest.
  • Individual Differences: The study acknowledges significant individual differences in planning strategies, suggesting that factors beyond current learning might influence capacity for sequential planning.
  • Clinical Implications: The authors suggest that understanding these distinctions could be crucial for understanding disorders linked to dysregulation of flexible decision-making, such as OCD.

Summary updated

Abstract

Thriving in changing environments requires the capacity to evaluate novel courses of action. This ability is hypothesized to depend on sequential planning via step-by-step simulations of the future, using cognitive maps or schemas of task contingencies. However, it is still unclear if, how and where in the brain such flexible planning is enacted. In parallel, it is thought that consolidation transforms memory representations over time to promote adaptive behaviour. Here, we hypothesize that consolidation strengthens cognitive maps of task contingencies used for simulation during sequential planning. To test this, we developed a novel behavioural task and a new multivariate method for analysis of magnetoencephalography (MEG) data. Using choice and reaction time data we dissociated flexible sequential planning from alternative non-sequential strategies, and identified this behaviour with robust neural markers of step-by-step simulation, localized to the anterior medial temporal lobe. Retesting a week later we showed that consolidation enhanced sequential planning and strengthened markers of sequential simulation in the prefrontal cortex, consistent with systems consolidation theory. By revealing that consolidation improves future simulations for flexible planning we open up a new frontier for the investigation of the functional interactions between memory and decision-making.
BibTeX
@misc{vikbladh2024consolidation,
  author = {Oliver M. Vikbladh and Evan M. Russek and Neil Burgess},
  title = {{Consolidation of Sequential Planning}},
  year = {2024},
  doi = {10.1101/2024.11.01.621475},
  howpublished = {bioRxiv},
}
Figure