The computational nature of memory modification

Samuel J. Gershman, Marie‐H. Monfils, Kenneth A. Norman, Yael NiveLife 2017 · DOI

TL;DR

This paper introduces a novel computational theory of memory modification, proposing that memories are updated based on inferring "latent causes" that explain sensory experiences. By integrating structure learning (identifying causes) with associative learning (updating memory strength), the model offers a unified explanation for various memory modification phenomena in Pavlovian conditioning, moving beyond traditional "unlearning" models.

What Problem Does This Paper Address?

The field of memory research has long grappled with how memories change after retrieval. While it's clear that retrieving a memory can make it malleable, the underlying mechanisms are not fully understood. Traditional associative learning models, like the Rescorla-Wagner model, often treat extinction (presenting a conditioned stimulus without the unconditioned stimulus) as simply "unlearning" the association. However, phenomena like spontaneous recovery, reinstatement, and renewal suggest that extinguished memories aren't truly gone, but rather that new information competes with or modifies the original memory. This paper aims to bridge this gap by proposing a computational framework that explains how the brain decides whether to update an existing memory or form a new one, specifically focusing on Pavlovian conditioning paradigms.

Methods

The authors propose a "latent cause theory" that integrates two key computational processes:

  1. Structure Learning: This mechanism infers "latent causes" – underlying environmental factors that explain sequences of sensory observations. It segments experience into statistically distinct clusters, essentially identifying distinct contexts or scenarios. This is framed as an approximate Bayesian inference process.
  2. Associative Learning: This process updates the strength of associations between stimuli (e.g., conditioned stimulus - CS, unconditioned stimulus - US). Crucially, these associations are now modulated by the inferred latent causes.

The core of their computational implementation is an Expectation-Maximization (EM) algorithm. This algorithm iteratively alternates between:

  • E-step (Structure Learning): Inferring the posterior probability distribution over latent causes given the observed data and current associative weights.
  • M-step (Associative Learning): Updating the associative weights for each latent cause to minimize prediction error, conditional on the inferred latent cause probabilities.

This EM approach allows the model to learn both the structure of the environment (latent causes) and the associations within those structures. They also incorporate a power law kernel for temporal dependencies between latent causes, which helps explain phenomena like spontaneous recovery.

Datasets

The paper does not introduce new datasets. Instead, it uses existing experimental findings from the Pavlovian conditioning literature (primarily fear conditioning studies in rodents and humans) as benchmarks to validate its model's predictions. These findings include:

  • Retrieval-induced memory modification (e.g., Nader et al., 2000; Monfils et al., 2009).
  • Boundary conditions on memory modification (e.g., memory age, strength, reexposure duration).
  • Cue-specificity of amnestic treatments.
  • The Monfils-Schiller paradigm.
  • Paradoxical enhancement of memory.
  • Transience of amnesia.
  • State-dependency effects.

Key Insights and Ideas

  • Memory is Inferential: The central idea is that memory modification isn't just about strengthening or weakening associations, but about inferring the underlying causes of events. When new sensory data is encountered, the brain infers whether it was generated by an existing latent cause or a new one.
  • Latent Causes as the Key to Malleability: The eligibility of a memory trace for modification hinges on the inference of its associated latent cause. If a new observation is attributed to an existing cause, the memory associated with that cause can be updated. If a new cause is inferred, a new memory is formed.
  • Extinction as Structure Learning: The model reframes extinction not as unlearning, but as a process where prediction errors can lead to the inference of new latent causes, effectively creating a new memory trace for the extinction context, rather than erasing the original acquisition memory.
  • The EM Algorithm as a Cognitive Process: The authors propose that the iterative nature of the EM algorithm, alternating between structure and associative learning, provides a normative account of how the brain might dynamically update memories. This "rumination" process is crucial for explaining phenomena like the Monfils-Schiller effect.
  • Temporal Dynamics Matter: The inclusion of a temporal kernel (power law) highlights the importance of the timing of events in memory modification, explaining phenomena like spontaneous recovery and the age-dependency of memory updating.

Key Results

The model successfully reproduces a wide range of experimental findings in Pavlovian conditioning, including:

  • Retrieval-Induced Disruption: The model predicts that protein synthesis inhibitors (PSIs) disrupt memories when administered shortly after retrieval (when the memory is considered labile due to inferred latent cause activity), but not when delayed.
  • Monfils-Schiller Paradigm: The model explains how a short interval between retrieval and extinction training leads to persistent fear reduction, while very short or very long intervals do not. This is attributed to the EM algorithm's iterative updates during the interval, which effectively weaken the CS-US association for the acquisition cause only under specific temporal conditions.
  • Boundary Conditions: The model accounts for why older memories, stronger memories, and memories with less reexposure are more resistant to modification.
  • Spontaneous Recovery and Transience of Amnesia: The temporal dynamics of latent cause inference, driven by the power law kernel, explain the gradual recovery of extinguished responses over time and the transient nature of amnesia.
  • State-Dependency: The model can accommodate state-dependency by treating the amnestic agent itself as an additional CS.

Limitations and Future Work

  • Abstract Level of Analysis: The authors acknowledge that their theory is formulated at an abstract, cognitive level and does not make strong claims about biologically plausible neural implementations.
  • Neural Implementation: While they propose a tentative mapping onto brain circuits (amygdala, hippocampus, VTA), a detailed neural implementation is a future direction.
  • Parameter Sensitivity: While the model reproduces qualitative patterns, quantitative accuracy can be parameter-dependent. They suggest further work on parameter sensitivity and fitting to specific datasets.
  • Controversial Findings: The authors note that some findings in the literature (e.g., regarding the Monfils-Schiller paradigm) are controversial, with replication failures. They suggest their framework might help delineate systematic boundary conditions for these effects.
  • Complete Spontaneous Recovery: The model does not fully explain instances of complete spontaneous recovery, suggesting this might require additional mechanisms.
  • Future Work: The authors explicitly mention exploring biologically plausible neural implementations and applying the model to more complex memory phenomena. They also suggest further experimental work to test predictions regarding diagnosticity and the role of context similarity.

Summary updated

Abstract

Retrieving a memory can modify its influence on subsequent behavior. We develop a computational theory of memory modification, according to which modification of a memory trace occurs through classical associative learning, but which memory trace is eligible for modification depends on a structure learning mechanism that discovers the units of association by segmenting the stream of experience into statistically distinct clusters (latent causes). New memories are formed when the structure learning mechanism infers that a new latent cause underlies current sensory observations. By the same token, old memories are modified when old and new sensory observations are inferred to have been generated by the same latent cause. We derive this framework from probabilistic principles, and present a computational implementation. Simulations demonstrate that our model can reproduce the major experimental findings from studies of memory modification in the Pavlovian conditioning literature.
BibTeX
@inproceedings{gershman2017computational,
  author = {Samuel J. Gershman and Marie‐H. Monfils and Kenneth A. Norman and Yael Niv},
  title = {{The computational nature of memory modification}},
  booktitle = {eLife},
  year = {2017},
  doi = {10.7554/elife.23763},
}
Figure