Short-term plasticity recalls forgotten memories through a trampoline mechanism

Massimo Del Gaudio, Federico Ghimenti, Surya GanguliarXiv 2025 · DOI · arXiv

TL;DR

This paper reveals how short-term synaptic plasticity (STP) allows neural networks to recall memories that would normally be lost to "catastrophic forgetting." By dynamically reshaping the network's energy landscape—a process the authors liken to a "trampoline mechanism"—STP traps transient signals near stored patterns, effectively enabling the retrieval of memories even when the network is loaded far beyond its theoretical static capacity.

What Problem Does This Paper Address?

In classical associative memory models like the Hopfield network, there is a hard limit to how much information can be stored. Once the number of stored patterns exceeds a "critical capacity" (roughly 14% of the number of neurons), the network undergoes a phase transition into a "spin-glass" state. In this state, the network experiences catastrophic forgetting: instead of settling into a stored memory, the dynamics get lost in a forest of "spurious states"—meaningless noise that shares no overlap with the original memories.

While biological brains clearly use long-term Hebbian learning to store memories, their synapses are also highly dynamic on short timescales. The authors sought to understand if this short-term plasticity (STP) could help the network "rescue" memories that are technically still there but are no longer reachable via standard fixed-weight dynamics.

Methods

The researchers analyzed a continuous-time Hopfield Recurrent Neural Network (RNN) where the total synaptic weight is the sum of a fixed, long-term Hebbian matrix and a rapidly evolving, activity-dependent short-term matrix.

To study this system, they employed three powerful tools from theoretical physics and nonlinear dynamics:

  1. The Cavity Method: Used to perform a static analysis of the network’s fixed points and determine the phase boundaries between retrieval and forgetting.
  2. Dynamical Mean Field Theory (DMFT): A sophisticated framework that allowed them to track the statistical evolution of the network's state over time, rather than just looking at the final result.
  3. Numerical Simulations: They validated their theoretical predictions using large-scale simulations of networks with up to 20,000 neurons.

Datasets

As a theoretical study in computational neuroscience, the paper does not use external real-world datasets (like ImageNet). Instead, it uses synthetic memory patterns: $P$ independent, identically distributed (i.i.d.) random binary vectors $\xi \in \{ \pm 1 \}^N$. The performance is measured by the "overlap" (a measure of similarity) between the network's current state and a target memory pattern.

Key Insights and Ideas

  • The Trampoline Mechanism: This is the paper's central metaphor. In a standard network, the energy landscape is "solid." If a memory isn't a deep enough valley, the "ball" (the network's state) rolls past it. With STP, the landscape becomes reactive, like a trampoline. When the network state moves near a memory, the synapses rapidly adjust to "dent" the landscape at that location, deepening the valley and trapping the state there.
  • Static Analysis is Deceiving: If you only look at the fixed points (the static "capacity"), STP seems to offer only marginal improvements. The real benefit is hidden in the transient dynamics.
  • Trapping the Escapees: Even above the critical capacity, a network often briefly moves toward a stored memory before being pulled away by noise. STP acts during this brief window, "catching" the transient signal before it can escape into the spin-glass phase.
  • The Goldilocks Timescale: There is an optimal time constant ($p$) for plasticity. If the synapses change too fast, they track noise and prevent stable representations. If they change too slowly, they miss the transient window and the memory is lost. The authors found that the best results occur when the synaptic timescale matches the natural decay time of the network's transient retrieval.

Key Results

  • Retrieval Above Capacity: The authors demonstrated "plastic retrieval," where a network loaded at $\alpha = 0.20$ (well above the standard limit of $\alpha_c \approx 0.138$) could still successfully recall memories.
  • Phase Diagram: They mapped out a new phase diagram showing that while STP doesn't significantly move the absolute limit of what can be stored, it dramatically expands the "retrieval phase" for moderate neuronal gains.
  • Stability: Jacobian analysis confirmed that STP creates stable basins of attraction around memories that would otherwise be unstable, effectively "freezing" the otherwise chaotic dynamics of an overloaded network.

Limitations and Future Work

  • Hebbian Assumptions: The model relies on a specific form of associative short-term plasticity. Whether more complex, biologically diverse forms of STP (like facilitation or depression) produce the same "trampoline" effect remains to be seen.
  • Structured Backgrounds: The study assumes a relatively unstructured connectivity background. In real brains, short-term plasticity acts on top of highly structured, pre-trained long-term weights.
  • Modern Hopfield Networks: A promising future direction is applying these insights to "Dense Associative Memories" or Modern Hopfield Networks, which are mathematically related to the Attention mechanisms in Transformers. Adding STP to these models could potentially improve the efficiency or robustness of AI architectures.

Summary updated

Abstract

We analyze continuous Hopfield associative memories augmented by additional, rapid short-term associative synaptic plasticity. Through the cavity method, we determine the boundary between the retrieval and forgetting, or spin-glass phase, of the network as a function of the fraction of stored memories and the neuronal gain. We find that short-term synaptic plasticity yields marginal improvements in critical memory capacity. However, through dynamical mean field theory, backed by extensive numerical simulations, we find that short-term synaptic plasticity has a dramatic impact on memory retrieval above the critical capacity. When short-term synaptic plasticity is turned on, the combined neuronal and synaptic dynamics descends a high-dimensional energy landscape over both neurons and synapses. The energy landscape over neurons alone is thus dynamic, and is lowered in the vicinity of recent neuronal patterns visited by the network, just like the surface of a trampoline is lowered in the vicinity of regions recently visited by a heavy ball. This trampoline-like reactivity of the neuronal energy landscape to short-term plasticity in synapses can lead to the recall of stored memories that would otherwise have been forgotten. This occurs because the dynamics without short-term plasticity transiently moves towards a stored memory before departing away from it. Thus short-term plasticity, operating during the transient, lowers the energy in the vicinity of the stored memory, eventually trapping the combined neuronal and synaptic dynamics at a fixed point close to the stored memory. In this manner, short-term plasticity enables the recall of memories that would otherwise be forgotten, by trapping transients that would otherwise escape. We furthermore find an optimal time constant for short-term synaptic plasticity, matched to the transient dynamics, to empower recall of forgotten memories.
BibTeX
@misc{gaudio2025short,
  author = {Massimo Del Gaudio and Federico Ghimenti and Surya Ganguli},
  title = {{Short-term plasticity recalls forgotten memories through a trampoline mechanism}},
  year = {2025},
  eprint = {2511.22848},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.22848},
  howpublished = {arXiv},
}
Figure