Stability of working memory in continuous attractor networks under the control of short-term plasticity

Alexander Seeholzer, Moritz Deger, Wulfram Gerstner

Abstract

Continuous attractor models are fundamental to our understanding of how the brain stores continuous-valued information, such as spatial locations or visual orientations. However, these models are notoriously sensitive to noise, which causes stored memories to degrade over time. This research provides a comprehensive theoretical framework to describe how short-term synaptic plasticity (STP)—specifically facilitation and depression—governs the stability of these memory representations. By reducing complex spiking network dynamics to a one-dimensional Langevin equation, the authors demonstrate that synaptic facilitation acts as a powerful stabilizer against both random diffusion and systematic drift, while synaptic depression increases volatility.

Introduction: The Fragility of Continuous Memory

In computational neuroscience, Continuous Attractor Neural Networks (CANNs) are the standard paradigm for working memory. In these networks, a "bump" of neural activity represents a specific value along a continuous manifold. While theoretically elegant, CANNs face two existential threats in biological systems:

  1. Diffusion (Fast Noise): Stochastic neuronal spiking causes the memory bump to wander randomly like a random walk, leading to a loss of precision over time.
  2. Drift (Frozen Noise): Biological "hardware" is rarely perfect. Heterogeneity in neurons or random synaptic connection strengths creates energy landscapes where the bump is pulled toward specific "gutters," destroying the continuity of the memory store.

A Unified Theoretical Framework

To address these challenges, the researchers developed a mathematical bridge between microscopic synaptic properties and macroscopic network stability. They reduced the high-dimensional dynamics of a spiking neural network into a simple, one-dimensional Langevin equation:

$$\dot{\phi} = \sqrt{B} \eta(t) + A(\phi)$$

Where $\phi$ represents the memory position, $B$ denotes the diffusion strength, and $A(\phi)$ represents the drift field. The core contribution of this work is the derivation of analytical expressions for $B$ and $A$ that explicitly incorporate parameters for short-term facilitation (STF) and short-term depression (STD), along with their respective time constants.

The Antagonistic Roles of Facilitation and Depression

The study reveals that STP functions as a biological "control knob" for memory stability. Facilitation and depression play diametrically opposite roles in the maintenance of the memory bump:

  • Facilitation as a Stabilizer: STF effectively "pins" the activity bump to its current location. By increasing the synaptic strength of active neurons, it creates a local attractor that resists both random noise and the pull of network heterogeneities. The theory predicts that increasing facilitation can reduce the diffusion constant and the magnitude of directed drifts by over an order of magnitude.
  • Depression as a Destabilizer: Conversely, STD makes the memory bump "slippery." As active synapses tire, the bump becomes more susceptible to wandering or being captured by fixed-point attractors caused by hardware imperfections.

Key Results and Biological Implications

The theoretical predictions were validated using large-scale simulations of Leaky Integrate-and-Fire (LIF) neurons with conductance-based synapses. Several key insights emerged:

  • Unified Scaling: A remarkable finding is that the same mathematical factors scale both diffusion and drift. This implies a fundamental link in how the brain manages different types of uncertainty; if facilitation suppresses random noise, it simultaneously mitigates systematic errors from uneven connectivity.
  • Network Scale and Realism: To match the levels of memory stability observed in human behavioral studies (e.g., drift rates under 1 degree per second) in the presence of realistic biological heterogeneity, the theory predicts a required network size of approximately $10^5$ neurons. This aligns well with the estimated size of cortical columns in the mammalian brain.
  • Information Retention: Using Mutual Information (MI) metrics, the authors showed that strong facilitation significantly slows the decay of information over delay periods, effectively "rescuing" the network's functional capacity as a reliable memory store.

Conclusion and Future Directions

This work provides a rigorous foundation for understanding how synaptic-level processes like STP enable stable cognitive functions. While the current theory assumes the network is near a stationary state, it sets the stage for future research into transient dynamics and the integration of other stabilizers like firing-rate adaptation. Beyond neuroscience, these insights offer potential applications in neuromorphic engineering, where "frozen noise" in hardware often limits the performance of artificial attractor networks. By modulating facilitation—perhaps via neuromodulators like dopamine in the brain—systems can transition between a flexible "update" mode and a stable "storage" mode.

Summary updated

Abstract

Abstract Continuous attractor models of working-memory store continuous-valued information in continuous state-spaces, but are sensitive to noise processes that degrade memory retention. Short-term synaptic plasticity of recurrent synapses has previously been shown to affect continuous attractor systems: short-term facilitation can stabilize memory retention, while short-term depression possibly increases continuous attractor volatility. However, it currently remains unclear to which degree these two short-term plasticity mechanisms interact, what their combined quantitative effect on working memory stability is, and whether these effects persist in neuronal networks with spike-based transmission. Here, we present a comprehensive description of the effects of short-term plasticity on noise-induced memory degradation in one-dimensional continuous attractor models. Our theoretical description, applicable to spiking and rate-based models alike, accurately describes the slow dynamics of stored memory positions in separate processes of diffusion due to spiking variability and drift due to sparse connectivity and neuronal heterogeneity. We find that facilitation decreases both diffusion and directed drifts, while short-term depression tends to increase both. Using mutual information, we evaluate the combined impact of short-term facilitation and depression on the ability of networks to retain stable working memory. Finally, our theory establishes links to experiments: we are able to predict the sensitivity of continuous working memory to distractor inputs and place constraints on network and synapse properties necessary to implement stable working memory. Author summary The ability to transiently memorize positions in the visual field is crucial for behavior. Models and experiments have shown that such memories can be maintained in networks of cortical neurons with a continuum of possible activity states, that reflects the continuum of positions in the environment. However, the accuracy of positions stored in such networks will degrade over time due to the noisiness of neuronal signaling and imperfections of the biological substrate. Previous work in simplified models has shown that synaptic short-term plasticity could stabilize this degradation by dynamically up- or down-regulating the strength of synaptic connections, thereby ”pinning down” memorized positions. Here, we present a general theory that accurately predicts the extent of this ”pinning down” by short-term plasticity in a broad class of biologically plausible models, thereby untangling the interplay of varying biological sources of noise with short-term plasticity. Importantly, our work provides a direct and novel theoretical link from the microscopic substrate of working memory – neurons and synaptic connections – to observable behavioral correlates. This allows us to constrain properties of cortical networks that are currently hard to assess experimentally, which we hope will help guide future theoretical and experimental work.
BibTeX
@article{seeholzer2019stability,
  author = {Alexander Seeholzer and Moritz Deger and Wulfram Gerstner},
  title = {{Stability of working memory in continuous attractor networks under the control of short-term plasticity}},
  journal = {PLOS Computational Biology},
  year = {2019},
  doi = {10.1371/journal.pcbi.1006928},
}
Figure