A Synaptical Story of Persistent Activity with Graded Lifetime in a Neural System

Yuanyuan Mi, Luozheng Li, Dahui Wang, Si WuNeurIPS 2014 · OpenReview

Overview

Researchers have proposed a novel model for working memory that challenges the traditional reliance on "permanent" attractor states. By balancing short-term synaptic facilitation (STF) and depression (STD), they demonstrate how a neural network can enter a "marginally unstable" state. This state holds information for a specific, tunable duration before naturally fading away, elegantly eliminating the need for an external "shut-off" signal to clear the memory.

The Problem: Turning Off Working Memory

In neuroscience, persistent activity is the phenomenon where neurons continue to fire even after a stimulus is removed. This sustained firing is widely considered the biological substrate of working memory.

The conventional view relies on Attractor Networks: once a stimulus pushes the network into an "active" state, positive feedback loops keep it there indefinitely. However, this model has a major biological flaw: how does the brain turn it off? Real-world tasks—like remembering a phone number for a few seconds—require memories to last for a specific time and then disappear. Standard attractor models require a strong, global inhibitory "reset" signal to clear the memory, a mechanism that often feels biologically unrealistic. The authors sought a mechanism where memory has a "graded lifetime"—staying active only as long as needed and then terminating naturally.

The Mechanism: Short-Term Plasticity

The authors employed a combination of mathematical modeling and computer simulation, utilizing a network of 1,000 "Integrate-and-Fire" neurons. To make the math tractable, they used a mean-field approximation to simplify the complex spiking of thousands of neurons into differential equations describing the average firing rate.

The "secret sauce" of their model is Short-Term Plasticity (STP). They incorporated two opposing types of STP:

  • Short-Term Depression (STD): Synapses get weaker as they are used, acting as a "brake" due to neurotransmitter depletion.
  • Short-Term Facilitation (STF): Synapses get stronger as they are used, acting as an "accelerator" due to calcium buildup.

Key Insights and Discoveries

The "Marginally Unstable" State and Natural Termination

Instead of creating a deep mathematical "valley" (a stable attractor) where the network gets stuck, the authors tuned the system to a "plateau." In this state, the network is technically unstable, but it decays so slowly that it mimics persistent activity for a long time. Because the state is marginally unstable, the activity eventually "falls off" the plateau and returns to silence on its own.

Firing rate over time showing a natural drop-off

Tunable Lifetimes

The "lifetime" of the memory isn't hard-coded. By adjusting the time constants of the synapses ($\tau_f$ for facilitation and $\tau_d$ for depression), the system can hold a memory for 500 milliseconds, 2 seconds, or indefinitely.

Heatmap showing how synaptic time constants dictate memory lifetime

This functional diversity explains why different parts of the brain (like the prefrontal cortex versus the sensory cortex) might have different ratios of STF and STD—they are optimized for different memory durations. It also suggests the brain could use neuromodulators, like dopamine, to dynamically "set" the expiration date of a memory.

Mathematical and Simulated Proof

The authors derived a formula that accurately predicts the lifetime of the persistent activity based on synaptic parameters. They identified a critical connection strength threshold: if connections are too weak, the memory dies instantly; if too strong, it becomes a permanent attractor. The "sweet spot" is just below this critical threshold. Crucially, the theoretical predictions from their mean-field equations matched the behavior of the 1,000-neuron spiking simulation perfectly.

Limitations and Future Directions

While elegant, the model has limitations. It assumes a "homogeneous" network where neurons are randomly and sparsely connected, whereas real cortical architecture is highly structured and modular. Additionally, the relationship between synaptic input and firing rate was assumed to be a simple linear-threshold function, whereas real neurons have more complex, non-linear input-output curves.

Future work will need to investigate how the brain might actively regulate these lifetimes through neuromodulators in vivo. Further experimental studies are also needed to confirm if biological synapses are indeed tuned to these "marginally unstable" regimes during active working memory tasks.

Summary updated

Abstract

Persistent activity refers to the phenomenon that cortical neurons keep firing even after the stimulus triggering the initial neuronal responses is moved. Persistent activity is widely believed to be the substrate for a neural system retaining a memory trace of the stimulus information. In a conventional view, persistent activity is regarded as an attractor of the network dynamics, but it faces a challenge of how to be closed properly. Here, in contrast to the view of attractor, we consider that the stimulus information is encoded in a marginally unstable state of the network which decays very slowly and exhibits persistent firing for a prolonged duration. We propose a simple yet effective mechanism to achieve this goal, which utilizes the property of short-term plasticity (STP) of neuronal synapses. STP has two forms, short-term depression (STD) and short-term facilitation (STF), which have opposite effects on retaining neuronal responses. We find that by properly combining STF and STD, a neural system can hold persistent activity of graded lifetime, and that persistent activity fades away naturally without relying on an external drive. The implications of these results on neural information representation are discussed.
BibTeX
@inproceedings{mi2014synaptical,
  author = {Yuanyuan Mi and Luozheng Li and Dahui Wang and Si Wu},
  title = {{A Synaptical Story of Persistent Activity with Graded Lifetime in a Neural System}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2014},
  url = {https://openreview.net/forum?id=HyNuWKWdbB},
}
Figure