TL;DR
This paper presents a theoretical framework and simulation results for a biologically plausible model of working memory in spiking neural networks. It demonstrates how such networks can achieve both irregular neural firing, characteristic of real brains, and store a significant number of memories, with storage capacity scaling linearly with the number of excitatory connections.
What Problem Does This Paper Address?
Understanding how the brain implements working memory – the ability to hold and manipulate information over short timescales (seconds to minutes) – is a fundamental challenge in neuroscience. While attractor networks are a leading theoretical candidate, existing models often struggle to reconcile key biological observations:
- Irregular Firing: Real neurons involved in working memory tasks exhibit highly irregular firing patterns, which is difficult to reproduce in simple attractor network models that tend to produce regular firing.
- Biologically Realistic Conditions: Many models violate biological constraints like Dale's Law (neurons are either exclusively excitatory or inhibitory) or assume unrealistic connectivity patterns.
- Storage Capacity: The number of memories a network can store under biologically realistic conditions is not well understood.
This paper aims to bridge this gap by developing and testing a model of a spiking neural network that addresses these issues.
Methods
The authors employ a combination of theoretical analysis and large-scale simulations:
- Mean-Field Analysis: They use self-consistent signal-to-noise analysis to derive mean-field equations that describe the network's behavior. This approach treats synaptic inputs as Gaussian random variables and solves for their means and variances self-consistently.
- Spiking Neural Network Model: They simulate a network of conductance-based quadratic integrate-and-fire (QIF) neurons, which are a more biologically realistic model than simpler neuron models.
- Connectivity Model: The network connectivity is designed to be sparse and to incorporate both Hebbian learning for memory storage and a background connectivity structure that aims to achieve a "balanced regime."
- Scaling Analysis: A key aspect of their theoretical approach is to analyze how network properties (like synaptic strengths) should scale with the number of neurons and connections to maintain balanced activity and irregular firing in large, realistic networks.
Datasets
Not discussed in the paper. The study focuses on a simulated neural network model rather than real-world datasets.
Key Insights and Ideas
- The "Balanced Regime" is Crucial for Irregular Firing: The paper strongly emphasizes that for both foreground (memory-related) and background neurons to fire irregularly, the network must operate in a "balanced regime." This means the mean synaptic input to a neuron is roughly on the same order as the fluctuations in that input, and this balance must be maintained independently of network size.
- Scaling is Key to Biological Plausibility: Achieving this balanced regime in large networks requires specific scaling of synaptic weights. Background connections should scale as $K^{-1/2}$ and memory-related connections as $K^{-1}$, where $K$ is the number of connections per neuron. This scaling ensures that the network's firing properties remain consistent as it grows.
- Storage Capacity Scales Linearly with Excitatory Connections: A significant finding is that the maximum number of memories a network can store ($p_{max}$) scales linearly with the number of excitatory connections per neuron ($K_E$), not the total number of neurons ($N_E$). This is a departure from some simpler models where capacity scales with $N_E$.
- Coding Level Influences Firing Rate: The "coding level" ($\alpha$), which represents the fraction of neurons involved in a memory, plays a critical role in determining the firing rate of memory-related neurons. Higher coding levels lead to lower, more biologically realistic firing rates during memory retrieval.
Key Results
- Irregular Firing Achieved: Simulations of large spiking neural networks, built according to the derived scaling principles, successfully exhibited irregular firing in both background and foreground neurons, quantified by a coefficient of variation (CV) around 0.8.
- Storage Capacity Proportional to $K_E$: The study found that the maximum number of storable memories ($p_{max}$) scales linearly with the number of excitatory connections per neuron ($K_E$). Specifically, simulations showed $p_{max} \approx 0.01 K_E$.
- Biologically Plausible Retrieval: The model demonstrated memory retrieval states with firing rates that could be adjusted to be biologically realistic (around 20 Hz) by tuning the coding level ($\alpha$).
Limitations and Future Work
- Foreground vs. Background Firing Rate: While the model achieves irregular firing, the foreground neurons (involved in memory) tend to fire at a higher rate than background neurons, a distinction that is less pronounced in some experimental recordings. The authors suggest this might be due to the simplified nature of their model (e.g., only two time scales) or the use of binary patterns.
- Fine-tuning of Synaptic Weights: The model requires specific scaling of synaptic weights during learning, which might imply a degree of fine-tuning. However, the authors show that the storage capacity is robust to some level of noise in this learning process.
- Future Work: The authors suggest investigating more realistic scenarios, such as using real-valued patterns instead of binary ones, incorporating more complex neuronal dynamics (e.g., multiple time scales), and exploring modular network architectures to potentially increase storage capacity beyond the linear scaling with $K_E$. They also note that the precise scaling of $\beta$ (memory strength) in simulations deviates slightly from theoretical predictions due to finite $K$ effects, which warrants further investigation.
Summary updated