Abstract
Biological sequences, such as birdsong or speech, are inherently probabilistic. While the brain encodes these statistics, the local unsupervised mechanisms allowing synapses to capture intermediate probabilities remain poorly understood. This study demonstrates that the biophysical site of synaptic competition determines whether a network learns forward or backward transition probabilities. By balancing competitive forces (differentiation) with homogenizing forces (stabilization), Hebbian plasticity can accurately encode the complex probability distributions found in natural behaviors.
Introduction
Biological life is composed of sequences—from phonemes in speech to motor commands in a dance. These sequences are rarely deterministic; they are probabilistic Markovian chains where transitions between states occur with specific likelihoods. Standard Hebbian learning ("neurons that fire together, wire together") often suffers from a "winner-take-all" problem, driving synaptic weights to binary extremes (0 or 1) and destroying the nuanced information of intermediate probabilities. This research investigates the biophysical conditions under which local plasticity rules can stably encode the full range of transition probabilities.
Methods
The researchers employed a three-tiered approach to bridge theory and biology:
- Analytic Proofs: Using a simplified model of binary neurons to mathematically prove that correlation-based Hebbian rules can converge to conditional probabilities.
- Rate-Based Simulations: Implementing a Hebbian Covariance Plasticity (HCP) rule in recurrent networks. This rule is sensitive to deviations from mean firing rates. Two key parameters were manipulated: $\alpha$ (the ratio of synaptic depression to potentiation) and $\beta$ (the weight-dependence of the change).
- Spiking Network Simulations: Testing Spike-Timing Dependent Plasticity (STDP) in networks of leaky integrate-and-fire neurons to ensure biological plausibility.
Crucially, they tested two "sites" of competition: pre-synaptic (limiting the total output weight of a single neuron) and post-synaptic (limiting the total input weight a single neuron can receive).
Datasets
The study utilized both synthetic and ethological data. Transition matrices were derived from the songs of 16 adult male Bengalese Finches, known for their complex, probabilistic syntax. Additionally, synthetic Gaussian matrices were used to create 19-state Markov chains with varying entropy, ranging from deterministic (0 bits) to nearly uniform (4.25 bits).
Key Insights and Results
The most striking finding is that the site of competition dictates the "arrow of time." Pre-synaptic competition leads the network to learn forward probabilities ($P(\text{future} | \text{present})$), while post-synaptic competition leads to learning backward probabilities ($P(\text{past} | \text{present})$). This suggests a biophysical link between a neuron's structure and its computational role, such as prediction versus recognition.
The balance between $\alpha$ (competitive force) and $\beta$ (homogenizing force) acts as a biophysical "prior." High competitive force favors deterministic, low-entropy environments, while high homogenizing force favors noisy, high-entropy environments. Stability requires this balance; without a homogenizing force (typically multiplicative weight dependence), synapses binarize, and probabilistic structure is lost.
When optimized, learned synaptic weights were highly correlated with true transition probabilities ($R = 0.97$ for forward sequences). The authors identified an "80% rule," where a single balanced set of plasticity parameters could accurately learn roughly 80% of different tested distributions, suggesting biological systems do not require perfect tuning for every task.
Discussion
While the study provides a robust framework for understanding probabilistic encoding, it relies on a strong external "teaching signal" to drive sequences. Future work should explore the interaction between learned weights and internal dynamics in autonomous systems. Furthermore, the study focused on 1st-order Markov sequences; real-world sequences often involve longer-term dependencies. Experimental validation is also needed to confirm if specific brain regions, such as motor versus sensory areas, exhibit the predicted pre- or post-synaptic competitive biases.
Summary updated