In traditional machine learning, a synaptic weight is treated as a deterministic scalar value. Optimization is a quest to find a single point estimate—the perfect set of numbers—that minimizes a loss function.
Biological brains, however, operate on a fundamentally different substrate. Biological synapses are highly volatile, frequently failing to transmit signals, and continuously fluctuating in size and strength even in the absence of external stimuli. Rather than treating this pervasive noise as an evolutionary flaw or a limitation of biological "wetware," a growing body of research suggests that synaptic stochasticity is an algorithmic feature. By treating network parameters as stochastic variables, neural circuits can perform continuous mathematical optimization and maintain a representation of uncertainty through a strategy known as synaptic sampling.
The Mathematics of Volatility: Drift and Diffusion
To understand how a noisy synapse can compute reliably, we must look at how its changes are modeled over time. Instead of a deterministic gradient update, the trajectory of a synaptic weight can be formalized as a Stochastic Differential Equation (SDE). This framework splits synaptic plasticity into two distinct forces: a deterministic drift term and a stochastic diffusion term.
$$\mathrm{d}w = \mu(w, t)\mathrm{d}t + \sigma(w, t)\mathrm{d}W$$
Here, $\mu(w, t)$ represents the directed drift—driven by local learning rules like Spike-Timing-Dependent Plasticity (STDP) or global reward signals—while $\sigma(w, t)\mathrm{d}W$ represents random fluctuations modeled as a continuous noise process (such as a Wiener or Ornstein-Uhlenbeck process).
As explored by [Genki Shimizu] et al. in Computational Roles of Intrinsic Synaptic Dynamics, this formulation accounts for both directed learning and the inherent, activity-independent volatility observed in biology. While individual synaptic trajectories are random, the statistical behavior of the entire network remains highly predictable. By applying the Fokker-Planck equation, researchers can analytically track how the probability distribution of these weights evolves.
As demonstrated in early work by [David Horn 0001] et al. in Distributed Synchrony of Spiking Neurons in a Hebbian Cell Assembly and later by [Lars Buesing] et al. in Simplified Rules and Theoretical Analysis for Information Bottleneck Optimization and PCA with Spiking Neurons, the interaction between structured drift and random diffusion allows the network to settle into a stable stationary probability distribution rather than dissolving into chaos. This balance ensures that the network maintains structural stability while preserving its exploratory nature.
Escaping Optimization Traps
Why inject noise directly into the parameters of a network? The primary advantage becomes clear when visualizing a complex, non-convex error landscape. Deterministic gradient descent is notoriously susceptible to getting trapped in local minima or flat saddle points where the gradient vanishes, halting the learning process.
Deterministic Learning: [Initial State] ----> [Stuck in Saddle Point/Local Minimum]
Stochastic Learning: [Initial State] --(Noise)--> [Escapes Saddle] ----> [Global Optimum]
Synaptic noise provides the kinetic energy required to escape these optimization traps. [Ying Fang] et al. in Noise helps optimization escape from saddle points in the neural dynamics proved that synaptic fluctuations satisfy the mathematical "strict saddle" condition, meaning the injected noise guarantees that the network can escape from suboptimal flat regions.
Crucially, this noise is not completely uniform. In biological systems, larger, more consolidated synaptic connections exhibit greater absolute fluctuations. By scaling the variance of the noise proportionally to the magnitude of the synaptic weights—a mechanism modeled by [Ying Fang] et al. in Noise Helps Optimization Escape From Saddle Points in the Synaptic Plasticity—the network achieves a structured form of exploration. Small synapses can delicately probe local features, while large synapses undergo fluctuations that allow the global network structure to shake itself free from deep local minima, ultimately settling into flatter, more robust regions of the parameter space.
Representational Drift as Bayesian Exploration
If synaptic weights are in a state of constant flux, one might expect learned behaviors and memories to rapidly degrade. The resolution to this paradox lies in the concept of posterior sampling.
In Synaptic Sampling: A Bayesian Approach to Neural Network Plasticity and Rewiring, [David Kappel] et al. propose that the brain does not maintain a single "correct" weight configuration. Instead, there is an entire manifold of redundant weight configurations that can solve a given task equally well. By utilizing a Langevin-type equation, the synaptic weights continuously sample from this posterior distribution.
This framework provides a functional explanation for a widely observed biological phenomenon known as representational drift—the tendency of the specific neurons representing a task to change over days and weeks even when performance remains stable. As reviewed by [Paul Masset] et al. in Drifting neuronal representations: Bug or feature?, this drift is a feature of a system performing ongoing Bayesian inference.
When a network faces uncertainty or encounters a redundant solution space, it samples different valid configurations. This ongoing exploration prevents overfitting, enables continuous structural rewiring, and ensures that the network remains highly adaptable to sudden environmental changes. Rather than dismissing trial-to-trial variability as meaningless noise, the authors in Network Plasticity as Bayesian Inference demonstrate that this variability is the geometric signature of a network exploring its posterior distribution.
Communication Noise as a Computational Resource
Stochasticity in neural circuits operates across multiple timescales. While weight diffusion changes the network over hours and days, probabilistic synaptic transmission introduces randomness at the millisecond scale of individual spikes. When an action potential arrives at a biological synapse, the probability of successfully releasing a neurotransmitter is often less than 50%.
Shifting the source of randomness from the processing units (the neurons) to the communication channels (the synapses) yields massive computational and energy efficiencies:
- Monte Carlo Sampling: [Yanping Huang] et al. demonstrate in Neurons as Monte Carlo Samplers: Bayesian Inference and Learning in Spiking Networks that probabilistic release provides the exact stochastic substrate needed for a spiking network to act as a Monte Carlo sampler, allowing it to track and infer hidden environmental states in real time.
- Generative Modeling without Hardware Overhead: In conventional artificial networks, generating randomness requires dedicated noise-generation hardware or complex activation functions. By contrast, as shown by [Gert Cauwenberghs] in A learning framework for winner-take-all networks with stochastic synapses, synapses that randomly omit signals effectively transform deterministic spike trains into stochastic processes.
- Preventing Pathological Synchronization: In large recurrent networks, deterministic signaling can lead to catastrophic, over-synchronized cascades of firing. As shown by [Johannes Lengler] et al. in Reliable Neuronal Systems: The Importance of Heterogeneity, inherent synaptic unreliability breaks up these pathological loops, ensuring the network remains in a highly responsive, asynchronous state.
This mechanism allows feedforward and recurrent spiking networks to function as efficient generative models. As argued by [Bruno U. Pedroni] in Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines, leveraging transmission failures allows brain-inspired hardware to execute complex probabilistic inference at a fraction of the energy budget required by deterministic digital systems.
Exploration in Reinforcement Learning
The exploratory power of synaptic sampling extends naturally to closed-loop reinforcement learning (RL) environments. In an RL context, an agent must balance exploiting known rewards with exploring the environment to find better strategies.
Instead of relying on complex action-selection policies (like $\epsilon$-greedy action choices), a network built on synaptic sampling embeds exploration directly into its connectivity. [Jacques Kaiser] et al. in Embodied Synaptic Plasticity with Online Reinforcement learning demonstrate how synaptic weights can be modeled as stochastic variables that sample from a distribution to maximize a global reward signal.
When the agent receives a reward, a global signal (such as dopamine) interacts with local eligibility traces to alter the drift term of the SDE, anchoring successful weight configurations. This principle dates back to models like the one proposed by [Eleni Vasilaki] et al. in Spike-based reinforcement learning of navigation, which demonstrated that modulating the release probability of binary synapses can solve complex spatial navigation tasks. By adjusting the underlying probability distributions rather than fixed scalar values, the network maintains a diverse pool of potential behavioral strategies, allowing it to rapidly adapt when reward contingencies shift.
Key Foundational Work
Neuromorphic Bisable VLSI Synapses with Spike-Timing-Dependent Plasticity (2002) — contributed 1 concept elements to this direction.
Distributed Synchrony of Spiking Neurons in a Hebbian Cell Assembly (1999) — contributed 1 concept elements to this direction.