Abstract
Synaptic plasticity is implemented and controlled through over a thousand different types of molecules in the postsynaptic density and presynaptic boutons. One of the most prominent molecules in the postsynaptic density is CaMKII, described in molecular biology as a "memory molecule" that can integrate calcium-influx signals through autophosphorylation on a time scale of dozens of seconds. We show that the experimental data on CaMKII activation suggest a general functional role in speeding up reward-guided search for network configurations that maximize reward expectation. Our theoretical analysis shows that CaMKII contributes a momentum term that substantially speeds up stochastic search, allowing the network to overcome saddle points of the fitness function. This improved stochastic policy search can be understood as Hamiltonian sampling, one of the most efficient stochastic search methods known.
Introduction
Learning in the brain is inherently noisy and stochastic. Previous theoretical frameworks, such as "synaptic sampling," suggested that the brain optimizes its connections by performing Langevin sampling—essentially a version of stochastic gradient descent where noise is added to help explore the parameter space. However, Langevin sampling is often slow in the high-dimensional "fitness landscapes" of neural networks, where learning frequently gets stuck at saddle points. This paper investigates whether specific biological molecules, like the protein kinase CaMKII, provide a functional "speed boost" to help the brain overcome these optimization hurdles.
The Biological Momentum of CaMKII
The researchers developed a mathematical model integrating the known dynamics of CaMKII into synaptic plasticity rules. Instead of updating synaptic weights ($\theta_i$) directly based on a reward signal, the reward signal updates the concentration of activated CaMKII ($\Gamma_i$), which then slowly drives the change in synaptic weight.
Because CaMKII stays active for dozens of seconds, it acts as a low-pass filter for calcium signals. In optimization terms, this temporal integration is equivalent to a momentum term. The authors proved using Stochastic Differential Equations that this dynamics implements Hamiltonian sampling with friction, a well-known method in physics and machine learning for faster convergence compared to standard Langevin sampling.
Results and Performance
The model was evaluated across several computational tasks, including sigmoidal emulation, a blind reaching task, and classification on the MNIST dataset.
Key results include:
- Faster Learning: In sigmoidal emulation and reaching tasks, Hamiltonian dynamics (with CaMKII) reached high reward levels significantly faster than standard synaptic sampling.
- Escaping Plateaus: In the MNIST task, non-Hamiltonian networks frequently hit performance plateaus at saddle points. When switched to Hamiltonian dynamics, the same networks escaped these plateaus almost immediately.
- Global Optimization: In the XOR task, using a "cooling" schedule (simulated annealing) with Hamiltonian sampling allowed the network to find the optimal solution in 90% of trials, compared to only 40% when the temperature was kept constant.
Conclusion
This work demonstrates that a biological implementation (CaMKII molecules) directly enables a specific algorithm (Hamiltonian sampling) to solve a fundamental computational problem (reward-based optimization). By providing momentum, CaMKII allows the network's parameters to "slide" through flat regions of the fitness landscape rather than getting stuck, significantly accelerating the brain's ability to learn and adapt.
Summary updated