In neural network optimization, a fundamental tension exists between locality and global coordination. Traditional Hebbian learning is entirely local: a synapse updates its weight based solely on the correlation between pre-synaptic and post-synaptic activity. While biologically realistic, this "two-factor" approach is blind to global goals, making it incapable of solving complex reinforcement learning or deep credit assignment problems. As noted by Friedemann Zenke et al. in Hebbian and non-Hebbian plasticity orchestrated to form and retrieve memories in spiking networks, purely autonomous local rules inevitably suffer from runaway activity because they lack global regulatory feedback.
Conversely, error backpropagation solves global credit assignment elegantly but relies on non-local information, symmetric backward weights, and distinct forward/backward phases. These requirements are biologically implausible and introduce severe memory bottlenecks in neuromorphic hardware.
Three-factor learning rules resolve this dilemma. By introducing a third, globally or contextually broadcasted signal to gate local Hebbian updates, these rules allow distributed networks to optimize for global objectives entirely in situ.
The Anatomy of a Three-Factor Update
To understand why three-factor learning works, consider the mathematical intuition behind a synaptic weight update ($\Delta w_{ij}$) between a pre-synaptic neuron $i$ and a post-synaptic neuron $j$. Instead of relying on two local factors, the update is formulated as a joint product of three distinct signals:
$$\Delta w_{ij} \propto \text{Factor 1 (Pre)} \times \text{Factor 2 (Post)} \times \text{Factor 3 (Modulator)}$$
Each factor serves a specific computational purpose:
- The Pre-synaptic Factor: A trace of recent spikes or activations from the sending neuron, capturing historical input context.
- The Post-synaptic Factor: The current state, firing rate, or membrane potential of the receiving neuron. In deep spiking neural networks (SNNs), this factor often incorporates a surrogate gradient to estimate derivatives through non-differentiable spiking thresholds, a mechanism leveraged by Melika Payvand et al. in Error-triggered Three-Factor Learning Dynamics for Crossbar Arrays and Fernando M. Quintana et al. in ETLP: event-based three-factor local plasticity for online learning with neuromorphic hardware.
- The Third Factor (The Modulator): An extrinsic signal broadcasted to the synapse. As reviewed by Nicolas Frémaux in Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules, this third factor acts as a strict global gate. Local coincidences between pre- and post-synaptic neurons are transiently ignored unless this third signal validates them.
Bridging Time with Eligibility Traces
A major hurdle in credit assignment—especially in reinforcement learning—is the temporal gap between cause and effect. If an agent takes an action now, but the reward or error signal arrives seconds later, how does a local synapse know it was responsible for the outcome?
Three-factor rules solve this via an eligibility trace. When pre- and post-synaptic neurons fire together, they do not immediately alter the synaptic weight. Instead, they generate a temporary, decaying biochemical or digital footprint at the synapse, flagging it as "eligible" for future modification.
Pre-synaptic Spike ──┐
├─► Local Coincidence ─► Eligibility Trace ──┐
Post-synaptic Spike ──┘ │
▼
Delayed Global Modulator (Reward/Error) ────────────────────────► [Weight Update]
When the third factor finally arrives, it interacts with this active eligibility trace to compute the permanent weight change. Eleni Vasilaki et al. demonstrated the power of this mechanism in Spike-based reinforcement learning of navigation, showing how it enables spiking networks to solve complex spatial navigation tasks. Similarly, Barna Zajzon et al. confirmed in Toward reproducible models of sequence learning: replication and analysis of a modular spiking network with reward-based learning that eligibility traces successfully bridge the temporal gap, allowing networks to learn both the order and duration of sequential events.
Functional Diversity of the Third Factor
The mathematical framework of three-factor plasticity is highly versatile because the third factor can represent entirely different types of information depending on the target application.
Reinforcement Learning and Actor-Critic Frameworks
In reward-modulated systems, the third factor typically represents a global dopamine signal acting as a Temporal Difference (TD) error. Abigail Morrison et al. implemented this in A spiking temporal-difference learning model based on dopamine-modulated plasticity to build biologically plausible actor-critic architectures. Even when this biological feedback is constrained—such as having a low baseline firing rate that limits negative error representation—the authors of An Imperfect Dopaminergic Error Signal Can Drive Temporal-Difference Learning proved that the three-factor interaction remains robust enough to drive complex behavioral adaptation.
Surprise and Environmental Volatility
The third factor can also encode "surprise"—the statistical mismatch between expected and actual sensory inputs. In Surprise-based learning: neuromodulation by surprise in multi-factor learning rules, the authors demonstrated that using surprise as a meta-signal to gate plasticity allows networks to adapt to non-stationary environments far faster than standard models. Abigail Morrison expanded on this in Fast adaptation to rule switching using neuronal surprise, showing that a surprise signal emerging naturally from excitation-inhibition imbalances enables networks to achieve "one-shot" adaptation to shifting environmental rules without overwriting older memories.
Approximating Backpropagation
To train deep networks without a standard backward pass, the third factor can be engineered as a parallel error broadcast. Mete Erdogan et al. developed Error Broadcast and Decorrelation as a Potential Artificial and Natural Learning Mechanism, proving that multiplying local post-synaptic derivatives by a globally broadcasted error signal eliminates the need for symmetric weight transport entirely.
Hardware Efficiency and In Situ Learning
Beyond biological modeling, three-factor learning rules provide a massive efficiency breakthrough for neuromorphic hardware. Traditional deep learning requires constantly moving weight and gradient data between memory arrays and processors, incurring a massive energy penalty.
Because three-factor rules rely on locally available pre- and post-synaptic signals combined with a single broadcasted scalar, the entire weight update can be computed directly within memory cells.
┌────────────────────────┐
│ Global Error/Reward │
└───────────┬────────────┘
│ (Single Broadcast Line)
▼
┌─────────────────────────────────────────────────────────────┐
│ Memristor Crossbar Array │
│ │
│ Pre-synaptic Trace ──► [ Synapse (In Situ Update) ] │
│ ▲ │
│ │ │
│ Post-synaptic State ─────────────┘ │
└─────────────────────────────────────────────────────────────┘
Kevin Portner et al. demonstrated this hardware advantage in Actor–critic networks with analogue memristors mimicking reward-based learning, showing that executing three-factor updates directly within analogue memristor crossbars can reduce energy consumption by up to 39x compared to traditional GPU training. This decentralized architecture allows physical neuromorphic systems, such as those evaluated by Korbinian Schreiber et al. in Closed-loop experiments on the BrainScaleS-2 architecture, to interact with and adapt to real-world environments in real time at micro-watt power scales.
Key Foundational Work
Correction: Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail (2009) — contributed 1 concept elements to this direction.
Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail (2009) — contributed 1 concept elements to this direction.
Spike-based reinforcement learning of navigation (2008) — contributed 1 concept elements to this direction.