Deep learning’s success is built on backpropagation (BP), an algorithm that calculates precise gradients to update neural weights. However, BP is notoriously "biologically implausible." It requires weight symmetry (the backward pass must use the exact same weights as the forward pass) and creates a global locking problem where layers must wait for a sequential chain of calculations to finish.
Recent research seeks to bridge this gap by providing mathematical rigor to bio-inspired alternatives. The goal is to prove that local, decentralized rules—like those found in the brain—can achieve the same global optimization as backpropagation.
The Alignment Phenomenon
The most prominent alternative to backpropagation is Feedback Alignment (FA). Instead of using the transpose of the forward weights to communicate error signals, FA uses a fixed, random matrix. For years, it was a mystery why a network would listen to random noise.
Maria Refinetti and Sebastian Goldt describe this as a two-step process: align, then memorise. In The dynamics of learning with feedback alignment, they show that in the early stages of training, the forward weights actually rotate to align themselves with the fixed random feedback weights. Once aligned, the random feedback starts acting like a true gradient, allowing the network to "memorise" the data.
This alignment isn't just a lucky accident; it is driven by an underlying conservation law. Oluwasanmi O Koyejo and Zachary Robertson discovered that neural networks trained with FA undergo Implicit Regularization. The network naturally moves toward a state where the product of the forward and backward weights is maximized, forcing the system into a regime where local updates contribute to global error reduction.
Local Rules as Global Optimizers
If the brain doesn't use backpropagation, what does it use? Researchers are increasingly finding that biological mechanisms like Spike-Timing-Dependent Plasticity (STDP) are actually "cloaked" versions of standard optimization algorithms.
Yoshua Bengio and colleagues have shown that if you define synaptic updates as the product of presynaptic activity and the rate of change of postsynaptic activity, STDP emerges naturally as a form of stochastic gradient descent. This suggests that the brain's local timing rules are a mathematically efficient way to minimize a global objective function.
Similarly, Target Propagation (TP)—where each layer learns to invert the transformation of the next—has been shown to have deep roots in classical optimization. In Deriving Differential Target Propagation from Iterating Approximate Inverses, Bengio demonstrates that these local target-based updates implicitly perform Gauss-Newton optimization, a higher-order method that can sometimes be more robust than the first-order gradients used in BP.
Equivalence to Statistical Frameworks
To move beyond heuristics, researchers are mapping bio-inspired networks onto established statistical frameworks. Predictive Coding Networks (PCNs), which minimize the difference between top-down predictions and bottom-up sensory input, are a prime example.
Beren Millidge and Thomas Lukasiewicz provided a Theoretical Framework proving that PCNs are mathematically equivalent to the Expectation-Maximization (EM) algorithm. This alignment allows researchers to borrow convergence proofs from the world of statistics to guarantee that these biological models will actually reach an optimal state.
Even the "surprise" felt by biological organisms can be formalized. Vasiliki Liakoni and Alireza Modirshanechi derived Bayes Factor Surprise, showing that biological learning in volatile environments is an approximation of exact Bayesian inference.
Why Rigor Matters
The push for mathematical foundations is not just academic. Without rigorous proofs, bio-inspired models remain "black boxes" that often fail when scaled. For instance, Shivam Garg and Santosh S. Vempala used low-rank matrix factorization to show that while Feedback Alignment works in overparameterized settings, it fundamentally diverges from gradient descent in rank-deficient scenarios.
By establishing these theoretical bounds, researchers like Mathias Lechner can develop more robust versions, such as monotone Direct Feedback Alignment (mDFA), which comes with a mathematical guarantee of convergence for binary classification. This theoretical grounding is the necessary bridge to move bio-inspired AI from interesting simulations to high-performance, hardware-efficient reality.