Modern artificial intelligence is built on backpropagation, an algorithm that requires "weight transport." For a network to learn, the feedback weights used to propagate error signals must be the exact mathematical transpose of the feedforward weights used for processing. In the biological brain, however, synapses are unidirectional; there is no known mechanism for one neuron to "copy" the synaptic weights of another distant neuron. This creates a fundamental tension: how can the brain achieve the efficiency of gradient-based learning without the physically impossible requirement of symmetric weight transport?
A growing body of research suggests that the solution is not to hard-code symmetry, but to let it emerge. By using local learning rules, feedforward and feedback pathways can autonomously align their weights over time, allowing the network to "discover" the gradients it needs to learn.
Symmetry through Shared Local Updates
The most intuitive mechanism for achieving symmetry is the principle of co-adaptation. If two independent synapses—one forward and one backward—are subjected to the same local update signals and the same rate of weight decay, they will eventually converge to the same value, regardless of where they started.
Mohamed Akrout and colleagues in Deep Learning without Weight Transport demonstrate that this simple mathematical property allows deep networks to solve the weight transport problem. By applying identical local updates to both pathways, the feedback weights naturally evolve to mirror the feedforward ones. This concept traces back to the "Recirculation" algorithm introduced by Geoffrey E. Hinton and James L. McClelland, who showed that in closed-loop architectures, weights can naturally converge toward symmetry without explicit weight sharing.
More recently, Roy Abel and Shimon Ullman have extended this to "Counter-Hebbian" models, where a single top-down stream provides both attentional guidance and the mathematical equivalent of backpropagation signals. Their work in Biologically Inspired Learning Model for Instructed Vision shows that weight symmetry is a dynamic, self-correcting property of the system rather than a static architectural requirement.
Feedback as Causal Inference
If the brain cannot copy weights, how does a feedback neuron know what its "partner" forward neuron is doing? One perspective is that neurons treat their own activity as a natural experiment.
In Spike-based causal inference for weight alignment, Jordan Guerguiev and Blake A. Richards propose that neurons use spikes to estimate their own causal influence on the network's output. By using techniques like Regression Discontinuity Design—borrowed from econometrics—feedback synapses can autonomously adjust their strengths to match feedforward synapses based on these local causal signals.
Similarly, Benjamin Lansdell and Prashanth Ravi Prakash suggest a hybrid approach where local, reinforcement-learning-style perturbations act as a supervisory signal. In their framework, Learning to solve the credit assignment problem, the feedback system learns to approximate the true gradient by co-adapting to these local stochastic estimates.
Functional Alignment and the Pseudoinverse
Symmetry is not just about making two numbers equal; it is about ensuring the feedback path accurately represents the "inverse" or "transpose" of the forward mapping.
Davide Casnici and colleagues show in Accelerated Predictive Coding Networks via Direct Kolen-Pollack Feedback Alignment that adaptive learning rules can drive feedback weights to converge toward the Moore-Penrose pseudoinverse of forward weights. This ensures that the error signals sent backward are as informative as possible, even if the weights aren't perfectly symmetric in a literal sense.
This functional alignment is also central to the work of Ravi Francesco Srinivasan and Francesca Mignacco. In Forward Learning with Top-Down Feedback, they characterize a class of "forward-only" algorithms where weights align during training to approximate standard gradient descent. This suggests that the "backward pass" of backpropagation might actually be happening through existing forward connections or recurrent loops, as explored by Adam A. Kohan and Hava T. Siegelmann in their "Error Forward-Propagation" framework.
Self-Organizing Dynamics
The overarching theme across these papers is that weight symmetry is an emergent phenomenon driven by the dynamics of the network itself. Benjamin Scellier and Yoshua Bengio demonstrate in Equilibrium Propagation that even in networks with general, asymmetric dynamics, feedforward and feedback connections can naturally align to optimize a global objective.
This self-organization is further refined by Yukun Yang and Peng Li in their BioLeaF framework. They show that local plasticity rules, specifically those that favor strengthening weak synapses and weakening strong ones, create a natural predisposition toward symmetry.
By moving from "hard-coded" symmetry to "learned" alignment, this research direction provides a bridge between the high-performance optimization of AI and the local, messy, but incredibly efficient reality of biological neural circuits. As Tahereh Toosi and Elias B. Issa argue in Feedback-Feedforward Alignment, this mutualistic interaction allows pathways to serve as each other's teachers, leading to robust visual inference and credit assignment without the need for global controllers.