In the quest to build energy-efficient, brain-like artificial intelligence, Spiking Neural Networks (SNNs) and biologically plausible learning rules offer a compelling alternative to standard deep learning. On simple benchmarks like MNIST, shallow networks utilizing local learning rules can achieve remarkable success. For instance, Wulfram Gerstner et al. showed in Biologically plausible deep learning - but how far can we go with shallow networks? that a single hidden layer can achieve over 98% accuracy, raising questions about whether simple datasets truly require deep architectures.
However, shallow networks quickly hit a representational ceiling when confronted with the high-variation, high-dimensional complexity of natural images (Activity-dependent bidirectional plasticity and homeostasis regulation governing structure formation in a model of layered visual memory; Discriminative Ferns Ensemble for Hand Pose Recognition). Insights from computational neuroscience demonstrate that biological vision relies on deep, hierarchical processing to achieve object invariance. As shown by Charles F. Cadieu et al. in Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition, replicating the complex transformations of the primate inferior temporal cortex requires deep convolutional hierarchies. Without depth, unsupervised models remain restricted to early sensory pathways and fail to map higher-level cortical regions responsible for semantic object recognition (Unsupervised Feature Learning Improves Prediction of Human Brain Activity in Response to Natural Images; Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition).
The Spatial Credit Assignment Problem
If depth is essential, why not simply stack local learning rules like Spike-Timing-Dependent Plasticity (STDP)? The fundamental obstacle is the spatial credit assignment problem.
In a single-layer network, learning is entirely local: a synapse modifies its weight based on the immediate timing of its pre- and post-synaptic neurons (Constraints on Hebbian and STDP learned weights of a spiking neuron). However, in a multi-layer hierarchy, a global outcome (such as a reward or an error signal) is evaluated at the final output layer. A hidden neuron deep within the network has no direct access to this global signal, making it incredibly difficult to determine how its individual activity contributed to the final outcome.
Consider reward-modulated STDP (R-STDP), where a global neuromodulator like dopamine scales local synaptic updates. As Robert Legenstein et al. demonstrated in Theoretical Analysis of Learning with Reward-Modulated Spike-Timing-Dependent Plasticity, R-STDP effectively solves credit assignment in single-layer tasks. But when applied to deep hierarchies, a uniform global reward fails to provide specific feedback to intermediate layers, limiting the network's capacity to learn complex hierarchical features (First-Spike-Based Visual Categorization Using Reward-Modulated STDP; BioLCNet: Reward-modulated Locally Connected Spiking Neural Networks).
Similarly, highly efficient event-driven rules (Fast and Accurate Classification with a Multi-Spike Learning Algorithm for Spiking Neurons; Toward Efficient Processing and Learning With Spikes: New Approaches for Multispike Learning) and structural plasticity mechanisms (Learning Spike time codes through Morphological Learning with Binary Synapses; Jointly Learning Network Connections and Link Weights in Spiking Neural Networks) face a coordination breakdown when stacked. Without a mechanism to propagate credit through successive layers, local updates cannot self-organize into a coherent global representation.
Mechanistic Solutions to Hierarchical Coordination
To bridge this gap, researchers are developing biologically plausible mechanisms that mimic the credit-routing capabilities of backpropagation without violating biological constraints.
One major constraint is the weight transport problem: backpropagation requires the backward error-propagating weights to be identical to the forward feature-extracting weights, which real neurons cannot do. To circumvent this, N. Ahmad et al. proposed a solution in Overcoming the Weight Transport Problem via Spike-Timing-Dependent Weight Inference. Their framework, Spike-Timing-Dependent Weight Inference (STDWI), allows synapses to mathematically infer their own weights using only local spike timings, enabling deep coordination without symmetric weight transport.
Another challenge is the temporal separation of forward and backward passes. Standard deep learning alternates between a forward inference phase and a backward learning phase. In contrast, the brain operates continuously. In Single-phase deep learning in cortico-cortical networks, Will Greedy et al. introduced Bursting Cortico-Cortical Networks (BurstCCN). By leveraging multi-compartment neurons, short-term plasticity, and interneurons, BurstCCN multiplexes forward and backward signals simultaneously within a single phase, allowing deep networks to approximate backpropagation gradients in real time.
Furthermore, online learning frameworks are exploring target propagation to route feedback. Lukas Braun et al. showed in Online Learning Of Neural Computations From Sparse Temporal Feedback that local parameter tuning (such as Event-Dependent Scaling) can be paired with hierarchical credit assignment strategies to recover complex temporal computations across deep architectures.
Architectural Constraints as Inductive Priors
Beyond designing smarter learning rules, researchers also leverage structural constraints to ease the burden of multi-layer credit assignment.
Instead of fully connected layers—where the credit assignment problem scales quadratically with network depth—imposing local connectivity restricts the scope of competition. In Locally Connected Spiking Neural Networks for Unsupervised Feature Learning, Daniel J. Saunders et al. demonstrated that restricting competition to localized patches allows unsupervised STDP to converge rapidly and robustly. Stacking these locally connected or convolutional layers (STDP Learning of Image Patches with Convolutional Spiking Neural Networks) provides a structural inductive prior that helps maintain stable learning across a hierarchy.
Alternatively, hybrid architectures can sidestep the multi-layer scaling problem entirely by combining traditional deep learning with spiking layers. For example, the CSNN framework introduced by Qi Xu et al. in CSNN: An Augmented Spiking based Framework with Perceptron-Inception uses a hybrid Perceptron-Inception architecture to handle complex spatial feature extraction, mitigating the depth scalability limitations of purely local spiking networks.
Key Foundational Work
A robust biologically plausible implementation of ICA-like learning (2009) — contributed 1 concept elements to this direction.
Theoretical Analysis of Learning with Reward-Modulated Spike-Timing-Dependent Plasticity (2007) — contributed 1 concept elements to this direction.
Activity-dependent bidirectional plasticity and homeostasis regulation governing structure formation in a model of layered visual memory (2009) — contributed 1 concept elements to this direction.