Abstract
Neuromorphic computing is emerging to be a disruptive computational paradigm that attempts to emulate various facets of the underlying structure and functionalities of the brain in the algorithm and hardware design of next-generation machine learning platforms. This work goes beyond the focus of current neuromorphic computing architectures on computational models for neuron and synapse to examine other computational units of the biological brain that might contribute to cognition and especially self-repair. We draw inspiration and insights from computational neuroscience regarding functionalities of glial cells and explore their role in the fault-tolerant capacity of Spiking Neural Networks (SNNs) trained in an unsupervised fashion using Spike-Timing Dependent Plasticity (STDP). We characterize the degree of self-repair that can be enabled in such networks with varying degree of faults ranging from 50% - 90% and evaluate our proposal on the MNIST and Fashion-MNIST datasets.
TL;DR
Neuromorphic hardware is highly energy-efficient but notoriously prone to physical faults. Drawing inspiration from the biological brain, this paper introduces an Astrocyte-Augmented Spike-Timing Dependent Plasticity (A-STDP) learning rule that mimics the self-repair functions of glial cells, allowing Spiking Neural Networks (SNNs) to autonomously recover from severe synaptic failures without forgetting their originally learned representations.
What Problem Does This Paper Address?
Next-generation neuromorphic computing platforms, especially those based on emerging post-CMOS technologies like memristors or spintronics, suffer from significant hardware non-idealities. A common failure mode is synapses getting "stuck at zero," which causes artificial neural network accuracy to plummet.
In the biological brain, neurons and synapses are supported by glial cells—specifically astrocytes—which make up a massive portion of the brain and act as a communication and sensing channel. When biological synapses fail, astrocytes use retrograde signaling to increase the transmission probability of neighboring healthy synapses, effectively repairing the network's firing rate.
Standard artificial SNNs and traditional STDP (Spike-Timing Dependent Plasticity) learning rules completely ignore this astrocyte functionality. The authors tackle the challenge of translating this complex biological self-repair mechanism into a computationally efficient, macro-modeled learning rule that can autonomously heal faulty SNNs at scale.
Methods
The authors train a single-layer Leaky Integrate and Fire (LIF) SNN using unsupervised trace-based STDP, lateral inhibition, and homeostasis. Once the network is trained, they simulate hardware damage by permanently setting a random subset of the learned synaptic weights to zero (ranging from 50% to 90% of the connections).
To repair the network, they introduce Astrocyte Augmented STDP (A-STDP). Detailed computational models show that when a neuron's firing rate drops due to faulty synapses, astrocytes release chemicals like glutamate to globally boost the transmission probability of remaining healthy synapses. Simulating these complex chemical pathways is too computationally expensive for AI hardware, so the authors developed a macro-model.
Instead of standard retraining, which often causes the network to "forget" original representations, A-STDP uses a surrogate signal: the $\alpha$-th percentile of the network's weight distribution ($w_\alpha$, typically the 98th percentile). The A-STDP rule aggressively increases the learning rate for the healthiest surviving synapses (where $w > w_\alpha$) while keeping the learning rate low for weaker connections. As the network heals and $w_\alpha$ saturates, the enhanced learning process naturally tapers off.
Key Insights and Ideas
- Standard STDP is destructive during repair: Simply turning standard STDP back on to retrain a damaged network causes neurons to change their class associations, leading to distorted receptive fields.
- Healthy synapses must carry the load: Biological simulations revealed that synapses with a higher initial transmission probability play a much larger role in restoring a damaged neuron's firing rate than weaker synapses.
- Percentile-based surrogate signals work: By using the 98th percentile of the network's weights as a dynamic threshold, the network can autonomously gauge the severity of the fault and scale the learning window accordingly, without needing to know the "ideal" firing rate of every individual neuron.
Key Results
The A-STDP rule consistently outperformed standard STDP retraining across all fault levels (50% to 90% synaptic deletion).
- MNIST (225 neurons, 80% faults): Baseline accuracy of 89.53% crashed to 30.42% after fault injection. Standard STDP retraining recovered accuracy to 67.42%, while A-STDP reached 75.42%.
- MNIST (Extreme 90% faults): A-STDP achieved an 8.89% accuracy gain over standard STDP retraining.
- Fashion-MNIST (400 neurons, 80% faults): Baseline accuracy dropped from 77.35% to 10.00%. A-STDP successfully repaired the network to 70.82% accuracy, compared to 68.80% for standard STDP.
- Stability: The standard deviation of the final recovered accuracy across multiple runs was significantly smaller for A-STDP, proving the method is highly stable regardless of where random faults occur.
Limitations and Future Work
The current work is a proof-of-concept limited to unsupervised STDP learning. Future research should extend astrocyte feedback to supervised, gradient-descent-based training and evaluate the approach on native neuromorphic datasets, such as event-based camera data. Additionally, the authors suggest exploring how neuroglia might assist in other complex cognitive tasks like temporal binding.
Summary updated