Abstract
Neuromorphic computing is emerging to be a disruptive computational paradigm that attempts to emulate various facets of the underlying structure and functionalities of the brain in the algorithm and hardware design of next-generation machine learning platforms. This work goes beyond the focus of current neuromorphic computing architectures on computational models for neuron and synapse to examine other computational units of the biological brain that might contribute to cognition and especially self-repair. We draw inspiration and insights from computational neuroscience regarding functionalities of glial cells and explore their role in the fault-tolerant capacity of Spiking Neural Networks (SNNs) trained in an unsupervised fashion using Spike-Timing Dependent Plasticity (STDP). We characterize the degree of self-repair that can be enabled in such networks with varying degree of faults ranging from 50 to 90% and evaluate our proposal on the MNIST and Fashion-MNIST datasets.
Introduction
Neuromorphic computing aims to achieve the brain's remarkable efficiency by utilizing specialized hardware. However, hardware based on emerging technologies—such as spintronics or memristors—is highly susceptible to manufacturing defects and operational "non-idealities." A prevalent issue is the "stuck-at-zero" fault, where a synaptic connection effectively disappears from the network. While Spiking Neural Networks (SNNs) possess inherent redundancy, high fault rates typically lead to catastrophic performance degradation. Current self-repair strategies often rely on external labels or complex global controllers, which are impractical for autonomous, large-scale unsupervised SNNs.
Methods: Astrocyte-Augmented STDP (A-STDP)
This research draws inspiration from astrocytes, star-shaped glial cells that play a critical role in brain homeostasis and synaptic modulation. In biological systems, astrocytes monitor neuronal firing rates; when a drop is detected (signaling potential damage), they trigger feedback loops to enhance the transmission probability of remaining healthy synapses.
The researchers implemented this biological mechanism through a simplified mathematical rule called Astrocyte-Augmented STDP (A-STDP). In a standard single-layer SNN using Leaky Integrate-and-Fire (LIF) neurons, A-STDP modifies the learning rate dynamically. Instead of a constant rate, updates are scaled based on the current weight's strength relative to a network-wide "health monitor"—the 98th percentile of the weight distribution ($w_\alpha$). When faults occur and $w_\alpha$ drops, the learning rule becomes more aggressive, "throttling" back only once the network stabilizes and $w_\alpha$ recovers.
Key Results and Discussion
The A-STDP mechanism was evaluated on MNIST and Fashion-MNIST datasets with fault rates between 50% and 90%. The results demonstrate significant resilience compared to standard retraining methods.
- Massive Recovery: At a 90% fault rate on MNIST, where accuracy typically collapses to near-random (~10%), A-STDP restored performance to 65-68%.
- Superiority over Standard STDP: A-STDP consistently outperformed standard STDP retraining by 1% to 9%, with the performance gap widening as the fault density increased.
- Feature Preservation: Unlike standard retraining, which often distorts learned representations, A-STDP maintains clear "receptive fields," ensuring neurons retain their original classification identities (e.g., a neuron trained to recognize the digit '4' remains a '4' neuron).
Conclusion
By mimicking the "system administrator" role of astrocytes, A-STDP provides a robust, autonomous self-repair mechanism for unsupervised SNNs. While this work focuses on a macro-model of astrocytic behavior, future research could explore more complex biological functions like temporal binding or hardware-level CMOS/spintronic implementations to realize truly autonomous, fault-tolerant neuromorphic chips.
Summary updated