Abstract
In this study, we propose and analyze in simulations a new, highly flexible method of implementing synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. Flexibility is achieved by embedding a general-purpose processor dedicated to plasticity into the wafer. To evaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spike train learning task. We find that probabilistic updates can increase the performance of low-resolution weights, a simple interface between analog synapses and processor is sufficient for learning, and performance is insensitive to mismatch. However, for highly accelerated systems, communication latency must be kept to a minimum.
Introduction
Synaptic plasticity is the cornerstone of learning in biological systems. For neuromorphic engineers, implementing this plasticity in silicon presents a significant challenge: balancing the efficiency of hard-wired circuits with the flexibility required to explore diverse learning rules. Hard-wiring specific rules like Spike-Timing Dependent Plasticity (STDP) into silicon makes chips inflexible, while offloading computation to external CPUs creates communication bottlenecks. This paper investigates a hybrid approach: embedding a small RISC processor directly into a wafer-scale neuromorphic substrate to manage synaptic updates.
System Architecture
The researchers utilized the BrainScaleS system, a wafer-scale hardware platform operating at $10^5$ times biological real-time. The core innovation is the integration of an Embedded Plasticity Processor (EPP) within the wafer building blocks.
In this architecture, analog circuits in each synapse track the timing between pre- and post-synaptic spikes to generate an "eligibility trace." The EPP, a digital RISC processor based on the PowerISA, reads these analog values, calculates weight changes based on a global reward signal, and writes the updated weights back to the synapses.
The Embedded Plasticity Processor (EPP)
The EPP is designed for area efficiency and specialized functionality. Occupying only 0.14 mm² in 65nm technology, it includes a dedicated "SYNAPSE" functional unit in its backend to facilitate high-speed interaction with the analog synaptic array.
Learning under Hardware Constraints
To evaluate the system, the authors implemented Reward-modulated STDP (R-STDP) in a spike train learning task. A single layer of neurons was trained to fire at specific target times using only a global reward signal. The study simulated several real-world hardware constraints:
- Discretization: Weights are restricted to 4-bit or 8-bit integers.
- Thresholded Readout: The processor uses a 1-bit comparison for analog trace values rather than high-precision ADCs.
- Analog Mismatch: Variations in analog components due to manufacturing imperfections.
- Latency: Delays in the reward signal arrival from the external environment.
Key Results and Insights
The simulations revealed several critical findings for neuromorphic design:
- Probabilistic Rounding: Standard rounding in 4-bit hardware significantly degrades learning (reward drops from 0.54 to 0.37). However, using probabilistic updates—where the probability of a bit flip is proportional to the calculated update—restores performance to 0.46.
- Robustness to Analog Imperfections: The system proved remarkably resilient to analog circuit mismatch. The learning algorithm naturally compensates for physical variations across synapses.
- The Latency Bottleneck: Due to the $10^5$ acceleration factor, even a 1ms real-world delay in reward delivery equates to a 100-second delay for the neurons. This causes eligibility traces to decay, highlighting the need for low-latency environments in accelerated hardware.
Conclusion
This study demonstrates that embedding general-purpose RISC processors into neuromorphic wafers provides a viable and flexible path for implementing complex learning rules. While hardware constraints like low weight resolution and communication latency pose challenges, they can be mitigated through algorithmic techniques like probabilistic rounding and careful system design. Future work will explore the scalability of this architecture to deeper networks and more complex, closed-loop reinforcement learning tasks.
Summary updated