Abstract
In this study, we propose and analyze in simulations a new, highly flexible method of implementing synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. The study focuses on globally modulated STDP, as a special use-case of this method. Flexibility is achieved by embedding a general-purpose processor dedicated to plasticity into the wafer. To evaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spike train learning task. A single layer of neurons is trained to fire at specific points in time with only the reward as feedback. This model is simulated to measure its performance, i.e., the increase in received reward after learning. Using this performance as baseline, we then simulate the model with various constraints imposed by the proposed implementation and compare the performance. The simulated constraints include discretized synaptic weights, a restricted interface between analog synapses and embedded processor, and mismatch of analog circuits. We find that probabilistic updates can increase the performance of low-resolution weights, a simple interface between analog synapses and processor is sufficient for learning, and performance is insensitive to mismatch. Further, we consider communication latency between wafer and the conventional control computer system that is simulating the environment. This latency increases the delay, with which the reward is sent to the embedded processor. Because of the time continuous operation of the analog synapses, delay can cause a deviation of the updates as compared to the not delayed situation. We find that for highly accelerated systems latency has to be kept to a minimum. This study demonstrates the suitability of the proposed implementation to emulate the selected reward modulated STDP learning rule. It is therefore an ideal candidate for implementation in an upgraded version of the wafer-scale system developed within the BrainScaleS project.
Introduction
Neuromorphic hardware aims to achieve the brain's computational efficiency by using analog circuits to emulate neural and synaptic dynamics. However, these systems often face a critical trade-off between flexibility and efficiency. Most current implementations are "fixed-function," with learning rules like Spike-Timing-Dependent Plasticity (STDP) hard-wired into the silicon. This rigidity makes it difficult to implement more complex, globally modulated algorithms such as reinforcement learning.
To bridge this gap, this research proposes a hybrid architecture that integrates a programmable Embedded Plasticity Processor (EPP)—a general-purpose RISC core—directly into a wafer-scale analog substrate. This approach maintains the high-speed operation of analog synapses while providing the flexibility needed to handle diverse learning logic.
System Architecture
The proposed system extends the existing BrainScaleS wafer-scale framework. By embedding the EPP and dedicated memory into each wafer building block, the system can locally manage synaptic updates without relying on a bottlenecked external controller.
The EPP is designed as a standard RISC pipeline augmented with a specialized synapse interface. This interface allows the processor to read analog synaptic states and write back updated digital weights, effectively acting as the bridge between the flexible digital logic and the high-speed analog environment.
Methodology and Hardware Constraints
To evaluate the system, the researchers conducted extensive simulations using the Brian simulator, focusing on a reward-modulated STDP (R-STDP) rule. The task required a single layer of neurons to learn specific firing times based on a global reward signal. The study specifically addressed the "reality gap" by modeling several hardware constraints:
- Weight Discretization: Reducing synaptic weights from 64-bit floating-point to 8, 6, and 4-bit resolutions.
- Probabilistic Updates: Implementing stochastic rounding to prevent small weight changes from being lost in low-resolution systems.
- Restricted Interface: Simulating a simplified, low-power thresholded readout between analog synapses and the EPP.
- Analog Imperfections: Modeling manufacturing variations (mismatch) and capacitor leakage (drift) in the analog components.
- Communication Latency: Accounting for the delay in reward signals returning from an external environment to the accelerated hardware.
Key Results
Resilience to Low Resolution
While 8-bit and 6-bit resolutions performed nearly as well as the continuous baseline (Reward $\approx 0.54$), 4-bit deterministic updates saw performance drop significantly to 0.37. However, by employing probabilistic updates, the system recovered to a reward of 0.46, demonstrating that stochasticity can effectively compensate for limited bit-depth.
Interface and Analog Robustness
Surprisingly, a simple thresholded readout (a 1-bit ADC equivalent) did not hinder learning and, in some cases, improved performance to 0.59 by filtering extreme values. Furthermore, the system proved highly robust to analog mismatch; even with time constant variations of up to 150%, learning remained stable.
The Latency Challenge
Because the hardware is accelerated by a factor of $10^4$ to $10^5$ relative to biological time, communication latency is a critical factor. The study found that for a system accelerated by $10^5$, the round-trip latency must remain under 20 microseconds. Beyond this, the "eligibility trace" of neural activity decays before the reward signal arrives, causing learning to fail.
Conclusion
This study demonstrates that embedding a general-purpose RISC processor into a neuromorphic wafer is a viable and flexible solution for implementing complex learning rules. Despite the inherent constraints of analog hardware and low-resolution digital weights, techniques like probabilistic rounding and simplified interfaces allow these systems to achieve high performance. These findings provide a roadmap for the next generation of the BrainScaleS wafer-scale system, enabling more sophisticated, reward-based learning in accelerated hardware.
Summary updated