Abstract
Neuromorphic devices represent an attempt to mimic aspects of the brain's architecture and dynamics with the aim of replicating its hallmark functional capabilities in terms of computational power, robust learning and energy efficiency. We employ a single-chip prototype of the BrainScaleS 2 neuromorphic system to implement a proof-of-concept demonstration of reward-modulated spike-timing-dependent plasticity in a spiking network that learns to play the Pong video game by smooth pursuit. This system combines an electronic mixed-signal substrate for emulating neuron and synapse dynamics with an embedded digital processor for on-chip learning, which in this work also serves to simulate the virtual environment and learning agent. The analog emulation of neuronal membrane dynamics enables a 1000-fold acceleration with respect to biological real-time, with the entire chip operating on a power budget of 57mW. Compared to an equivalent simulation using state-of-the-art software, the on-chip emulation is at least one order of magnitude faster and three orders of magnitude more energy-efficient. We demonstrate how on-chip learning can mitigate the effects of fixed-pattern noise, which is unavoidable in analog substrates, while making use of temporal variability for action exploration. Learning compensates imperfections of the physical substrate, as manifested in neuronal parameter variability, by adapting synaptic weights to match respective excitability of individual neurons.
Introduction
Modern artificial intelligence faces a significant sustainability challenge. While biological brains operate on approximately 20 watts, the high-performance GPUs required for training state-of-the-art models consume kilowatts. Neuromorphic engineering addresses this gap by mimicking the brain’s architecture, utilizing analog circuits to emulate the physics of neurons and synapses.
However, analog hardware is susceptible to device mismatch—manufacturing variations that cause individual silicon neurons to behave inconsistently, a phenomenon known as "fixed-pattern noise." Traditionally, this requires intensive, chip-specific calibration. This study explores whether on-chip learning can circumvent manual calibration by allowing the system to adapt to its own hardware imperfections during real-time interaction with an environment.
Methods
The researchers utilized a prototype of the BrainScaleS 2 (BSS2) system, a mixed-signal chip that emulates Leaky Integrate-and-Fire (LIF) neurons in the analog domain while using digital logic for communication and learning.
The architecture includes:
- Plasticity Processing Unit (PPU): An on-chip digital processor that executes learning rules and, in this pilot, simulates the Pong environment.
- Reward-modulated Spike-Timing-Dependent Plasticity (R-STDP): A three-factor learning rule that integrates local synaptic timing with a global reward signal based on task performance (e.g., hitting the ball).
- Hybrid Learning Loop: Analog neurons generate spikes, while the PPU reads on-chip correlation sensors to update synaptic weights.
The task involved a "smooth pursuit" Pong simulation where a 32-neuron network learned to align a paddle with a moving ball. Performance was benchmarked against the NEST simulator running on a conventional Intel i7 CPU.
Results and Discussion
The BSS2 system demonstrated significant advantages in both performance and robustness. By operating on physical time constants, the chip achieved a 1,000x acceleration relative to biological real-time.
Efficiency and Speed
The on-chip emulation was approximately 1,000x more energy-efficient than the software baseline, operating on a power budget of just 57 mW. Furthermore, it was at least 10x faster than the state-of-the-art software simulation.
Learning as Calibration
A key finding was that the R-STDP rule naturally compensated for fixed-pattern noise. The network learned to adjust synaptic weights to account for individual neuron variability (e.g., "sluggish" vs. "hyperactive" neurons). This suggests that neuromorphic hardware does not require perfect manufacturing if it possesses sufficient plasticity. Additionally, the inherent thermal noise and crosstalk in the analog circuits provided a natural source of stochasticity for action exploration in reinforcement learning, turning hardware "imperfections" into a functional resource.
Conclusion
This pilot study confirms the potential of accelerated neuromorphic hardware for energy-efficient, real-time learning. While the current prototype is limited in scale (32 neurons), it successfully demonstrates that on-chip learning can mitigate the challenges of analog device mismatch. Future work will focus on scaling the BrainScaleS 2 architecture to hundreds of thousands of synapses and implementing more complex Actor-Critic models for advanced strategic tasks.
Summary updated