Abstract
Spike-Timing-Dependent Plasticity (STDP) is a bio-inspired local incremental weight update rule commonly used for online learning in spike-based neuromorphic systems. In STDP, the intensity of long-term potentiation and depression in synaptic efficacy (weight) between neurons is expressed as a function of the relative timing between pre- and post-synaptic action potentials (spikes), while the polarity of change is dependent on the order (causality) of the spikes. Online STDP weight updates for causal and acausal relative spike times are activated at the onset of post- and pre-synaptic spike events, respectively, implying access to synaptic connectivity both in forward (pre-to-post) and reverse (post-to-pre) directions. Here we study the impact of different arrangements of synaptic connectivity tables on weight storage and STDP updates for large-scale neuromorphic systems. We analyze the memory efficiency for varying degrees of density in synaptic connectivity, ranging from crossbar arrays for full connectivity to pointer-based lookup for sparse connectivity. Finally, we present an alternative formulation of STDP via a delayed causal update mechanism that permits efficient weight access, requiring no more than forward connectivity lookup.
The Connectivity Bottleneck in Neuromorphic Hardware
In neuromorphic engineering, the goal is to build hardware that mimics the brain's efficiency. A cornerstone of this is STDP, a learning rule where synaptic weights change based on the relative timing of spikes from pre-synaptic and post-synaptic neurons. However, standard STDP requires bidirectional lookups:
- Acausal updates: Triggered by a pre-synaptic spike (requires "forward" access: which post-synaptic neurons do I connect to?).
- Causal updates: Triggered by a post-synaptic spike (requires "reverse" access: which pre-synaptic neurons connect to me?).
Most hardware is optimized for forward access. To support reverse access, designers often have to double their memory to store a second "reverse" connectivity table or perform exhaustive, slow searches. For large-scale sparse networks, this bidirectional requirement is a massive bottleneck for both power and area.
Methods: Delayed Causal Updates and Sparse Data Structures
The authors analyze four synaptic weight data structures: Crossbar (Dense), PB-CSR (Pointer-Based Compressed Sparse Row), PB-RLE (Run-Length Encoding), and PB-BMP (Bitmap).
The core innovation is the Delayed Causal Update. Instead of performing a causal update the moment a post-synaptic neuron spikes, the system waits until the pre-synaptic neuron’s STDP timer expires. At that moment, the system performs a forward lookup to update all its connections at once. Because the update is mathematically equivalent to the original rule but only uses forward pointers, it bypasses the need for reverse connectivity tables entirely.
Key Results and Insights
- Functional Equivalence: The delayed causal update achieved a Mean Squared Error (MSE) of zero compared to the original STDP formulation when using multiple timers to handle high firing rates.
- Memory Savings: By eliminating reverse connectivity tables, the proposed method can reduce the memory footprint of sparse synaptic connectivity by nearly 50%.
- Density Dictates Structure: Traditional crossbars only become more efficient than pointer-based sparse structures when connectivity density exceeds ~80%, which is rare in biological modeling. PB-CSR is the most efficient for forward access in sparse regimes.
Conclusion
This paper introduces a mechanism that eliminates the need for costly reverse connectivity lookups in neuromorphic hardware. By restructuring how and when synaptic weights are updated, the authors enable the use of memory-efficient compressed data structures, significantly reducing storage and access costs for large-scale, sparse neural networks. Future work includes extending this "forward-only" logic to more complex rules, such as triplet-based plasticity.
Summary updated