Energy Efficient RRAM Spiking Neural Network for Real Time Classification

Yu Wang, Tianqi Tang, Lixue Xia, Boxun Li, Peng Gu, Huazhong Yang, Hai Li, Yuan XieACM Great Lakes Symposium on VLSI 2015 · DOI

TL;DR

This paper presents an energy-efficient spiking neural network (SNN) implemented using RRAM (Resistive Random-Access Memory) for real-time classification. By leveraging RRAM's non-volatility and in-memory computation capabilities, the authors demonstrate a system that significantly reduces power consumption compared to traditional CMOS-based approaches, while achieving competitive accuracy on tasks like MNIST digit recognition.

What Problem Does This Paper Address?

The paper tackles two major hurdles in realizing the full potential of Spiking Neural Networks (SNNs): energy-efficient hardware implementation and effective training for real-world applications. Traditional CMOS-based computing faces limitations in power efficiency and the "memory wall" problem due to data movement. SNNs, inspired by the brain's efficiency, offer a promising alternative by encoding information in sparse, time-based spikes. However, implementing SNNs efficiently in hardware and training them to achieve high accuracy for complex tasks like real-time classification remains a significant challenge.

Methods

The authors propose an RRAM-based SNN system that combines analog spiking neuron circuits with RRAM crossbar arrays for synapse implementation. This architecture leverages RRAM's ability to perform matrix-vector multiplication directly within the memory, drastically reducing data movement and power consumption.

Two offline training algorithms for SNNs are explored:

  • Spike Timing Dependent Plasticity (STDP): An unsupervised learning method primarily used for feature extraction.
  • Neural Sampling Method: A supervised method that involves training an equivalent Artificial Neural Network (ANN) and then transferring its weights to an SNN. This method is also referred to as "Transferring ANN to SNN."

The system also includes an input encoding module to convert input data into spike trains and a mapping algorithm to configure the RRAM crossbar.

Datasets

  • MNIST: Used for evaluating the real-time classification performance of the proposed RRAM-based SNN system.

Key Insights and Ideas

  • RRAM as a Game-Changer for SNNs: RRAM's non-volatility and in-memory computation capabilities are crucial for overcoming the power and memory bandwidth limitations of CMOS-based SNN implementations. The ability to perform matrix-vector multiplication directly in the RRAM crossbar is a key enabler of energy efficiency.
  • SNNs Offer Significant Power Savings: Even when compared to ANN systems implemented on RRAM, SNNs demonstrate substantially lower power consumption. This is attributed to their sparse, event-driven nature and the ability to use lower voltage supplies due to binary coding.
  • Trade-offs in Training Algorithms: While STDP is useful for feature extraction, it doesn't achieve competitive accuracy for classification tasks on its own. The Neural Sampling method, which transfers ANN knowledge, yields better classification performance, though it may still lag slightly behind pure ANN performance.
  • Robustness through Sparsity: The inherent sparsity of spike trains in SNNs contributes to their robustness against input signal fluctuations and variations in RRAM device characteristics.
  • Potential for Boosting SNN Accuracy: The paper suggests that the lower power and area overhead of SNNs compared to ANNs opens up possibilities for integrating multiple SNNs and using ensemble methods (like boosting) to further enhance classification accuracy and robustness.

Key Results

  • MNIST Recognition Accuracy:
    • The "784x100SNN + 100x50x10ANN" system achieved 91.5% accuracy on CPU and 90% on the RRAM-based crossbar model.
  • A three-layer ANN ("784x100x10") achieved 94.3% on CPU and 92% on the RRAM-based crossbar model.
  • A "784x50 STDP + 50x10 SVM" system achieved 90% accuracy, which was lower than a baseline PCA-based system (94%).
  • Power Efficiency: The RRAM-based SNN system ("784x100 SNN + 100x50x10 ANN") consumed 327.36mW, significantly less than a comparable ANN system ("784x100x10 ANN") which consumed 2273.60mW.
  • RRAM Device Impact: The paper shows that 8-bit RRAM devices can achieve nearly 90% recognition accuracy. Input signals above 6-bit level also yield satisfactory accuracy (>85%).
  • Robustness: The system demonstrated robustness to signal fluctuation (around 3% accuracy drop with 20% variation) and RRAM process variation (minimal decrease with 20% variation).
  • Real-time Performance: The system can complete recognition tasks in approximately 1µs/sample, despite the conversion to spike trains.

Limitations and Future Work

  • STDP Performance: STDP alone did not achieve competitive accuracy for MNIST digit recognition, highlighting the need for supervised methods or hybrid approaches.
  • Accuracy Gap: While SNNs offer power advantages, their classification accuracy can be slightly lower than equivalent ANN systems.
  • Future Work:
    • Scalable SNN Processor Design: Developing a core structure for large-scale SNN "processors" based on RRAM crossbars, moving beyond fixed accelerators.
  • On-line Learning SNN Systems: Investigating how to leverage RRAM's inherent learning features for building on-line learning systems that can adapt and learn autonomously.
  • Boosting SNN Accuracy: Further exploring and developing boosting algorithms specifically tailored for SNNs to improve accuracy and robustness, considering the unique error types in SNNs.

Summary updated

BibTeX
@inproceedings{wang2015energy,
  author = {Yu Wang and Tianqi Tang and Lixue Xia and Boxun Li and Peng Gu and Huazhong Yang and Hai Li and Yuan Xie},
  title = {{Energy Efficient RRAM Spiking Neural Network for Real Time Classification}},
  booktitle = {ACM Great Lakes Symposium on VLSI},
  year = {2015},
  doi = {10.1145/2742060.2743756},
}
Figure