Ultra-High-Speed Accelerator Architecture for Convolutional Neural Network Based on Processing-in-Memory Using Resistive Random Access Memory

Hongzhe Wang, Junjie Wang, Hao Hu, Li Guo, S. G. Hu, Qi Yu, Zhen Liu, T. P. Chen, Shijie Zhou, Yang LiuSensors 2023 · DOI

TL;DR

This paper introduces a novel Processing-in-Memory (PIM) architecture using Resistive Random Access Memory (RRAM) for ultra-high-speed Convolutional Neural Network (CNN) acceleration. By eliminating Analog-to-Digital Converters (ADCs) and Digital-to-Analog Converters (DACs) and employing a partial quantization technique, the proposed architecture significantly reduces power consumption and data transportation overhead, achieving impressive speedups for image recognition tasks.

What Problem Does This Paper Address?

The increasing complexity and computational demands of modern deep learning models, particularly Convolutional Neural Networks (CNNs), necessitate efficient hardware accelerators. Traditional hardware architectures often face bottlenecks due to data movement between memory and processing units, leading to high power consumption and limited speed. Processing-in-Memory (PIM) architectures aim to mitigate this by performing computations directly within the memory array. However, existing RRAM-based PIM solutions often rely on ADCs and DACs for data conversion, which are power-hungry and occupy significant chip area, especially for high-precision applications. This paper addresses the challenge of creating a high-speed, low-power RRAM PIM architecture for CNNs that avoids these costly conversion circuits while minimizing accuracy loss.

Methods

The core innovation lies in the proposed RRAM PIM architecture, which fundamentally differs from conventional approaches by eliminating ADCs and DACs. Instead, it utilizes a digital domain computation approach within the RRAM array. Key components include:

  • Pipeline Register: Stores intermediate features from previous layers.
  • Feature Reshaping Module: Prepares input features for the RRAM array, handling padding.
  • Process Element (PE) Array: The heart of the PIM, built with RRAM arrays. It performs convolution operations directly in the digital domain using D-flip-flops and ripple counters for data conversion to pulses, avoiding analog-to-digital conversions.
  • Post-Feature Processing Module: Handles operations like biasing and activation functions (e.g., ReLU).
  • Configuration Register and SRAM Module: Manage the overall operation and store weights.

A crucial technique introduced is partial quantization. While full quantization (quantizing both weights and features) can lead to accuracy degradation, this paper proposes quantizing only the weights. This involves training weights as floating-point numbers, then quantizing them into 8-bit fixed-point integers, while using a 32-bit floating-point scale factor and 8-bit fixed-point zero-point parameters. This approach aims to leverage RRAM's inherent characteristics while preserving accuracy.

Datasets

The YOLOv3 algorithm was used to build a system model for simulation and evaluation. For testing the overall accuracy and robustness of the proposed architecture, 5000 images from the COCO dataset were used for identification.

Key Insights and Ideas

  • Eliminating ADCs/DACs is Key: The most significant insight is that by performing computations in the digital domain within the RRAM array, the need for power-hungry and area-intensive ADCs and DACs can be completely removed, leading to substantial power and area savings.
  • Partial Quantization Strikes a Balance: Quantizing only the weights, rather than both weights and features, proves to be an effective strategy. It significantly reduces RRAM storage requirements without a substantial drop in accuracy, a common pitfall of full quantization.
  • RRAM's Digital Prowess: The paper demonstrates that RRAM, often associated with analog computation, can be effectively utilized for digital-domain PIM operations, opening new avenues for its application.
  • Speed-Accuracy Trade-off Management: The proposed partial quantization method shows that it's possible to achieve high-speed processing with minimal accuracy loss, a critical factor for real-world deployment of AI accelerators.

Key Results

The proposed RRAM PIM architecture achieved remarkable results:

  • Ultra-High Speed: An image recognition speed of 284 frames per second (FPS) at 50 MHz was achieved for the YOLOv3 algorithm. This is approximately 7 times faster than a high-end GPU (NVIDIA RTX 3080) and significantly faster than other RRAM PIM architectures with ADCs.
  • Accuracy Preservation: The partial quantization technique (8-bit and 16-bit precision) resulted in accuracies of 53.588% and 53.386%, respectively. These accuracies remained almost unchanged compared to the algorithm without quantization (54.135% mAP). In contrast, full quantization led to a significant drop in accuracy (50.319% mAP).
  • Reduced Resource Consumption: By eliminating ADCs and DACs, the proposed architecture significantly reduces power consumption and chip area compared to PIM designs that utilize these components.

Limitations and Future Work

  • Idealized RRAM Model: The simulations utilized an idealized RRAM model. Real-world RRAM devices exhibit variations and non-idealities that could impact performance and accuracy.
  • Simulation-Based: The results are based on simulations, and experimental validation on a fabricated chip would be necessary to confirm the performance and efficiency gains.
  • Specific Algorithm Focus: While YOLOv3 is a representative CNN, further evaluation on a wider range of CNN architectures and tasks would be beneficial.

The authors suggest that future work could focus on experimental validation and exploring the application of this architecture to other complex deep learning models. The potential for further optimization through more advanced quantization schemes or RRAM device engineering also remains an open area.

Summary updated

Abstract

Processing-in-Memory (PIM) based on Resistive Random Access Memory (RRAM) is an emerging acceleration architecture for artificial neural networks. This paper proposes an RRAM PIM accelerator architecture that does not use Analog-to-Digital Converters (ADCs) and Digital-to-Analog Converters (DACs). Additionally, no additional memory usage is required to avoid the need for a large amount of data transportation in convolution computation. Partial quantization is introduced to reduce the accuracy loss. The proposed architecture can substantially reduce the overall power consumption and accelerate computation. The simulation results show that the image recognition rate for the Convolutional Neural Network (CNN) algorithm can reach 284 frames per second at 50 MHz using this architecture. The accuracy of the partial quantization remains almost unchanged compared to the algorithm without quantization.
BibTeX
@inproceedings{wang2023ultra,
  author = {Hongzhe Wang and Junjie Wang and Hao Hu and Li Guo and S. G. Hu and Qi Yu and Zhen Liu and T. P. Chen and Shijie Zhou and Yang Liu},
  title = {{Ultra-High-Speed Accelerator Architecture for Convolutional Neural Network Based on Processing-in-Memory Using Resistive Random Access Memory}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23052401},
}
Figure