A Time- and Energy-Efficient CNN with Dense Connections on Memristor-Based Chips

Wenyong Zhou, Yuan Ren, Jiajun Zhou, Tianshu Hou, Ngai WongASICON 2023 · DOI

TL;DR

This paper proposes a novel RRAM-friendly Convolutional Neural Network (CNN) architecture that significantly reduces time and energy consumption for inference on memristor-based chips. By intelligently concatenating feature maps from earlier layers instead of relying on inefficient depthwise convolutions or the channel-expanding nature of DenseNets, the authors achieve competitive accuracy with substantial hardware efficiency gains.

What Problem Does This Paper Address?

Deploying deep learning models, particularly Convolutional Neural Networks (CNNs), on edge devices is a major goal for AI. However, traditional computing architectures (von Neumann) suffer from significant energy and time overheads due to data movement between memory and processing units. Compute-in-Memory (CIM) architectures, which perform computations directly where data is stored, offer a promising solution. Resistive Random-Access Memory (RRAM) is a leading candidate for CIM due to its reliability and multi-bit storage capabilities.

The challenge lies in efficiently mapping CNN operations onto RRAM crossbar arrays. While lightweight CNN designs like MobileNet and ShuffleNet reduce model size and FLOPs, they often employ depthwise convolutions. These convolutions are inefficient on RRAM crossbars because they require padding and lead to under-utilization of the crossbar's dense weight-to-cell mapping. Even architectures like DenseNet, known for their parameter efficiency and high accuracy, face similar issues. DenseNet's core mechanism of concatenating feature maps from all preceding layers leads to a linear increase in channel dimensions. This expansion necessitates larger RRAM crossbars or results in significant under-utilization of existing ones, leading to wasted time and energy. This paper aims to bridge this gap by designing a CNN architecture that is both RRAM-friendly and computationally efficient.

Methods

The authors first analyze the inefficiencies of existing CNN architectures when mapped to RRAM crossbars. They highlight two key issues:

  1. Inefficiency of Depthwise Convolutions: These convolutions, while good for reducing FLOPs on GPUs, are not well-suited for RRAM crossbars. The mapping requires padding input channels with zeros, leading to wasted chip area and computation.
  2. Inefficiency of DenseNet: DenseNet's dense connections, while beneficial for accuracy, cause a linear increase in the number of channels across layers. This leads to low crossbar utilization because the kernel size needs to expand to accommodate the growing channel dimension, leaving many RRAM cells unused.

Based on these analyses, the paper proposes a novel RRAM-friendly CNN module. Instead of the standard DenseNet concatenation, their module concatenates feature maps from the previous three layers to form the input for the last layer within each stage. This approach maintains a fixed channel length throughout the module, simplifying the mapping to RRAM crossbars and ensuring high utilization. They also employ a simplified circuit design by selecting specific portions (quarters or halves) of feature maps from earlier layers to be concatenated, based on an ablation study showing negligible impact on accuracy. This strategy reduces the need for additional 1x1 convolutions for channel dimension reduction, further saving time and energy.

Datasets

The proposed model is trained and evaluated on the following datasets:

  • CIFAR-10: Used for evaluating latency, energy consumption, and accuracy across different model sizes and architectures.
  • ImageNet: Used for comparing the performance of traditional CNN models and the proposed model.
  • CIFAR-100: Used for evaluating the accuracy of quantized models.

Key Insights and Ideas

  • RRAM Crossbar Mapping is Crucial: The efficiency of a CNN on RRAM chips is heavily dependent on how well its operations map to the RRAM crossbar structure, not just its theoretical FLOP count or parameter size.
  • Depthwise Convolutions are Problematic for RRAM: The padding requirements for depthwise convolutions lead to significant under-utilization of RRAM crossbars.
  • DenseNet's Channel Growth is a Bottleneck: While DenseNet's dense connections are good for accuracy, the resulting linear increase in channel dimensions is a major source of inefficiency on RRAM due to poor crossbar utilization.
  • Selective Concatenation is Key: By intelligently concatenating feature maps from a limited set of preceding layers (e.g., the last three) and maintaining a fixed channel length, significant improvements in RRAM crossbar utilization, time, and energy efficiency can be achieved.
  • Simplified Circuit Design Benefits: Reducing the complexity of feature map concatenation (e.g., using only portions of feature maps) can further enhance efficiency without sacrificing accuracy.

Key Results

  • The proposed RRAM-friendly CNN architecture achieves lower time and energy consumption compared to standard ResNet and DenseNet models on CIFAR-10.
  • The model maintains competitive accuracy on CIFAR-10 and ImageNet datasets, comparable to or exceeding that of ResNet and DenseNet.
  • For CIFAR-10, the proposed model shows significant improvements in latency and energy consumption, especially at smaller model widths, while achieving similar accuracy.
  • The paper demonstrates that residual connections in ResNet contribute approximately 10% of the latency and energy consumption, which their proposed architecture avoids.
  • Ablation studies confirm that the proposed concatenation strategy and selection of feature map portions yield optimal results.

Limitations and Future Work

  • Limitations: The paper focuses on the efficiency of CNNs on RRAM crossbars. While it shows competitive accuracy, further exploration into achieving state-of-the-art accuracy with this architecture might be needed. The specific RRAM technology and its characteristics are assumed, and performance might vary with different RRAM implementations.
  • Future Work: The authors suggest that their proposed RRAM-friendly architecture, which inherits the merits of dense connections while balancing time and efficiency, is a promising direction for future research in efficient CNN design for edge AI applications on memristor-based hardware. Further optimization of the concatenation strategy and exploration of different RRAM configurations could be potential avenues.

Summary updated

BibTeX
@inproceedings{zhou2023time,
  author = {Wenyong Zhou and Yuan Ren and Jiajun Zhou and Tianshu Hou and Ngai Wong},
  title = {{A Time- and Energy-Efficient CNN with Dense Connections on Memristor-Based Chips}},
  booktitle = {ASICON},
  year = {2023},
  doi = {10.1109/asicon58565.2023.10396267},
}
Figure