HFNet: A CNN Architecture Co-designed for Neuromorphic Hardware With a Crossbar Array of Synapses

Roshan Gopalakrishnan, Yansong Chua, Pengfei Sun, Ashish Jith Sreejith Kumar, Arindam BasuFrontiers in Neuroscience 2020 · DOI

TL;DR

This paper introduces HFNet, a novel Convolutional Neural Network (CNN) architecture specifically designed for efficient mapping onto neuromorphic hardware featuring crossbar arrays of synapses. By co-designing the network architecture with hardware constraints in mind, HFNet achieves comparable accuracy to established models like VGG-16 while drastically reducing the number of neuromorphic cores required for deployment.

What Problem Does This Paper Address?

The burgeoning field of neuromorphic computing promises highly energy-efficient hardware for AI tasks. However, a significant challenge lies in effectively mapping complex neural network architectures, particularly CNNs, onto these specialized hardware platforms. Neuromorphic chips often have inherent constraints, such as limited core sizes and low precision for synaptic weights. Existing methods for mapping CNNs to these architectures can lead to inefficient utilization of hardware resources, requiring many cores or even "core matrix splitting" (where a single layer's computation is spread across multiple cores), which can complicate hardware design and increase overhead. This paper addresses the need for a CNN architecture that is inherently compatible with neuromorphic hardware, minimizing these mapping challenges and maximizing hardware efficiency without sacrificing accuracy.

Methods

The authors propose an iterative co-design process that bridges the gap between CNN architecture design and neuromorphic hardware constraints. Key methodological contributions include:

  • Convolutional Layer Analysis: They analyze different convolution techniques (standard, depthwise, pointwise, grouped) to identify which are more hardware-friendly for crossbar architectures. Depthwise separable convolutions are highlighted as particularly suitable due to their lower fan-in degree, which helps avoid core matrix splitting.
  • Mapping Techniques: The paper explores and compares various mapping strategies for CNN layers onto crossbar arrays, including block, toeplitz, and hybrid methods. They demonstrate that hybrid mapping can significantly reduce the number of cores needed compared to the toeplitz method.
  • Hardware-Friendly Architecture Design (HFNet): Based on their insights, they propose HFNet, a modular CNN architecture that incorporates hardware-friendly layers. This involves using standard convolutions in shallow layers, depthwise separable convolutions in deeper layers, and replacing fully connected layers with Global Average Pooling (GAP) to reduce fan-in requirements.
  • Co-design Framework: A Python wrapper (MaD framework) is developed to facilitate the iterative co-design process, allowing for mapping, validation, and estimation of core requirements. This framework enables systematic modification of network parameters (like the number of feature maps) to adhere to hardware constraints, such as axon size (AS) and core size.

Datasets

The primary dataset used for training and evaluating the HFNet architecture is IMAGENET. The authors also mention using CIFAR-10 for benchmarking mapping methods.

Key Insights and Ideas

  • Co-design is Crucial: The paper strongly advocates for a co-design approach where CNN architecture design and hardware mapping are considered simultaneously, rather than as separate steps.
  • Depthwise Separable Convolutions are Hardware-Friendly: These layers significantly reduce the fan-in degree per neuron, which is critical for avoiding core matrix splitting on neuromorphic hardware.
  • Global Average Pooling (GAP) is a Viable Replacement for Fully Connected Layers: GAP offers a hardware-friendly alternative to large fully connected layers, which often pose mapping challenges due to high fan-in.
  • Square Core Shapes are Generally Optimal: For minimizing the number of axons required for mapping, selecting neurons in a square shape (Neuronrow = Neuroncol) is mathematically shown to be optimal.
  • Mapping Techniques Matter: The choice of mapping strategy (e.g., hybrid vs. toeplitz) can dramatically impact the number of neuromorphic cores required.

Key Results

  • Core Reduction: The proposed hybrid mapping technique reduced the number of cores by 4.9–13.8 times compared to the toeplitz method for a seven-layered CNN across various crossbar sizes.
  • HFNet Performance: The HFNet architecture achieved a classification accuracy of 71.3% on IMAGENET, comparable to VGG-16.
  • Efficiency Gains: HFNet used 11 times fewer cores than a standard CNN mapping on a 1,024 × 1,024 neuromorphic core.
  • Flexibility: Different versions of HFNet were developed and evaluated for various core sizes, demonstrating adaptability.
  • Impact of Design Choices: Experiments showed that adding pooling layers or fully connected layers to HFNet-V3 provided only marginal accuracy improvements but significantly increased the number of cores required, validating the hardware-friendly design choices.

Limitations and Future Work

  • Core Matrix Splitting: While the paper aims to avoid core matrix splitting, it acknowledges that for very large fan-in degrees, splitting might still be necessary, requiring additional hardware considerations.
  • Limited Number of Cores: The current HFNet requires thousands of cores for mapping, which is still more than what many existing neuromorphic chips offer. Future work will focus on mapping HFNet onto chips with a limited number of cores.
  • Fault Tolerance: The authors suggest exploring how to reuse synapses for fault tolerance and investigate redundancy to mitigate hardware imperfections.
  • Quantization: Future work will consider quantized CNNs (both weights and activations) to further reduce hardware requirements.
  • Synaptic Noise: The impact of synaptic noise in novel devices will be investigated.
  • Skip Connections: The current HFNet does not include skip connections (as in ResNets). Mapping these architectures, which increase fan-in/fan-out degrees, is a promising avenue for future research.
  • Core Shapes: While the paper explores different core sizes, further investigation into how various core shapes (e.g., rectangular vs. square) affect mapping efficiency is suggested.

Summary updated

Abstract

The hardware-software co-optimization of neural network architectures is a field of research that emerged with the advent of commercial neuromorphic chips, such as the IBM TrueNorth and Intel Loihi. Development of simulation and automated mapping software tools in tandem with the design of neuromorphic hardware, whilst taking into consideration the hardware constraints, will play an increasingly significant role in deployment of system-level applications. This paper illustrates the importance and benefits of co-design of convolutional neural networks (CNN) that are to be mapped onto neuromorphic hardware with a crossbar array of synapses. Toward this end, we first study which convolution techniques are more hardware friendly and propose different mapping techniques for different convolutions. We show that, for a seven-layered CNN, our proposed mapping technique can reduce the number of cores used by 4.9-13.8 times for crossbar sizes ranging from 128 × 256 to 1,024 × 1,024, and this can be compared to the toeplitz method of mapping. We next develop an iterative co-design process for the systematic design of more hardware-friendly CNNs whilst considering hardware constraints, such as core sizes. A python wrapper, developed for the mapping process, is also useful for validating hardware design and studies on traffic volume and energy consumption. Finally, a new neural network dubbed HFNet is proposed using the above co-design process; it achieves a classification accuracy of 71.3% on the IMAGENET dataset (comparable to the VGG-16) but uses 11 times less cores for neuromorphic hardware with core size of 1,024 × 1,024. We also modified the HFNet to fit onto different core sizes and report on the corresponding classification accuracies. Various aspects of the paper are patent pending.
BibTeX
@inproceedings{gopalakrishnan2020hfnet,
  author = {Roshan Gopalakrishnan and Yansong Chua and Pengfei Sun and Ashish Jith Sreejith Kumar and Arindam Basu},
  title = {{HFNet: A CNN Architecture Co-designed for Neuromorphic Hardware With a Crossbar Array of Synapses}},
  booktitle = {Frontiers in Neuroscience},
  year = {2020},
  doi = {10.3389/fnins.2020.00907},
}
Figure