Heterogeneous Multi-core Array-based DNN Accelerator

Mohammad Ali Maleki, Mehdi Kamal, Ali Afzali-KushaarXiv 2022 · DOI · arXiv

Introduction

Deep Neural Networks (DNNs) have become essential for modern AI, yet their deployment in embedded systems is limited by high power consumption. While array-based accelerators—utilizing 2D grids of processing elements (PEs)—have improved efficiency, most current designs rely on a static, "one-size-fits-all" architecture. This paper argues that such static designs create a performance bottleneck because different network topologies (e.g., VGG16 vs. MobileNet) exhibit vastly different data movement patterns.

Hardware Architecture and Dataflow

The study utilizes a baseline architecture centered on a row-stationary dataflow. This design manages data movement across a three-level memory hierarchy: off-chip DRAM, a shared Global Buffer (GB), and local Register Files (RF) within each PE.

Figure 2: The baseline array-based accelerator architecture showing the memory hierarchy and PE array.

As shown in Figure 2, the Global Buffer is partitioned to store input feature maps (ifmap), weights, and partial sums (psum). The efficiency of the system depends heavily on the balance between the size of these buffers and the dimensions of the PE array. If the ifmap buffer is too small to feed a large array, the system is forced to frequently access off-chip DRAM, leading to a "breakpoint" where energy efficiency drops sharply.

Heterogeneous Multi-core Strategy

To address the diversity of DNN topologies, the authors propose a heterogeneous multi-core architecture. Rather than using identical cores, the chip features specialized cores tuned for specific network "families." For instance, heavy-duty networks like VGG require different hardware parameters than lightweight models like MobileNet to reach peak efficiency.

To further enhance performance, the paper introduces a branch-and-bound algorithm for load balancing across these cores. This ensures that when a network is parallelized, no single layer becomes a bottleneck for the entire pipeline.

Figure 6: Partitioning of the PE array to support multi-core parallelism and load balancing.

Results and Impact

The evaluation across 18 benchmark networks, ranging from AlexNet to MobileNet V2, demonstrated significant gains. By mapping networks to their near-optimal specialized cores, the researchers achieved up to a 36% reduction in energy consumption and a 67% improvement in the Energy-Delay Product (EDP).

The study also highlighted the cost of architectural mismatch: processing a network like GoogleNet on the "wrong" core type resulted in a 67.96% increase in EDP. Furthermore, the branch-and-bound load balancing algorithm enabled near-linear speedups (up to 3.92x on 4 cores) for many architectures.

Conclusion

This work proves that tailoring hardware to specific neural network topologies is essential for the next generation of AI accelerators. While the current study focuses on static heterogeneous cores, future research may extend these findings to reconfigurable hardware, allowing cores to adapt dynamically to different layers within a single network execution.

Summary updated

Abstract

In this article, we investigate the impact of architectural parameters of array-based DNN accelerators on accelerator's energy consumption and performance in a wide variety of network topologies. For this purpose, we have developed a tool that simulates the execution of neural networks on array-based accelerators and has the capability of testing different configurations for the estimation of energy consumption and processing latency. Based on our analysis of the behavior of benchmark networks under different architectural parameters, we offer a few recommendations for having an efficient yet high performance accelerator design. Next, we propose a heterogeneous multi-core chip scheme for deep neural network execution. The evaluations of a selective small search space indicate that the execution of neural networks on their near-optimal core configuration can save up to 36% and 67% of energy consumption and energy-delay product respectively. Also, we suggest an algorithm to distribute the processing of network's layers across multiple cores of the same type in order to speed up the computations through model parallelism. Evaluations on different networks and with the different number of cores verify the effectiveness of the proposed algorithm in speeding up the processing to near-optimal values.
BibTeX
@misc{maleki2022heterogeneous,
  author = {Mohammad Ali Maleki and Mehdi Kamal and Ali Afzali-Kusha},
  title = {{Heterogeneous Multi-core Array-based DNN Accelerator}},
  year = {2022},
  eprint = {2206.12605},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2206.12605},
}
Figure