Introduction
Deep Neural Networks (DNNs) have become essential for modern AI, yet their deployment in embedded systems is limited by high power consumption. While array-based accelerators—utilizing 2D grids of processing elements (PEs)—have improved efficiency, most current designs rely on a static, "one-size-fits-all" architecture. This paper argues that such static designs create a performance bottleneck because different network topologies (e.g., VGG16 vs. MobileNet) exhibit vastly different data movement patterns.
Hardware Architecture and Dataflow
The study utilizes a baseline architecture centered on a row-stationary dataflow. This design manages data movement across a three-level memory hierarchy: off-chip DRAM, a shared Global Buffer (GB), and local Register Files (RF) within each PE.
As shown in Figure 2, the Global Buffer is partitioned to store input feature maps (ifmap), weights, and partial sums (psum). The efficiency of the system depends heavily on the balance between the size of these buffers and the dimensions of the PE array. If the ifmap buffer is too small to feed a large array, the system is forced to frequently access off-chip DRAM, leading to a "breakpoint" where energy efficiency drops sharply.
Heterogeneous Multi-core Strategy
To address the diversity of DNN topologies, the authors propose a heterogeneous multi-core architecture. Rather than using identical cores, the chip features specialized cores tuned for specific network "families." For instance, heavy-duty networks like VGG require different hardware parameters than lightweight models like MobileNet to reach peak efficiency.
To further enhance performance, the paper introduces a branch-and-bound algorithm for load balancing across these cores. This ensures that when a network is parallelized, no single layer becomes a bottleneck for the entire pipeline.
Results and Impact
The evaluation across 18 benchmark networks, ranging from AlexNet to MobileNet V2, demonstrated significant gains. By mapping networks to their near-optimal specialized cores, the researchers achieved up to a 36% reduction in energy consumption and a 67% improvement in the Energy-Delay Product (EDP).
The study also highlighted the cost of architectural mismatch: processing a network like GoogleNet on the "wrong" core type resulted in a 67.96% increase in EDP. Furthermore, the branch-and-bound load balancing algorithm enabled near-linear speedups (up to 3.92x on 4 cores) for many architectures.
Conclusion
This work proves that tailoring hardware to specific neural network topologies is essential for the next generation of AI accelerators. While the current study focuses on static heterogeneous cores, future research may extend these findings to reconfigurable hardware, allowing cores to adapt dynamically to different layers within a single network execution.
Summary updated