TL;DR
This paper presents a novel deep learning accelerator design that significantly reduces power consumption by minimizing global buffer access. It achieves this by intelligently reusing data through a shared local register file and a rearranged computational sequence, leading to substantial reductions in memory access and power usage.
What Problem Does This Paper Address?
Deep learning accelerators are crucial for handling the computational demands of neural networks. However, a major bottleneck and power drain in these accelerators is the frequent access to global buffers, which are used to store input data and weights. While existing methods attempt to mitigate this through data reuse, they often struggle with limited reuse factors or require significant hardware overhead. This paper tackles the challenge of reducing global buffer access and its associated power consumption by proposing a more efficient data reuse strategy within a deep learning accelerator.
Methods
The core of this paper's contribution lies in two key innovations:
- Rearranged Computational Sequence: The authors propose a novel way to reorder the sequence of computations. This rearrangement ensures that input data pixels are accessed in a more predictable and regular pattern, facilitating greater data reuse.
- Shared Local Register File: Instead of equipping each computation unit with its own register file (which is resource-intensive), this design utilizes a single local register file shared across a column of computation units. This shared register file stores frequently accessed input data, allowing subsequent computations to retrieve data from the faster, lower-power local register file instead of the power-hungry global buffer. The paper further introduces "intrablock" and "interblock" register files to manage data reuse within and between blocks of input data, respectively.
Datasets
The paper evaluates its proposed accelerator using the MNIST dataset for recognizing Arabic numbers. It also mentions testing on standard DNN models like VGG16, AlexNet, and ResNet50, as well as lightweight CNNs such as MobileNetV2 and EfficientNet-B0.
Key Insights and Ideas
- Data Reuse is King (for Power Efficiency): The paper strongly emphasizes that the majority of power consumption in deep learning accelerators comes from data movement, not arithmetic operations. Maximizing data reuse is therefore paramount for power efficiency.
- Predictability Enables Reuse: The key insight is that by rearranging the computational sequence, the authors create a predictable pattern of data access. This predictability is what unlocks the potential for efficient data reuse with local register files.
- Sharing is Caring (for Resources): The decision to share a local register file across multiple computation units (MAC units) is a clever way to reduce hardware overhead while still achieving significant data reuse benefits.
- Intra- vs. Inter-Block Reuse: The distinction between reusing data within a single block of input data (intrablock) and reusing data between consecutive blocks (interblock) is crucial for a comprehensive approach to minimizing global buffer access.
Key Results
The proposed accelerator demonstrates significant improvements:
- Global Buffer Access Reduction: The accelerator reduces global buffer accesses by up to 86.8%.
- Power Consumption Savings: This reduction in memory access translates to substantial power savings, with up to 72.3% less power consumed for input data memory access.
- Resource Utilization: While there's a minor increase in hardware resources (around 8.6% for flip-flops and 10.8% for look-up tables), this is deemed a worthwhile trade-off for the significant power and access gains.
- Performance on DNNs: When applied to standard DNN models like VGG16 and AlexNet, the accelerator achieves access gains of up to 86.75% and power gains of up to 72.28%. Even for models with less regular data reuse patterns like ResNet50, significant gains (45.10% access gain) are still observed.
Limitations and Future Work
- Output Feature Width Dependency: The paper notes that the full benefits of the proposed method are realized when the output feature width is a multiple of the temporal data reuse factor. If this condition isn't met, the global buffer access gain decreases due to remaining input data pixels.
- Weight Dimension Impact: The performance gain is influenced by the weight dimension. While the proposed method works well for common 3x3 and 5x5 weights, larger weights might require more complex management of inter-block register files.
- Future Work: The paper suggests further investigation into how the computation of remaining pixels (when the output feature width is not a multiple of the temporal data reuse factor) affects performance. They also implicitly point towards exploring this architecture for a wider range of DNN models and applications.
Summary updated