Minimizing Global Buffer Access in a Deep Learning Accelerator Using a Local Register File with a Rearranged Computational Sequence

Minjae Lee, Zhongfeng Zhang, Seungwon Choi, Jungwook ChoiSensors 2022 · OpenReview

TL;DR

This paper presents a novel deep learning accelerator design that significantly reduces power consumption by minimizing global buffer access. It achieves this by intelligently reusing data through a shared local register file and a rearranged computational sequence, leading to substantial reductions in memory access and power usage.

What Problem Does This Paper Address?

Deep learning accelerators are crucial for handling the computational demands of neural networks. However, a major bottleneck and power drain in these accelerators is the frequent access to global buffers, which are used to store input data and weights. While existing methods attempt to mitigate this through data reuse, they often struggle with limited reuse factors or require significant hardware overhead. This paper tackles the challenge of reducing global buffer access and its associated power consumption by proposing a more efficient data reuse strategy within a deep learning accelerator.

Methods

The core of this paper's contribution lies in two key innovations:

  • Rearranged Computational Sequence: The authors propose a novel way to reorder the sequence of computations. This rearrangement ensures that input data pixels are accessed in a more predictable and regular pattern, facilitating greater data reuse.
  • Shared Local Register File: Instead of equipping each computation unit with its own register file (which is resource-intensive), this design utilizes a single local register file shared across a column of computation units. This shared register file stores frequently accessed input data, allowing subsequent computations to retrieve data from the faster, lower-power local register file instead of the power-hungry global buffer. The paper further introduces "intrablock" and "interblock" register files to manage data reuse within and between blocks of input data, respectively.

Datasets

The paper evaluates its proposed accelerator using the MNIST dataset for recognizing Arabic numbers. It also mentions testing on standard DNN models like VGG16, AlexNet, and ResNet50, as well as lightweight CNNs such as MobileNetV2 and EfficientNet-B0.

Key Insights and Ideas

  • Data Reuse is King (for Power Efficiency): The paper strongly emphasizes that the majority of power consumption in deep learning accelerators comes from data movement, not arithmetic operations. Maximizing data reuse is therefore paramount for power efficiency.
  • Predictability Enables Reuse: The key insight is that by rearranging the computational sequence, the authors create a predictable pattern of data access. This predictability is what unlocks the potential for efficient data reuse with local register files.
  • Sharing is Caring (for Resources): The decision to share a local register file across multiple computation units (MAC units) is a clever way to reduce hardware overhead while still achieving significant data reuse benefits.
  • Intra- vs. Inter-Block Reuse: The distinction between reusing data within a single block of input data (intrablock) and reusing data between consecutive blocks (interblock) is crucial for a comprehensive approach to minimizing global buffer access.

Key Results

The proposed accelerator demonstrates significant improvements:

  • Global Buffer Access Reduction: The accelerator reduces global buffer accesses by up to 86.8%.
  • Power Consumption Savings: This reduction in memory access translates to substantial power savings, with up to 72.3% less power consumed for input data memory access.
  • Resource Utilization: While there's a minor increase in hardware resources (around 8.6% for flip-flops and 10.8% for look-up tables), this is deemed a worthwhile trade-off for the significant power and access gains.
  • Performance on DNNs: When applied to standard DNN models like VGG16 and AlexNet, the accelerator achieves access gains of up to 86.75% and power gains of up to 72.28%. Even for models with less regular data reuse patterns like ResNet50, significant gains (45.10% access gain) are still observed.

Limitations and Future Work

  • Output Feature Width Dependency: The paper notes that the full benefits of the proposed method are realized when the output feature width is a multiple of the temporal data reuse factor. If this condition isn't met, the global buffer access gain decreases due to remaining input data pixels.
  • Weight Dimension Impact: The performance gain is influenced by the weight dimension. While the proposed method works well for common 3x3 and 5x5 weights, larger weights might require more complex management of inter-block register files.
  • Future Work: The paper suggests further investigation into how the computation of remaining pixels (when the output feature width is not a multiple of the temporal data reuse factor) affects performance. They also implicitly point towards exploring this architecture for a wider range of DNN models and applications.

Summary updated

Abstract

We propose a method for minimizing global buffer access within a deep learning accelerator for convolution operations by maximizing the data reuse through a local register file, thereby substituting the local register file access for the power-hungry global buffer access. To fully exploit the merits of data reuse, this study proposes a rearrangement of the computational sequence in a deep learning accelerator. Once input data are read from the global buffer, repeatedly reading the same data is performed only through the local register file, saving significant power consumption. Furthermore, different from prior works that equip local register files in each computation unit, the proposed method enables sharing a local register file along the column of the 2D computation array, saving resources and controlling overhead. The proposed accelerator is implemented on an off-the-shelf field-programmable gate array to verify the functionality and resource utilization. Then, the performance improvement of the proposed method is demonstrated relative to popular deep learning accelerators. Our evaluation indicates that the proposed deep learning accelerator reduces the number of global-buffer accesses to nearly 86.8%, consequently saving up to 72.3% of the power consumption for the input data memory access with a minor increase in resource usage compared to a conventional deep learning accelerator.
BibTeX
@inproceedings{lee2022minimizing,
  author = {Minjae Lee and Zhongfeng Zhang and Seungwon Choi and Jungwook Choi},
  title = {{Minimizing Global Buffer Access in a Deep Learning Accelerator Using a Local Register File with a Rearranged Computational Sequence}},
  booktitle = {Sensors},
  year = {2022},
  url = {https://openreview.net/forum?id=UiIxrrcv8T},
}
Figure