Scaling Binarized Neural Networks on Reconfigurable Logic

Nicholas J. Fraser, Yaman Umuroglu, Giulio Gambardella, Michaela Blott, Philip Heng Wai Leong, Magnus Jahre, Kees A. VissersParma-ditam@hipeac 2017 · DOI

TL;DR

This paper tackles the challenge of scaling Binarized Neural Networks (BNNs) for high-performance inference on FPGAs. They introduce an improved framework that enables efficient BNN deployment on large FPGAs, achieving state-of-the-art classification rates with significantly lower power consumption compared to GPUs.

What Problem Does This Paper Address?

Deep learning models, particularly Convolutional Neural Networks (CNNs), achieve impressive accuracy but demand substantial computational and memory resources. This makes them challenging to deploy on resource-constrained hardware like FPGAs. Binarized Neural Networks (BNNs) offer a promising solution by drastically reducing computational and memory costs through the use of 1-bit weights and activations. However, deploying larger, more complex BNNs on FPGAs while maintaining high accuracy and performance has been a significant hurdle. Existing frameworks like FINN were effective for smaller networks but struggled with scalability, accuracy degradation due to issues like padding, and efficient resource utilization on larger FPGAs. This paper aims to bridge this gap by improving FINN to handle larger BNNs, addressing padding inefficiencies, and demonstrating their scalability on powerful FPGA platforms.

Methods

The core of this work builds upon the FINN framework, which generates custom FPGA accelerators for BNNs. Key methodological contributions include:

  • Improved Padding for BNNs: Traditional CNNs use zero-padding to prevent border pixels from being "washed out." However, BNNs operate with {-1, +1} values, making zero-padding problematic as it would require a 2-bit datapath. The authors propose using -1 as a padding value. Surprisingly, this "negative one padding" achieves comparable accuracy to zero-padding while maintaining the crucial 1-bit datapath, thus preserving the efficiency benefits of BNNs.
  • Scalability Analysis and Optimization: The paper analyzes FINN's performance on larger BNN topologies and FPGAs, identifying bottlenecks related to BRAM (Block RAM) usage and throughput. They demonstrate how to tailor compute resources per layer to meet specific throughput requirements.
  • Architectural Insights: The authors discuss how certain CNN layer properties map more efficiently to FINN and explore the trade-offs between network size, accuracy, and computational cost. They also identify BRAM under-utilization as a potential issue for very large networks and propose an architectural modification for future work to address this.

Datasets

The primary dataset used for training and evaluation in this paper is CIFAR-10. The authors trained and evaluated convolutional BNNs on this dataset to investigate the effects of different padding modes.

Key Insights and Ideas

  • -1 Padding is a Game Changer for BNNs: The most significant insight is that using -1 for padding in BNNs, instead of requiring a 2-bit datapath for zero-padding, works just as well for accuracy. This is a crucial finding that allows BNNs to maintain their full binary efficiency even when employing padding, a common practice in CNNs.
  • BNNs Scale to Impressive Performance on FPGAs: The paper demonstrates that BNNs, when deployed on powerful FPGAs like the ADM-PCIE-8K5, can achieve remarkably high inference rates and low latency, outperforming GPUs in terms of power efficiency for comparable tasks.
  • Padding Trade-offs are Critical: While padding improves accuracy, it comes at the cost of increased operations. The paper highlights that a smaller, padded BNN can sometimes be more advantageous than a larger, unpadded one, depending on the specific application's requirements.
  • BRAM Utilization is a Bottleneck for Extreme Scaling: For very large networks, BRAM efficiency becomes a concern. The current FINN architecture's way of handling convolutions leads to under-utilization of BRAM, which could limit future scaling to even larger models.

Key Results

  • State-of-the-Art Classification: The authors achieved a classification rate of 12 kFPS with 671 µs latency on the CIFAR-10 dataset using a large BNN (cnn(1)) on an ADM-PCIE-8K5 platform, consuming less than 41W of board power.
  • High Throughput: The implementation achieved 14.8 trillion operations per second (TOPS), which they claim is the fastest classification rate reported for this benchmark at this accuracy level.
  • Accuracy with -1 Padding: The -1-padded networks achieved accuracy comparable to 0-padded networks, with the largest network (cnn(1)) reaching 88.3% accuracy with -1 padding, compared to 88.6% with 0-padding and 84.2% with no-padding.
  • Power Efficiency: The demonstrated power consumption of <41W for 12 kFPS classification on the FPGA significantly outperforms GPUs, which might consume hundreds of watts for similar tasks.

Limitations and Future Work

  • BRAM Under-utilization: The paper acknowledges that the current FINN architecture exhibits low BRAM utilization for large networks. They propose an architectural modification to the Matrix-Vector-Threshold Unit (MVTU) to enable multiplying a matrix by multiple vectors in parallel, which could improve BRAM efficiency.
  • Power Measurement Granularity: While they report a board power benchmark of 41W, they note that per-experiment power measurements were not available, and expect real power dissipation for BNN accelerators to be even lower.
  • Future Work: The authors plan to further enhance FINN to support:
  • Partial binarization (where not all weights/activations are binary).
  • Different types of convolutional layers, such as inception layers and fire modules.
  • Implementation of networks trained on larger datasets like ImageNet.
  • Improved power measurements.

Summary updated

BibTeX
@inproceedings{fraser2017scaling,
  author = {Nicholas J. Fraser and Yaman Umuroglu and Giulio Gambardella and Michaela Blott and Philip Heng Wai Leong and Magnus Jahre and Kees A. Vissers},
  title = {{Scaling Binarized Neural Networks on Reconfigurable Logic}},
  booktitle = {Parma-ditam@hipeac},
  year = {2017},
  doi = {10.1145/3029580.3029586},
}
Figure