Efficient and Modularized Training on FPGA for Real-time Applications

Shreyas Kolala Venkataramanaiah, Xiaocong Du, Zheng Li, Shihui Yin, Yu Cao, Jae-sun SeoIJCAI 2020 · OpenReview

Abstract

Training of deep Convolutional Neural Networks (CNNs) requires a tremendous amount of computation and memory; thus, GPUs are widely used to meet these demands. However, GPUs lack the flexibility for architectural optimizations, leading to poor energy efficiency and making them difficult to deploy on energy-constrained platforms. FPGAs are highly suitable for real-time learning at the edge, providing higher energy efficiency and flexibility. This paper develops a training accelerator on FPGA with 16-bit fixed-point computing and modular training components. By leveraging model segmentation techniques from Progressive Segmented Training (PST), the accelerator achieves significantly lower computation costs. We demonstrate performance on CIFAR-10 using an Intel Stratix-10 MX FPGA, evaluating both conventional training and the online learning algorithm.

Introduction

Deep learning models are traditionally trained on massive GPU clusters in the cloud. While effective for static datasets, this paradigm faces significant hurdles in real-time edge applications, such as autonomous drones or vehicles. Key challenges include high latency and privacy risks associated with cloud communication, the prohibitive power consumption of GPUs for battery-operated devices, and the phenomenon of "catastrophic forgetting" during online learning.

While FPGAs offer the energy efficiency and flexibility required for edge deployment, most existing designs are optimized for inference. This work addresses the complexity of on-device training by introducing a modularized FPGA-based accelerator that supports full training pipelines—forward pass, backward pass, and weight updates—optimized for real-time adaptation.

System Architecture and Methodology

The proposed solution integrates hardware and algorithmic innovations to enable efficient on-site learning. The hardware architecture is centered around a modularized 16-bit fixed-point training engine.

Modular Hardware Design

The system utilizes an RTL Generator that automatically produces hardware logic based on specific network parameters, such as CNN structure and precision requirements. This modularity allows the design to be reconfigured for various applications without manual hardware redesign. The core of the accelerator is a Customized Training Accelerator featuring:

  • Systolic MAC Array: A 2D array of Multiply-Accumulate units optimized for high-throughput matrix multiplications.
  • HBM2 Integration: High-bandwidth memory is employed to manage the massive data movement required for gradient calculations and weight updates.
  • 16-bit Fixed-Point Arithmetic: By utilizing fixed-point rather than standard 32-bit floating-point math, the design significantly reduces power consumption and hardware area with minimal impact on accuracy.
Overview of the modularized FPGA accelerator and the online adaptation workflow.

Progressive Segmented Training (PST)

To address the computational intensity of training and the risk of forgetting, the researchers implemented Progressive Segmented Training (PST). Instead of updating the entire network for every new data stream, PST identifies and "freezes" weights critical to previous knowledge. Only a small, selected segment of the network is updated with new data.

This hardware-algorithm co-design allows the control logic to skip memory accesses and computations for frozen weights. This direct translation of algorithmic "freezing" into hardware-level power and time savings is a key driver of the system's efficiency.

Experimental Evaluation

The system was demonstrated on an Intel Stratix-10 MX FPGA board running at 150 MHz. The researchers used the CIFAR-10 dataset to evaluate the transition from pre-trained models to real-time data acquisition (e.g., learning to identify "ships" or "trucks" on-site).

Key Results

The primary performance metric was the latency of the Weight Update (WU) phase, which is typically the most demanding part of the training process. The combination of the FPGA accelerator and the PST algorithm achieved a 4.2x reduction in training latency compared to conventional training methods.

Breakdown of training latency showing a 4.2x reduction using Progressive Segmented Training (PST).

Furthermore, the accelerator demonstrates end-to-end capability, handling the entire training pipeline on a single chip, a significant advancement over previous works that only accelerated specific sub-tasks.

Conclusion and Future Work

This research demonstrates that low-power, on-site learning is achievable through specialized FPGA architectures and segmented training algorithms. By reducing training latency by over 4x, the system makes real-time adaptation feasible for edge devices. Future research will focus on scaling this modular design to handle higher-resolution datasets like ImageNet and exploring the impact of 16-bit fixed-point precision on the accuracy of deeper architectures, such as Transformers.

Summary updated

Abstract

Training of deep Convolution Neural Networks (CNNs) requires a tremendous amount of computation and memory and thus, GPUs are widely used to meet the computation demands of these complex training tasks. However, lacking the flexibility to exploit architectural optimizations, GPUs have poor energy efficiency of GPUs and are hard to be deployed on energy-constrained platforms. FPGAs are highly suitable for training, such as real-time learning at the edge, as they provide higher energy efficiency and better flexibility to support algorithmic evolution. This paper first develops a training accelerator on FPGA, with 16-bit fixed-point computing and various training modules. Furthermore, leveraging model segmentation techniques from Progressive Segmented Training, the newly developed FPGA accelerator is applied to online learning, achieving much lower computation cost. We demonstrate the performance of representative CNNs trained for CIFAR-10 on Intel Stratix-10 MX FPGA, evaluating both the conventional training procedure and the online learning algorithm.
BibTeX
@inproceedings{venkataramanaiah2020efficient,
  author = {Shreyas Kolala Venkataramanaiah and Xiaocong Du and Zheng Li and Shihui Yin and Yu Cao and Jae-sun Seo},
  title = {{Efficient and Modularized Training on FPGA for Real-time Applications}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  url = {https://openreview.net/forum?id=tnw3KdtkbOM},
}
Figure