End-to-end codesign of Hessian-aware quantized neural networks for FPGAs

Javier Campos, Jovan Mitrevski, Nhan Tran, Zhen Dong, Amir Gholaminejad, Michael W. Mahoney, Javier M. DuarteACM Trans. Reconfigurable Technol. Syst 2024 · DOI · OpenReview

TL;DR

This paper presents an end-to-end workflow for designing and deploying quantized neural networks (QNNs) on FPGAs and ASICs, specifically tailored for real-time applications like those in particle physics. By integrating Hessian-aware quantization (HAWQ) with the QONNX intermediate representation and the hls4ml tool, they enable efficient hardware implementation of QNNs, making them accessible to non-experts.

What Problem Does This Paper Address?

Deploying deep learning models on edge devices, especially for real-time applications with strict latency and resource constraints, is a significant challenge. This is particularly true in fields like particle physics, where massive amounts of data are generated at high rates (e.g., the 40 MHz collision rate at the LHC) and require immediate processing on specialized hardware like FPGAs and ASICs. Traditional neural network models are often too computationally expensive for these environments. Quantization, which reduces the precision of model parameters, is a key technique to address this, but finding the optimal quantization strategy (e.g., bit widths for different layers) that balances accuracy and hardware efficiency is complex and time-consuming. Existing tools and workflows often lack seamless integration, requiring expertise in both machine learning and hardware design.

Methods

The authors introduce an end-to-end workflow that combines several key components:

  • Hessian-Aware Quantization (HAWQ): This technique quantizes neural networks to very low bit widths by using second-order (Hessian) information to identify sensitive layers. Sensitive layers are kept at higher precision, while less sensitive layers are quantized more aggressively. This mixed-precision approach aims to minimize accuracy loss.
  • Quantized Open Neural Network Exchange (QONNX): To bridge HAWQ with hardware tools, the authors extend HAWQ to export QNNs into the QONNX format. QONNX is an intermediate representation designed to represent QNNs with arbitrary precision, offering greater flexibility than standard ONNX for quantized models.
  • hls4ml Tool Flow: The QONNX models are then ingested by hls4ml, an open-source Python library that translates neural networks into hardware descriptions (RTL) for FPGAs and ASICs using High-Level Synthesis (HLS). hls4ml is designed to be user-friendly for both ML and hardware experts.

The workflow involves training a QNN using HAWQ, exporting it to QONNX, optimizing the QONNX graph (e.g., constant folding, shape inference), and finally using hls4ml to generate hardware implementations.

Datasets

The paper uses a benchmark dataset for jet classification in particle physics. This dataset consists of 54 features from simulated particle jets produced in proton-proton collisions at the LHC. The task is to classify jets originating from one of five particle types: W bosons, Z bosons, light quarks (q), top quarks (t), or gluons (g). The dataset is divided into training (630,000 jets) and testing (240,000 jets) sets.

Key Insights and Ideas

  • Hessian-guided mixed-precision is crucial: Simply quantizing all layers uniformly leads to significant accuracy degradation, especially at very low bit widths (below INT8). Hessian information effectively identifies which layers are most sensitive to quantization, allowing for a more intelligent mixed-precision strategy that preserves accuracy.
  • QONNX bridges the gap: The extension of HAWQ to export to QONNX is a critical step, enabling interoperability with hardware synthesis tools like hls4ml. This intermediate representation is key to making custom quantization schemes accessible for hardware deployment.
  • hls4ml democratizes hardware deployment: By integrating with hls4ml, the workflow makes it significantly easier for researchers without deep hardware expertise to deploy optimized QNNs onto FPGAs and ASICs.
  • Real-world constraints drive optimization: The application in particle physics, with its stringent 40 MHz collision rate requirement, highlights the necessity of efficient hardware implementations and the trade-offs between accuracy, latency, and resource usage.

Key Results

The authors demonstrate their workflow on a particle physics jet classification task. They show that their Hessian-aware mixed-precision quantization approach can achieve competitive accuracy while significantly reducing hardware resource usage compared to a baseline floating-point model.

  • The Hessian-aware solution uses 42.2% fewer LUTs, 36.3% fewer FFs, and 95.7% fewer DSPs compared to the baseline model.
  • While accuracy drops slightly (e.g., from 76.85% for baseline to 75.78% for the Hessian-aware solution), this is considered a reasonable trade-off for the substantial resource savings.
  • The workflow successfully translates HAWQ models into QONNX and then into FPGA firmware, demonstrating the end-to-end capability.

Limitations and Future Work

  • Focus on MLPs: The current study primarily focuses on Multilayer Perceptrons (MLPs). While the authors state that the NN architectures can be exported to ONNX/QONNX, further work could explore the application of this workflow to more complex architectures like Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs).
  • Binary/Ternary Quantization: The paper mentions that binary and ternary quantization were not explored in this study, suggesting this as a potential area for future research.
  • ASIC Deployment: While the workflow is designed for both FPGAs and ASICs, the presented results focus on FPGA synthesis. Future work could include detailed ASIC synthesis results.
  • Automated Bit Width Selection: The paper utilizes an automated method for bit width selection based on Hessian information and BOPs constraints. Further refinement of this automated process could lead to even better trade-offs.

Summary updated

Abstract

We develop an end-to-end workflow for the training and implementation of co-designed neural networks (NNs) for efficient field-programmable gate array (FPGA) hardware. Our approach leverages Hessian-aware quantization of NNs, the Quantized Open Neural Network Exchange intermediate representation, and the hls4ml tool flow for transpiling NNs into FPGA firmware. This makes efficient NN implementations in hardware accessible to nonexperts in a single open sourced workflow that can be deployed for real-time machine-learning applications in a wide range of scientific and industrial settings. We demonstrate the workflow in a particle physics application involving trigger decisions that must operate at the 40-MHz collision rate of the CERN Large Hadron Collider (LHC). Given the high collision rate, all data processing must be implemented on FPGA hardware within the strict area and latency requirements. Based on these constraints, we implement an optimized mixed-precision NN classifier for high-momentum particle jets in simulated LHC proton-proton collisions.
BibTeX
@article{campos2024end,
  author = {Javier Campos and Jovan Mitrevski and Nhan Tran and Zhen Dong and Amir Gholaminejad and Michael W. Mahoney and Javier M. Duarte},
  title = {{End-to-end codesign of Hessian-aware quantized neural networks for FPGAs}},
  journal = {ACM Trans. Reconfigurable Technol. Syst.},
  year = {2024},
  doi = {10.1145/3662000},
}
Figure