PQA: Exploring the Potential of Product Quantization in DNN Hardware Acceleration

Ahmed F. AbouElhamayed, Angela Cui, Javier Fernández-Marqués, Nicholas D. Lane, Mohamed S. Abdelfattah

TL;DR

This paper introduces PQA, a novel hardware accelerator designed to speed up Deep Neural Networks (DNNs) using Product Quantization (PQ). By creating custom hardware, they demonstrate significant performance gains over conventional methods and existing PQ approaches, paving the way for more efficient DNNs on edge devices.

What Problem Does This Paper Address?

Deep Neural Networks (DNNs) are computationally intensive, making them challenging to deploy on resource-constrained edge devices. While techniques like pruning and quantization help, they often involve trade-offs in accuracy or hardware suitability. Product Quantization (PQ) offers a promising avenue by replacing computationally expensive multiply-accumulate operations with memory lookups. However, previous PQ research has often provided limited evaluations, focusing on specific layers or simple networks, and neglecting the crucial hardware efficiency aspects. This paper aims to bridge this gap by thoroughly investigating PQ's potential for DNN hardware acceleration, from algorithmic parameterization to custom hardware design.

Methods

The core of this work is the Product Quantization Accelerator (PQA), a custom hardware architecture implemented on an FPGA. PQA is designed to efficiently handle PQ operations by:

  • Parallelizing Subspace Computations: PQ divides input vectors into subspaces, and PQA leverages this independence to process them in parallel.
  • Optimized Memory Hierarchy: It features a partitioned memory array for efficient lookups of pre-computed dot products.
  • Distance Calculation Module: This module efficiently finds the closest prototype for each input subspace.
  • Product Lookup Module: This module retrieves pre-computed partial dot products from the partitioned memory.
  • Accumulator: This module sums up the partial results from different subspaces.

The authors also explore low numerical bitwidth for PQ operations, quantizing prototypes and lookup tables to reduce hardware area and memory bandwidth requirements. They investigate various PQ parameterizations (number of prototypes $N_p$ and prototype length $L_s$) and training techniques to minimize accuracy degradation.

Datasets

The paper uses the following datasets:

  • CIFAR-10: For training and evaluating ResNet-20.
  • EMNIST: For training and evaluating a custom CNN (DWEMNIST).
  • SpeechCommands: For training and evaluating MicroNet-PQ (a keyword spotting task).

Key Insights and Ideas

  • Commodity Hardware is Not Ideal for PQ: CPUs and GPUs, while good for conventional DNNs, are not well-suited for PQ due to their architectural differences. Custom hardware is essential to realize PQ's acceleration potential.
  • Hardware Acceleration is Key: The PQA hardware demonstrates significant speedups (up to 3.1x) compared to conventional DNN accelerators, proving PQ's viability when implemented efficiently.
  • Low Bitwidth is Crucial: Quantizing PQ operations to as low as 2-6 bits can drastically reduce hardware area and memory bandwidth without significant accuracy loss, especially with per-subspace quantization.
  • Parameter Tuning is Critical: The choice of PQ parameters ($N_p$ and $L_s$) has a profound impact on accuracy, compute, and memory footprint. There's a trade-off: longer prototypes ($L_s$) and fewer of them ($N_p$) generally lead to better accuracy and smaller compute footprints, but can increase memory usage.
  • FLOPs are Not the Whole Story: While PQ reduces FLOPs, the actual speedup is heavily influenced by memory access patterns, which commodity hardware doesn't handle efficiently.

Key Results

  • Significant Speedup: The PQA achieves up to 3.1x higher performance-per-area compared to a conventional DNN accelerator (DLA) on ResNet-20.
  • Outperforming Prior PQ Work: PQA-Q (quantized PQA) outperforms the latest PQ literature (PECAN) by 4x in performance-per-area with only a 0.6% accuracy degradation.
  • Low Bitwidth Tolerance: PQ operations can tolerate very low bitwidths (e.g., 2-5 bits) with minimal accuracy impact, especially with per-subspace quantization.
  • MicroNet-PQ Accuracy: Achieves 95.0% accuracy with a 2.4x reduction in inference FLOPs.
  • DWEMNIST Performance: PQA-Q achieves 1.1M parameters and 82.8 MHz frequency with a 31x performance/area improvement over the baseline.

Limitations and Future Work

  • Model Size: The study primarily focuses on relatively small ML models. Applying PQ to much larger models is a natural next step.
  • Training Time: Training PQ models currently requires significantly longer training times, which is a practical limitation.
  • Hyperparameter Sensitivity: PQ is highly sensitive to hyperparameter settings, necessitating extensive search.
  • Complementary Compression: The authors suggest exploring PQ in conjunction with other compression techniques like channel pruning for further gains.
  • MLP Corrector: While an MLP corrector was explored to improve PQ-DNN accuracy, its computational overhead made it impractical for this work, suggesting future research in this area.
  • Hardware Vectorization: Further increasing hardware vectorization parameters could lead to even better performance gains.

Summary updated

Abstract

Conventional multiply-accumulate (MAC) operations have long dominated computation time for deep neural networks (DNNs). Recently, product quantization (PQ) has been successfully applied to these workloads, replacing MACs with memory lookups to pre-computed dot products. While this property makes PQ an attractive solution for model acceleration, little is understood about the associated trade-offs in terms of compute and memory footprint, and the impact on accuracy. Our empirical study investigates the impact of different PQ settings and training methods on layerwise reconstruction error and end-to-end model accuracy. When studying the efficiency of deploying PQ DNNs, we find that metrics such as FLOPs, number of parameters, and even CPU/GPU performance, can be misleading. To address this issue, and to more fairly assess PQ in terms of hardware efficiency, we design the first custom hardware accelerator to evaluate the speed and efficiency of running PQ models. We identify PQ configurations that are able to improve performance-per-area for ResNet20 by 40%-104%, even when compared to a highly optimized conventional DNN accelerator. Our hardware performance outperforms recent PQ solutions by 4x, with only a 0.6% accuracy degradation. This work demonstrates the practical and hardware-aware design of PQ models, paving the way for wider adoption of this emerging DNN approximation methodology.
BibTeX
@article{abouelhamayed2025pqa,
  author = {Ahmed F. AbouElhamayed and Angela Cui and Javier Fernández-Marqués and Nicholas D. Lane and Mohamed S. Abdelfattah},
  title = {{PQA: Exploring the Potential of Product Quantization in DNN Hardware Acceleration}},
  journal = {ACM Trans. Reconfigurable Technol. Syst.},
  year = {2025},
  doi = {10.1145/3656643},
}
Figure