FAMES: Fast Approximate Multiplier Substitution for Mixed-Precision Quantized DNNs-Down to 2 Bits!

Yi Ren, Ruge Xu, Xinfei Guo, Weikang Qian

TL;DR

This paper introduces FAMES, a novel method for significantly reducing the energy consumption of deep neural networks (DNNs) by intelligently replacing exact multipliers with approximate ones, even in extremely low-bitwidth (2-bit) quantized models. FAMES achieves this with minimal accuracy loss and is dramatically faster than previous approaches, making energy-efficient AI more practical.

What Problem Does This Paper Address?

The drive for energy-efficient AI has led to two main techniques: quantization (reducing the precision of numbers in a model) and the use of approximate multipliers (AppMuls). While quantization has successfully pushed bitwidths down to 2 bits, most AppMul research still operates at much higher bitwidths (e.g., 8-bit). This creates a mismatch: high-bitwidth AppMuls are energy-hungry and negate the benefits of low-bitwidth quantization. The core problem this paper tackles is whether AppMuls can be effectively and efficiently applied to the ultra-low-bitwidth quantized DNNs that represent the state-of-the-art in energy efficiency. Additionally, existing methods for selecting AppMuls are computationally expensive, hindering their practical adoption.

Methods

FAMES is a three-step framework designed to address the challenges of applying AppMuls to low-bitwidth quantized DNNs:

  1. AppMul Error Modeling: The paper introduces a novel "counting matrix" to model the error introduced by an AppMul. This matrix captures how often specific input pairs occur during computation, allowing for a more precise estimation of the AppMul's impact.
  2. Perturbation Estimation: Using the error model, FAMES employs a Taylor expansion to efficiently estimate the loss perturbation caused by an AppMul in a specific layer. Crucially, the core components of this estimation (gradients and Hessians of the loss with respect to the error) are computed only once, making it efficient even when evaluating many AppMuls.
  3. Optimized AppMul Selection (ILP): The problem of selecting the best AppMul for each layer is formulated as an Integer Linear Programming (ILP) problem. This ILP minimizes the estimated loss perturbation while respecting energy constraints, offering a more systematic and efficient approach than traditional genetic algorithms.
  4. Calibration: After AppMul substitution, a calibration step using Learnable Weight Clipping (LWC) is applied. This process fine-tunes scaling factors and clipping bounds to recover accuracy lost due to approximation, without requiring full model retraining.

Datasets

The paper evaluates FAMES on the following datasets:

  • CIFAR-10: Used for evaluating models like ResNet-20, ResNet-8, ResNet-14, and VGG-19.
  • ImageNet: Used for evaluating ResNet-18.

The paper does not introduce new benchmarks but rather applies its method to existing models and datasets to demonstrate its effectiveness.

Key Insights and Ideas

  • AppMuls are Viable at 2-bit: The paper definitively shows that approximate multipliers can be effectively integrated into DNNs quantized to as low as 2 bits, challenging previous assumptions.
  • Error Modeling is Key: The novel counting matrix and Taylor expansion-based perturbation estimation are central to FAMES's success, enabling accurate prediction of AppMul impact without extensive simulation.
  • ILP for Efficient Selection: Formulating AppMul selection as an ILP problem provides a significant speedup over heuristic or evolutionary approaches, making the process practical.
  • Calibration Recovers Accuracy: The post-substitution calibration step is crucial for mitigating accuracy degradation, demonstrating that retraining isn't always necessary.
  • Bridging the Gap: FAMES successfully bridges the gap between the ultra-low bitwidths achieved by quantization and the energy-saving potential of approximate computing.

Key Results

  • Energy Reduction: FAMES achieves an average of 28.67% energy reduction over corresponding quantized models with the same bitwidth.
  • Accuracy Preservation: Accuracy loss is kept under 1% on average.
  • Speedup: FAMES is up to 300x faster than previous genetic algorithm-based AppMul selection methods.
  • Low Bitwidth Performance: The method demonstrates effectiveness even with 2-bit quantized models, achieving significant energy savings (e.g., 35.90% for ResNet-20 with 2-bit quantization).
  • Mixed-Precision Advantage: When combined with mixed-precision quantization, FAMES can further reduce energy consumption by up to 33.31%.

Limitations and Future Work

  • AppMul Library: The performance of FAMES is dependent on the available AppMul library. The paper uses AppMuls from EvoLib8b and ALSRAC, but a more comprehensive or specialized library could yield further improvements.
  • Model Complexity: While tested on common CNN architectures, the applicability to more complex models like large language models (LLMs) is mentioned as future work.
  • Hardware Implementation: The paper focuses on simulation-based results. Future work would involve actual hardware implementation to validate the energy savings and performance gains.

The authors suggest extending FAMES to other AI models, including large language models, and exploring more advanced AppMul design and selection strategies.

Summary updated

Abstract

A widely-used technique in designing energy-efficient deep neural network (DNN) accelerators is quantization. Recent progress in this direction has reduced the bitwidths used in DNN down to 2. Meanwhile, many prior works apply approximate multipliers (AppMuls) in designing DNN accelerators to lower their energy consumption. Unfortunately, these works still assume a bitwidth much larger than 2, which falls far behind the state-of-the-art in quantization area and even challenges the meaningfulness of applying AppMuls in DNN accelerators, since a high-bitwidth AppMul consumes much more energy than a low-bitwidth exact multiplier! Thus, an important problem to study is: Can approximate multipliers be effectively applied to quantized DNN models with very low bitwidths? In this work, we give an affirmative answer to this question and present a systematic solution that achieves the answer: FAMES, a fast approximate multiplier substitution method for mixed-precision DNNs. Our experiments demonstrate an average 28.67% energy reduction on state-of-the-art mixed-precision quantized models with bitwidths as low as 2 bits and accuracy losses kept under 1%. Additionally, our approach is up to 300x faster than previous genetic algorithm-based methods.
BibTeX
@misc{ren2024fames,
  author = {Yi Ren and Ruge Xu and Xinfei Guo and Weikang Qian},
  title = {{FAMES: Fast Approximate Multiplier Substitution for Mixed-Precision Quantized DNNs-Down to 2 Bits!}},
  year = {2024},
  eprint = {2411.18055},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.18055},
}
Figure