Abstract
Quantizing weights and activations of deep neural networks results in significant improvement in inference efficiency at the cost of lower accuracy. A source of the accuracy gap between full precision and quantized models is the quantization error. In this work, we focus on the binary quantization, in which values are mapped to -1 and 1. We provide a unified framework to analyze different scaling strategies. Inspired by the pareto-optimality of 2-bits versus 1-bit quantization, we introduce a novel 2-bits quantization with provably least squares error. Our quantization algorithms can be implemented efficiently on the hardware using bitwise operations. We present proofs to show that our proposed methods are optimal, and also provide empirical error analysis. We conduct experiments on the ImageNet dataset and show a reduced accuracy gap when using the proposed least squares quantization algorithms.
Introduction
Deep Neural Networks (DNNs) have revolutionized computer vision and natural language processing, but their deployment on edge devices—like smartphones, wearables, and IoT sensors—is often hindered by high computational and memory requirements. Binary Neural Networks (BNNs) represent a promising solution by quantizing weights and activations to a single bit (-1 or 1). This extreme compression allows for massive memory savings and enables the replacement of expensive floating-point multiplications with efficient bitwise XNOR and "popcount" (bit-counting) operations.
However, the transition from high-precision to binary values introduces significant "quantization error," leading to a substantial drop in model accuracy. While previous research utilized scaling factors to mitigate this error, these factors were often sub-optimal. This paper addresses this gap by treating quantization as a least squares optimization problem, deriving mathematically optimal scaling factors and introducing a novel 2-bit "foldable" quantization scheme that balances efficiency and performance.
The 2-Bit Pareto Frontier
A key motivation for this work is the observation of a "sweet spot" in the trade-off between computational budget and model error. While 1-bit quantization is the most efficient, the jump to 2-bit quantization yields a disproportionately large improvement in accuracy.
As shown in the figure above, 2-bit quantization (2/1) sits on the Pareto frontier, offering the best accuracy-to-cost trade-off. This insight led the authors to focus on developing a provably optimal 2-bit quantization method that remains hardware-efficient.
Methodology: Least Squares and Foldable Quantization
The authors propose a unified framework for binary quantization, treating the search for scaling factors as a minimization of the squared error between the original tensor $x$ and its quantized version $x^q$.
Optimal 1-Bit Scaling
For 1-bit quantization, where $x \approx v \cdot \text{sign}(x)$, the framework proves that the optimal scaling factor $v$ is the mean of the absolute values of the tensor entries. This provides a solid theoretical foundation for common practices in the field.
Least Squares 2-Bits (LS-2)
The core contribution is the LS-2 algorithm, which provides a closed-form solution for optimal 2-bit quantization. Unlike "greedy" methods that solve for bits sequentially, LS-2 finds two scaling factors ($v_1$ and $v_2$) simultaneously to minimize the total squared error.
Foldable Quantization
To maintain the speed of BNNs, the authors introduce Foldable Quantization. A $k$-bit value is represented as a sum of binary terms, ensuring that even with multiple bits, the core computation can be "folded" into efficient XNOR-based kernels. This ensures that the mathematical optimality of the LS approach does not come at the expense of hardware performance.
Layer-wise Error Analysis
To understand why the LS-2 approach outperforms standard greedy methods, the authors analyzed the quantization error across different layers of a neural network.
The analysis reveals that as data flows through the network, distributions often become skewed. Greedy Foldable (GF) methods, which pick bits one by one, struggle with these distributions. In contrast, the simultaneous optimization in LS-2 consistently maintains a lower error (represented by a smaller angle between the original and quantized vectors) throughout the entire depth of the model.
Experimental Results
The researchers evaluated their proposed methods on the ImageNet (ILSVRC12) benchmark using the ResNet-18 architecture. The results demonstrate a significant narrowing of the "quantization gap":
- 1-Bit Performance: The LS 1-bit implementation achieved 58.9% top-1 accuracy, outperforming classic XNOR-Net (51.2%) and Bi-Real Net (56.4%).
- 2-Bit Performance: The LS-2 method reached 63.4% top-1 accuracy, coming remarkably close to the full-precision baseline of 69.6%.
- Post-Training Quantization: In scenarios without fine-tuning, LS-2 proved to be more than an order of magnitude more accurate than greedy 2-bit alternatives, highlighting its robustness.
Conclusion and Future Work
By framing quantization as a least squares problem, this work provides both theoretical optimality and practical performance gains for binary neural networks. The introduction of foldable quantization ensures these gains are translatable to real-world hardware.
Future research directions include finding global optima for bit-widths greater than two, where the authors currently rely on a greedy heuristic. Additionally, while the theoretical framework is robust, further benchmarking on specific mobile hardware (CPUs/NPUs) will be essential to fully realize the deployment potential of these optimized models.
Summary updated