Abstract
Quantizing weights and activations of deep neural networks results in significant improvement in inference efficiency at the cost of lower accuracy. A source of the accuracy gap between full precision and quantized models is the quantization error. In this work, we focus on the binary quantization, in which values are mapped to -1 and 1. We provide a unified framework to analyze different scaling strategies. Inspired by the pareto-optimality of 2-bits versus 1-bit quantization, we introduce a novel 2-bits quantization with provably least squares error. Our quantization algorithms can be implemented efficiently on the hardware using bitwise operations. We present proofs to show that our proposed methods are optimal, and also provide empirical error analysis. We conduct experiments on the ImageNet dataset and show a reduced accuracy gap when using the proposed least squares quantization algorithms.
Introduction
Deep Neural Networks (DNNs) are notoriously resource-intensive, complicating deployment on edge devices like smartphones or wearables. Quantization—reducing the precision of weights and activations from 32-bit floats to ultra-low-bit representations (1 or 2 bits)—offers a path toward significant memory and power savings. However, a substantial "accuracy gap" persists between full-precision and quantized models. While 1-bit (binary) quantization is maximally efficient, it often incurs prohibitive accuracy loss. This research addresses the challenge of finding a "sweet spot" in the bit-width vs. accuracy trade-off by deriving mathematically optimal mappings that minimize quantization error.
Methodology: Optimal 2-Bit Quantization
The authors approach quantization through the lens of low-rank matrix analysis, treating a quantized tensor as a rank-1 approximation ($X \approx v \cdot \text{sign}(X)$). Their core contribution is a unified framework for "scaled binary quantization."
Least Squares 2-Bits (LS-2)
Instead of relying on heuristics, the authors derive exact optimality conditions for 2-bit quantization. They prove that optimal scaling factors ($v_1, v_2$) can be found by solving equations based on the data distribution. Crucially, this method is "foldable," meaning a $k$-bit value can be computed via a sequence of simple sign operations, maintaining compatibility with efficient hardware bitwise XNOR and bit-counting operations.
Minimizing Geometric Information Loss
A key metric for quantization quality is the angle $\angle(x, x^q)$ between the full-precision vector and its quantized counterpart. The proposed least-squares approach significantly minimizes this angle compared to standard greedy methods, particularly in deeper network layers where data distributions become increasingly skewed.
Key Insights: The 2-Bit Pareto Frontier
Empirical analysis reveals that moving from 1-bit to 2-bit quantization provides a disproportionately large jump in accuracy relative to the marginal increase in computational cost. This suggests that 2-bit quantization represents a "Pareto-optimal" point for many practical applications, balancing efficiency and performance.
Experimental Results
The proposed methods were evaluated on ImageNet (ILSVRC12) using a ResNet-18 architecture. The results demonstrate a significant reduction in the accuracy gap:
- 1-Bit (LS): Achieved 58.9% Top-1 accuracy, significantly outperforming the baseline XNor-Net (51.2%).
- 2-Bit (LS): Achieved 63.4% Top-1 accuracy, surpassing other state-of-the-art methods such as LQ-Net (62.6%) and HWGQ-Net (59.6%).
- Ternary (LS): Achieved 62.0% Top-1 accuracy.
- Post-Training Quantization: Even without retraining, the LS-2 method improved Top-1 accuracy by an order of magnitude (5.3% vs 0.3%) compared to greedy 2-bit approaches.
Discussion and Future Work
While the algorithms are designed for bitwise efficiency, future work is needed to measure actual power and latency on specific hardware like FPGAs or ASICs. Additionally, while highly effective for ResNet-18, the performance of these optimal derivations on modern, "quantization-unfriendly" architectures like Vision Transformers (ViTs) remains an open question. Finally, automating the selection of clipping boundaries, currently a manual process, could further streamline the training of ultra-low-bit networks.
Summary updated