The primary computational bottleneck in modern deep learning is the Multiply-Accumulate (MAC) operation. In standard hardware, digital multipliers are significantly more expensive than adders in terms of silicon area, latency, and power consumption. Power-of-two (PoT) quantization offers a path toward "multiplierless" inference by restricting weights, activations, or scaling factors to discrete powers of two ($2^n$). In this regime, a multiplication operation $x \times 2^n$ is replaced by a simple bit-shift, which is nearly free in hardware.
The Hardware Intuition: Shifts Over Multipliers
In binary arithmetic, multiplying a number by two is equivalent to shifting its bits one position to the left. As Lin et al. and Joshi et al. argue, designing quantization schemes specifically around PoT factors allows complex operations to be offloaded to these bit-shift operators. This hardware-algorithm co-design eliminates the need for dedicated floating-point units (FPUs) or large digital multipliers.
For example, Xing et al. demonstrated that mapping neural network activation outputs to the nearest PoT values can achieve nearly triple the throughput on FPGAs compared to linear quantization. By replacing resource-intensive multipliers with efficient bit-shift logic, specialized hardware can achieve massive energy efficiency gains—often exceeding 60 TOp/s/W in ultra-low-power designs like those proposed by Andri et al..
Scaling Factors and Dyadic Requantization
Even when weights and activations are quantized to standard integers (like INT8), the "scaling factors" used to align these values between layers often remain floating-point numbers. This forces the hardware to perform at least one multiplication per layer or channel.
To solve this, researchers like Marchisio et al. represent scaling factors as dyadic numbers—integers multiplied by a power of two. This allows the hardware to perform requantization using only bit-shifts and integer multiplications, avoiding resource-intensive division units. Yuan et al. further refined this by constraining multiple quantization scaling factors to be related by powers of two, enabling the alignment of disparate data distributions (such as those found in Vision Transformers) using low-latency bit-shifts.
Quantizing the "Unquantizable": Softmax and LayerNorm
Transformers are notoriously difficult to quantize because of non-linear layers like Softmax and LayerNorm, which involve exponentiation, division, and square roots. İslamoğlu et al. and Lin et al. address this by embedding PoT scaling factors directly into these operations.
By approximating the divisor in normalization layers as the nearest power of two, the authors of Sorbet showed that hardware can perform normalization using bit-shifts instead of complex division circuits. This approach allows the softmax module to occupy a negligible fraction of total chip area while maintaining functional parity with floating-point models.
Bridging the Accuracy Gap with Optimization
The main challenge of PoT quantization is the non-uniformity of the grid. Unlike standard linear quantization, where values are spaced evenly (e.g., 0, 1, 2, 3), a PoT grid is sparse and logarithmic (e.g., 1, 2, 4, 8). This can lead to significant quantization errors.
To mitigate this, several optimization strategies have emerged:
- Residual Decomposition: van Baalen et al. decompose quantization into a sequence of residual steps based on powers of two. This allows the model to discover hardware-friendly mixed-precision configurations through a single optimization problem.
- Bin Regularization: Han et al. introduced weight regularization that forces weights to cluster tightly around discrete PoT "bins," ensuring that the model remains accurate even when restricted to these hardware-friendly values.
- Additive Powers-of-Two (APoT): Instead of a single power of two, Ding et al. and Zhao et al. represent weights as a sum of PoT terms (e.g., $2^{-1} + 2^{-3}$). This "shift-and-add" approach provides a much finer-grained representation than a single PoT value while still avoiding general-purpose multipliers.
Deployment on the Edge
The ultimate goal of this research is to enable sophisticated AI on resource-constrained devices. Rusci et al. found that limiting quantization search spaces to PoT bit-widths (2, 4, or 8 bits) not only simplifies hardware bit-packing but also speeds up the automated search for optimal model configurations. Whether it is motor decoding for implantable brain-computer interfaces Xie et al. or real-time eye tracking on wearable devices Zhang et al., PoT quantization provides the mathematical framework to trade expensive computation for cheap, high-speed bit-shifts.