In the standard deep learning paradigm, the Multiply-Accumulate (MAC) operation is the fundamental building block. However, in hardware terms, digital multipliers are "expensive": they consume significant silicon area and power. As we move toward deploying massive models on edge devices, researchers are increasingly looking to eliminate the "multiplication tax" by replacing these operations with bitwise shifts and additions.
The core intuition behind this shift is simple: in binary representation, multiplying a number by a power of two ($2^k$) is equivalent to shifting its bits $k$ positions to the left. By constraining neural network parameters to powers of two, we can transform computationally heavy matrix multiplications into nearly "free" hardware operations.
The Power-of-Two Representation
The most direct way to eliminate multiplication is to restrict weights or activations to the set $\{ \pm 2^k \mid k \in \mathbb{Z} \}$. As early as 1993, Patrice Y. Simard and Hans Peter Graf demonstrated in Backpropagation without Multiplication that discretizing activations and gradients into powers of two allows for massive networks to be implemented on single silicon chips without losing accuracy.
In modern contexts, this is often framed as a form of non-uniform quantization. Unlike standard linear quantization, which uses equally spaced steps, power-of-two (Po2) quantization provides higher resolution for smaller values. This aligns well with the "bell-shaped" distribution of weights typically found in trained networks, where most values are clustered near zero. J. C. Ott and colleagues showed in Recurrent Neural Networks With Limited Numerical Precision that even complex recurrent models can scale to large tasks like speech recognition using these hardware-efficient bit-shifts.
Bridging the Accuracy Gap with Additive Powers-of-Two
While a single power-of-two term is efficient, it can be too coarse to capture the nuances of high-performance models. To solve this, researchers introduced Additive Powers-of-Two (APoT) quantization.
In Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks, Xin Dong and Yuhang Li proposed representing each value as a sum of multiple power-of-two terms (e.g., $2^{-2} + 2^{-5}$). This approach offers a "best of both worlds" scenario:
- Representational Power: It can approximate any value with high precision, matching the accuracy of full-precision models.
- Hardware Efficiency: It replaces one multiplier with a small number of bit-shifters and a single adder.
Sung-En Chang and others further validated this in Mix and Match, showing that these "sum-of-shift" operations map perfectly to the Look-Up Tables (LUTs) found in FPGAs, significantly reducing the logic footprint compared to standard digital signal processing (DSP) slices.
Quantizing the "Hidden" Costs
Even if the main weights are quantized, models often rely on floating-point operations for "elementwise" layers like Batch Normalization, LayerNorm, and Softmax. These layers can become a bottleneck when the rest of the network is running on high-speed integer hardware.
To achieve truly integer-only inference, researchers use dyadic arithmetic—representing scaling factors as an integer multiplied by a power-of-two exponent. Sehoon Kim and the team behind I-BERT: Integer-only BERT Quantization used this to replace the complex non-linearities in Transformers. By approximating functions like GELU and Softmax with integer-only logic, they achieved 2.4–4.0× speedups on GPUs.
Similarly, Yang Lin and Peiqin Sun developed FQ-ViT, which uses "Power-of-Two Factors" to handle the high-variance distributions in Vision Transformers. This allows the hardware to offload rescaling to bit-shift operators, eliminating the need for floating-point units entirely.
Multiplication-Free Training and Communication
The benefits of power-of-two logic extend beyond inference into the training process itself. In distributed learning, the communication of gradients between workers is a major bottleneck. Samuel Horváth and Marco Canini introduced Natural Compression, which rounds gradients to the nearest power of two. By effectively "ignoring the mantissa" of floating-point numbers, they achieved 3–4× speedups in training time with negligible impact on convergence.
For on-device training, Rui Zhang and Zidong Du proposed Ultra-low Precision Multiplication-free Training. Their architecture replaces multiplications with exponent additions, reducing energy consumption in linear layers by up to 95.8%.
Hardware-Algorithm Co-design
The success of this research direction stems from a tight loop between algorithmic constraints and hardware reality. Whether it is Haotong Qin optimizing Information Retention Networks for bitwise hardware, or Yu Zhang designing MixPE to perform dequantization via shifts after the core computation, the goal remains the same: to align the mathematical structure of neural networks with the most efficient physical operations available in silicon. By treating the power-of-two shift not just as a compression trick, but as a primary primitive for computation, these works pave the way for a new generation of "multiplication-free" AI.