Quantization is the process of mapping high-precision floating-point numbers (like FP32) to a small set of discrete integers (like INT4 or INT8). In an ideal world, neural network weights and activations would be perfectly centered at zero and follow a bell curve. If that were true, a simple symmetric quantizer—which spreads its available "buckets" evenly around zero—would be perfectly efficient.
However, modern architectures like Vision Transformers (ViTs) and Large Language Models (LLMs) break this assumption. Their activations are often strictly positive (Softmax), heavily biased (LayerNorm), or have long, lopsided tails (GELU). When we force these asymmetric distributions into a symmetric grid, we waste half of our representational power on empty space.
The Efficiency Gap: Why Symmetry Fails
The intuition behind asymmetric quantization is simple: don't waste bits on values that never occur.
Consider a Softmax output, which only produces values between 0 and 1. If we use a standard 8-bit symmetric quantizer, we map the range $[-1, 1]$ to integers $[-128, 127]$. Because the Softmax never produces negative values, the entire range from $-128$ to $-1$ is left unused. As Rong Tian and colleagues point out in their work on SAMP, this "range mismatch" effectively throws away one full bit of precision, leading to significant accuracy drops in deep Transformer architectures.
To solve this, researchers have moved toward affine quantization, which introduces a "zero-point" (an offset) that allows the quantization grid to shift. By centering the grid on the actual data, we can double the resolution for the values that actually matter.
Shifting the Grid: Learnable Offsets and Centering
The most direct way to handle asymmetry is to make the quantization grid flexible. In LSQ+, Yash Bhalgat introduced a learnable offset parameter that allows the grid to migrate during training. This is particularly vital for activation functions like Swish or PReLU, which have small negative tails but large positive ranges. Instead of clipping the tail or wasting bits on it, the grid shifts to accommodate both.
In the realm of post-training quantization (PTQ), where we cannot retrain the model, researchers use "migration" techniques. Xiuying Wei and the team behind Outlier Suppression+ found that LLM activations often contain "outlier channels" that are heavily biased away from zero. By mathematically shifting these distributions to be zero-centered and absorbing that shift into the model's weights, they effectively halved the required representation range, making 4-bit quantization viable for models like BERT.
Handling the "Twin" Nature of Transformers
Vision Transformers present a unique challenge because they contain multiple distinct types of distributions in a single block. Zhihang Yuan and the authors of PTQ4ViT identified that functions like GELU produce "twin" distributions: a high-density region near zero and a sparse region of high-magnitude outliers.
Their solution, Twin Uniform Quantization, uses two overlapping ranges with different scaling factors. This allows the model to maintain high precision for the dense "body" of the data while still capturing the critical "outliers" that drive the model's attention mechanisms. Similarly, Xianglong Yan in PT2-LLM applies this logic to ternary quantization (representing weights as just -1, 0, or +1). By using an asymmetric grid with a learned offset, they can represent biased weight distributions that a standard $\{-1, 0, 1\}$ grid would mangle.
The Hardware Paradox
If asymmetric quantization is so much more accurate, why isn't it the universal standard? The answer lies in hardware.
Symmetric quantization is "hardware-friendly" because it simplifies the math of matrix multiplication. As noted in the Panacea accelerator research, symmetric grids allow hardware to easily skip zero-valued weights to save energy. Asymmetric quantization, with its non-zero offsets, makes this "zero-skipping" much harder because a "zero" in the quantized space doesn't actually mean zero in the mathematical space.
To bridge this gap, researchers like Ting Cao in AFPQ have proposed using separate scaling factors for positive and negative values without a zero-point shift. This captures the asymmetry of the distribution while keeping the "zero" value fixed at zero, providing a middle ground between mathematical precision and hardware efficiency.
Beyond Information Maximization
Interestingly, sometimes asymmetry is useful even when it's not strictly "optimal" from an information-theory perspective. Hyungjun Kim demonstrated that for Binary Neural Networks (BNNs), intentionally creating an unbalanced activation distribution—where more bits are assigned to +1 than -1—actually improves accuracy. By shifting the binarization threshold, the network can better mimic the sparse, skewed behavior of a full-precision ReLU, proving that the goal of quantization isn't just to pack information, but to preserve the discriminative power of the original features.