Quantization—the process of mapping high-precision floating-point weights to low-bit integers—is the primary tool for deploying large models on edge devices. Traditionally, this was a manual process: an engineer might decide to use 8 bits for all weights or use heuristics to clip outliers.
However, a growing body of research suggests that quantization should not be a post-processing step, but a learnable part of the model's architecture. By relaxing discrete quantization choices into continuous, differentiable variables, researchers like Hsin-Pai Cheng and BoRui Wu have shown that models can "learn" their own optimal precision levels and scaling factors through standard gradient descent.
Relaxing the Discrete Bit-Width
The hardest part of optimizing quantization is that bit-width (e.g., choosing between 2, 4, or 8 bits) is a discrete choice. You cannot take the derivative of "4 bits" with respect to a loss function.
To solve this, researchers use a "super-net" approach. In Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search, the authors represent a single layer as a parallel set of possible bit-widths. During training, the model computes a weighted sum of these different precisions. By using the Gumbel-Softmax trick—a common tool in HMQ: Hardware Friendly Mixed Precision Quantization Block for CNNs—the model can gradually "harden" its choice, eventually converging on the single most efficient bit-width for that specific layer.
Other methods, such as FracBits and BitPruning, treat bit-width as a continuous value by interpolating between adjacent integer precisions. If the "fractional bit-width" is 3.5, the layer’s output is a 50/50 blend of 3-bit and 4-bit quantization. This allows Milos Nikolic and others to use standard backpropagation to find the exact point where a layer provides the most "accuracy per bit."
Learning the Quantization Grid
Even if the bit-width is fixed, the range of the quantization grid matters. If you quantize a weight distribution from -10 to 10 into 4 bits, but 99% of your weights are between -1 and 1, you waste almost all your precision on empty space.
Instead of using fixed clipping ranges, Adaptive Low-Precision Training (ALPT) and GTC treat the "step size" (the distance between discrete levels) as a trainable parameter. As Shiwei Li demonstrates, the model can then automatically squeeze or stretch its quantization grid to fit the actual distribution of weights and activations.
This is particularly powerful for handling "outliers"—extreme values that often break simple quantization schemes. By making the clipping threshold a learnable parameter, as seen in BASQ, the model learns to ignore a few extreme outliers to achieve better resolution for the majority of its weights.
Scaling to Large Language Models
As models grow to billions of parameters, the cost of "re-training" for quantization becomes prohibitive. Recent work has shifted toward "calibration-driven" optimization. In TEQ: Trainable Equivalent Transformation for Quantization of LLMs, the authors show that training less than 0.1% of a model's parameters—specifically the scaling factors—is enough to recover the accuracy lost when dropping to 3 or 4 bits.
The intuition here is that LLMs are highly sensitive to activation outliers. Methods like SmoothQuant by Guangxuan Xiao mathematically migrate the "difficulty" of quantization from the volatile activations to the more stable weights. By combining this with learnable dequantization scales, as in TesseraQ, researchers can now push models like Llama-2 into the "ultra-low" 2-bit regime while maintaining usable performance.
Why It Works: The Unified Search Space
The success of these methods stems from treating quantization as a joint optimization problem. In the past, we optimized for accuracy, then compressed. Differentiable search frameworks like AutoQNN optimize for accuracy and hardware constraints simultaneously.
When the bit-width and the weights are in the same gradient-based search space, the model can make trade-offs that a human engineer never could: it might decide to use 6 bits for a critical attention head while dropping a redundant feed-forward layer to a mere 2 bits. This automated, fine-grained allocation of "bit budget" is what allows modern quantized models to achieve 30x compression with negligible loss in fidelity.