Learnable Scaling Parameters for Neural Network Quantization

328 concepts216 papers

The Quantization Dilemma: Clipping vs. Rounding

Quantizing a neural network involves mapping its continuous, high-precision weights and activations onto a discrete, low-bitwidth grid. For uniform quantization, this mapping is governed by a scaling factor $s$ (or step size $\Delta$) that determines the width of the discrete intervals:

$$q = \text{clip}\left(\left\lfloor \frac{x}{s} \right\rceil, q_{\min}, q_{\max}\right)$$

Choosing the right value for $s$ introduces a fundamental tension between two types of error:

  1. Clipping Error: If $s$ is too small, the discrete grid is narrow. Large outlier values are forced to truncate to $q_{\max}$ or $q_{\min}$, destroying valuable signal tail information.
  2. Rounding Error: If $s$ is too large, the grid stretches to encompass the outliers. However, the spacing between steps becomes coarse, causing fine-grained variations in smaller values to be obliterated by rounding.
Small scale 's':  [ x . x . x . x ]  --> Narrow range, low rounding error, high clipping
Large scale 's':  [   x   .   x   .   x   ]  --> Wide range, high rounding error, low clipping

Historically, practitioners relied on static heuristics to determine $s$, such as tracking the absolute maximum value during a calibration phase or minimizing the Kullback-Leibler (KL) divergence between the original and quantized distributions. However, these heuristics are blind to the network's downstream task performance; they optimize for statistical similarity rather than task accuracy.

Making the Grid Trainable

The core insight of learnable quantization scaling is to treat the scaling factor $s$ not as a static hyperparameter, but as a continuous, trainable parameter optimized via backpropagation alongside or in place of the network's weights. By making the quantization grid dynamic, the network can automatically discover the mathematically optimal trade-off between clipping and rounding errors for each layer.

In Trained Ternary Quantization, Chenzhuo Zhu et al. demonstrated that simultaneously deriving gradients for both discrete weight assignments and continuous scaling factors allows a model's representation to evolve fluidly during training. Similarly, Xiandong Zhao et al. introduced Learned Linear Symmetric Quantization (LLSQ), which treats scaling factors as parameters during a retraining phase to adaptively map floating-point values to integer hardware targets.

Why does this work so effectively? Because the scaling factor directly scales the magnitude of the gradients passing backward through the layer. If a layer's output is highly sensitive to precision loss, the gradient with respect to $s$ will naturally force the step size to contract, preserving high fidelity for those features. As noted by Shiwei Li et al. in Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction, this optimization automatically balances quantization noise and value clipping based on the evolving distribution of weights during the training process itself.

Overcoming the Non-Differentiable Barrier

The primary obstacle to learning $s$ via gradient descent is the rounding operator $\lfloor \cdot \rceil$. Rounding is a step function; its derivative is zero everywhere except at half-integers, where it is undefined. This completely blocks standard backpropagation.

To circumvent this, researchers employ specialized gradient approximations or continuous relaxations:

  • The Straight-Through Estimator (STE): The simplest approach passes the upstream gradient directly through the rounding operator unchanged ($\frac{\partial \lfloor x \rceil}{\partial x} \approx 1$). This allows gradients to reach $s$ and update the step size.
  • Continuous Functional Relaxations: Instead of using hard rounding during training, some frameworks replace the step function with smooth, differentiable approximations. For example, the architecture outlined in Quantization Networks bypasses discrete operators entirely during training by modeling quantization as a linear combination of sigmoid functions, using an annealing temperature to gradually harden the function into a discrete grid.
  • Pseudo Quantization Noise: Rather than rounding, Alexandre Défossez et al. proposed in Differentiable Model Compression via Pseudo Quantization Noise to simulate quantization by injecting uniform noise proportional to the step size $s$. Because the noise variance is directly tied to $s$, the model can compute exact, clean gradients to optimize both the scaling factors and the continuous bit-depth allocations of individual weight groups.

Granularity and Structural Adaptation

Weight and activation distributions vary drastically across different layers, channels, or even execution timesteps in modern architectures. A single global scaling factor is too blunt an instrument to preserve accuracy at ultra-low bitwidths.

Consequently, modern learnable scaling frameworks apply optimization at fine-grained granularities:

[ Tensor-wise Scaling ]  --> One scale 's' for the entire weight matrix
[ Group-wise Scaling  ]  --> Distinct scales [s1, s2, s3...] for sub-segments

For instance, Wenjing Ke et al. developed DL-QAT, which introduces group-specific learnable magnitude parameters to align distinct weight segments with the quantization grid, preventing outliers in one group from corrupting the precision of another.

This structural adaptation is particularly vital for generative models. In the context of diffusion processes, Ziwei Wang demonstrated in Towards Accurate Post-Training Quantization for Diffusion Models that treating quantization parameters as a searchable architecture problem over specific timestep groups allows the model to handle the highly dynamic activation ranges characteristic of generation cycles. By shifting quantization from a rigid post-processing step to an active, optimization-driven parameterization, learnable scaling allows low-bit models to retain the nuanced representational capacity of their full-precision counterparts.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 227 papers
  2. Version 1 · 2026-10-04 · 292 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_learnable_quantization_scaling_f25b1413,
  title = {learnable quantization scaling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learnable-quantization-scaling/rdr_f25b14132ab34536a1f17e0397661d57},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_learnable_quantization_scaling_f25b1413_v2,
  title = {learnable quantization scaling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learnable-quantization-scaling/rdr_f25b14132ab34536a1f17e0397661d57/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake