Non-Uniform Quantization via Data-Driven Clustering

This direction is merged into another direction: distribution-aware quantization. Its last version stays readable and citable here.

Version 1 of 123 papers29 conceptsfrozen 2026-10-04

Most machine learning hardware operates on the assumption that numbers should be represented on a uniform grid. In this "linear" world, the distance between 0 and 1 is the same as the distance between 100 and 101. However, neural network parameters—the weights and activations that drive modern AI—are rarely distributed uniformly. They typically cluster around zero in bell-shaped or heavy-tailed distributions.

Recent research has converged on a powerful alternative: non-uniform quantization. Instead of forcing data onto a rigid, equal-interval grid, these methods use clustering algorithms to discover the "natural" centers of the data, placing more representational power where the data is densest.

The Limits of the Uniform Grid

Standard quantization maps high-precision values to a fixed set of integers. While computationally simple, this approach suffers from a "performance tax" because it treats all regions of the value space as equally important. As Annie Cherkaev and colleagues argue in Learning In Practice: Reasoning About Quantization, strategic placement of discrete points based on data density can significantly outperform uniform grids. When the "margin" of the data is larger than the quantization error, even simple algorithms like the Perceptron can function perfectly in discrete space.

The inefficiency of uniform grids becomes catastrophic at ultra-low bit-widths (1 or 2 bits). If you only have four possible values to represent a weight, placing those values at -2, -1, 1, and 2 is wasteful if 90% of your weights are between -0.5 and 0.5.

Quantization as a Clustering Problem

The core insight shared across this research direction is that quantization is essentially a 1D clustering problem. By using K-means clustering, researchers can identify optimal "centroids" that minimize the total distance between the original values and their quantized representations.

Yuhui Xu pioneered this with Single-Level and Multi-Level Quantization (SLQ/MLQ), demonstrating in Deep Neural Network Compression With Single and Multiple Level Quantization that data-driven clustering allows 5-bit models to sometimes outperform their full-precision counterparts. This happens because the quantization levels adapt to the specific parameter characteristics of each layer.

For Large Language Models (LLMs), this adaptive approach is even more critical. Runsheng Bai and the team behind SKIM (SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization) use K-means to identify centroids that better represent the weight distribution, reducing the noise floor without requiring extra memory. Similarly, Bei Liu introduced CLAQ (CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs), which applies this clustering at the column level of weight matrices to maintain accuracy at 2-3 bits.

Capturing Information Density and Outliers

Why does clustering work so much better than rounding? It comes down to two factors: resolution and outliers.

  1. Resolution in Dense Regions: In Ultra-Low Bit Post-Training Quantization of Large Speech Models, Bei Liu explains that non-linear quantization provides higher resolution in dense regions of the distribution. By concentrating "codebook" entries where the weights are most frequent, the model preserves more information.
  2. Handling Outliers: Standard uniform quantization often has to choose between "clipping" large outliers or "blurring" the dense central region to accommodate them. Constantin Eichenberg and Douglas Orr show in 1-Bit Wonder that non-linear K-means quantization effectively captures these outliers, which are often critical for model performance, without sacrificing the precision of the smaller values.

This principle extends beyond weights. Yanxin Peng and colleagues developed BitSnap (BitSnap: Checkpoint Sparsification and Quantization in LLM Training), which uses cluster-based quantization to compress 32-bit optimizer states down to 8 bits by allocating more precision to the most frequent value ranges.

Scaling to the 1-Bit Regime

The most provocative application of this idea is the "1-Bit Wonder" hypothesis. Sohir Maskey and Constantin Eichenberg demonstrate in 1-Bit Wonder: Improving QAT Performance in the Low-Bit Regime through K-Means Quantization that for a fixed memory budget, it is better to train a massive model with 1-bit non-linear weights than a smaller model with high-precision weights.

By using K-means during Quantization-Aware Training (QAT), they successfully scaled a 31B parameter model into the memory footprint of a 4B 16-bit model, achieving vastly superior performance. The non-linear centroids provide the stability needed to train at such extreme compression levels.

Hardware and Domain-Specific Adaptation

While non-uniform quantization is mathematically superior, it is traditionally harder to implement in hardware than simple bit-shifting. However, Jannis Schönleber and the team behind Stella Nera (Stella Nera: A Differentiable Maddness-Based Hardware Accelerator) have shown that specialized accelerators can use hash-based lookups to make non-linear quantization 25x more energy-efficient than conventional matrix multiplication.

The concept is also finding success in specialized domains:

By treating quantization as an act of learning the data's distribution rather than a rigid rounding rule, these researchers have unlocked a path toward massive models that can run on tiny devices.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_non_uniform_quantization_via_data_driven_692cbebe,
  title = {Non-Uniform Quantization via Data-Driven Clustering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-uniform-quantization-via-data-driven-clustering/rdr_692cbebe1a3e41c29d2e2192c619ea83},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_non_uniform_quantization_via_data_driven_692cbebe_v1,
  title = {Non-Uniform Quantization via Data-Driven Clustering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-uniform-quantization-via-data-driven-clustering/rdr_692cbebe1a3e41c29d2e2192c619ea83/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}