When a trained model is compressed to 4 bits it barely changes. At 2 bits it often falls apart: perplexity explodes and reasoning collapses. Nobody can say in advance where that cliff sits. Practitioners find it by trial and error, sweeping bit-widths, clipping thresholds and rotations. This note proposes a first exact theory of the cliff for a setting simple enough to solve and realistic enough to matter: a linear predictor (a regression model, a linear classifier, or the last layer of a network retrained on frozen features) that is fit on $n$ samples in $p$ dimensions and then rounded to a $b$-bit grid. The tool is the Convex Gaussian Min-Max Theorem (CGMT). It is a comparison argument that replaces a hard random optimization problem with a few deterministic scalar equations. As a by-product, it predicts the statistical distribution of every coordinate of the learned weight vector.
The central observation is that rounding is applied after training, coordinate by coordinate. Knowing the distribution of the coordinates is therefore enough to compute exactly how much test accuracy rounding destroys. That calculation covers any bit-width, clipping threshold or regularization strength, and it can be done with or without a random rotation of the weights before rounding. Without any network simulation, it yields the location of the cliff as a function of the sample-to-dimension ratio, the noise level, and the shape of the true signal. It also tells us whether the regularization chosen for full precision is still the right choice once the model will be quantized.
Core Claim
Earlier CGMT analyses of regularized regression and classification show that, in the proportional limit $n/p \to \delta$, the coordinates of a convex regularized estimator behave like a known scalar denoiser applied to "truth plus Gaussian noise" (Regularized Linear Regression: A Precise Analysis of the Estimation Error; The Impact of Regularization on High-dimensional Logistic Regression). For ridge or Lasso regression with isotropic Gaussian features, write $W^\star$ for a coordinate of the true weights, $H$ for an independent standard normal, and $\eta_\lambda$ for the proximal map of the regularizer at strength $\lambda$. The scalars $(\tau, \kappa)$ solve the usual CGMT fixed-point equations. Then a learned coordinate is distributed as $\eta_\lambda(W^\star + \tau H;\kappa)$.
For a quantizer $Q_{b,A}$ with $b$ bits, clipping level $A$ and uniform grid, the test error of the rounded model is
$$ \mathcal{E}(b,A,\lambda,\delta) = \mathbb{E}\big[(Q_{b,A}(\eta_\lambda(W^\star+\tau H;\kappa)) - W^\star)^2\big] + \sigma^2 , $$
where $\sigma^2$ is the label noise. This form follows because, with isotropic test features, the error of any fixed weight vector depends only on its squared distance to the truth. For classification the same reasoning gives the error through two overlaps, $\mathbb{E}[Q(\hat W)W^\star]$ and $\mathbb{E}[Q(\hat W)^2]$.
The rounding map is discontinuous, so the standard empirical-distribution convergence (stated for Lipschitz test functions) does not apply directly. The discontinuities form a null set under the continuous limit law, though, so a smoothing argument should close the gap. Making this rigorous is the first technical deliverable.
The quantity $\mathcal{E}$ is a one- or two-dimensional integral. From it we can read off three things. The first is the cliff bit-width: the smallest $b$ at which quantized error stays within a fixed factor of the full-precision error. The second is the optimal clipping level $A^\star(b,\delta)$. The third is the quantization-aware regularization $\lambda^\star(b)$, which we expect to differ from the full-precision optimum because stronger shrinkage narrows the coordinate distribution the grid has to cover.
Research Program
The first part derives and proves the formula above for ridge, Lasso, and $\ell_2$-regularized logistic regression, and computes the three derived quantities. A sharp qualitative prediction should follow. The textbook additive-noise model treats rounding error as independent uniform noise of variance $\Delta^2/12$, where $\Delta$ is the grid spacing. That model should be accurate at 4 or more bits and should fail exactly in the 2–3 bit range, where the grid has too few levels to resolve the spike of near-zero coordinates that Lasso produces, or the heavy tail of a sparse signal.
The second part analyzes rotation before rounding, the mechanism behind QuIP, QuaRot and SpinQuant. These papers rotate weights (and activations) by an orthogonal matrix so that outliers are spread across coordinates, round, and rotate back. SpinQuant reports that this drives activation kurtosis toward the Gaussian value of 3, and the research file notes that no high-dimensional justification of this "Gaussianization" exists. In our setting it can be derived directly. After a random orthogonal rotation, each pair of rotated coordinates of the learned and true weights becomes jointly Gaussian, with covariance given by the CGMT overlaps. The rotated-then-rounded error is then a two-dimensional Gaussian integral. That yields an exact criterion for when rotation helps, which we expect for sparse or heavy-tailed true signals and for Lasso. It also shows when rotation is neutral or slightly harmful, which we expect when coordinates are already near-Gaussian, as with ridge on dense signals. In a spiked-covariance extension, the criterion becomes a statement about the incoherence condition that QuIP identifies.
The third part treats clipping during training as a convex constraint. If training includes a box constraint $|w_i|\le A$, the problem stays convex and CGMT still applies. This is the learning-theory counterpart of the box-constrained one-bit precoder in Asymptotic Analysis of One-bit Quantized Box-Constrained Precoding in Large-Scale Multi-User Systems. That paper is the closest existing work, and we reuse its key insight: a box constraint shapes the distribution entering the quantizer and acts as a noise-control knob. Our setting differs in four ways. It is a statistical learning problem with generalization error as the target rather than a wireless signal-to-distortion ratio. It treats multi-bit uniform grids rather than sign quantization. It covers regularized and logistic estimators. And it studies the interaction with rotation and with regularization tuning. The Babai lattice view of GPTQ (The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm) also blames forced box clipping for catastrophic loss, but its bounds are worst-case. Our analysis should say how much clipping costs on average, as a function of $\delta$.
The fourth part transfers the theory to real models through retraining the last layer on frozen features. Thumb on the Scale applied CGMT to this setting and matched ResNet34 experiments on CelebA and CIFAR10 by plugging in a PCA-based "effective dimension". We adopt the same bridge. The theory is fed the effective $\delta$ and the empirical spectrum of the features, and its predictions are compared with what happens when the retrained head is quantized.
Baselines
The naive reference point is round-to-nearest with min-max scaling, meaning the clipping level equals the largest coordinate. The model is trained at the full-precision-optimal $\lambda$ and no rotation is used. Every proposed knob setting must beat it on measured test error under the same data and training run.
The strongest competing predictor is the additive uniform-noise model, which forecasts quantized error as full-precision error plus $p\Delta^2/12$ in suitable units. It is the standard back-of-envelope tool and, like Bussgang-style linearization in the precoding literature, it is cheap and widely trusted. We will compare predicted against measured error across $b\in\{1,\dots,8\}$, $\delta$ from 0.25 to 4, and several signal priors. Beating it means a substantially smaller prediction error at 2–3 bits while matching it at 4 or more bits. There is no reported result on Lacuna for the accuracy of the additive-noise model on quantized learned estimators, so this comparison is new.
The strongest competing method for choosing clipping is a grid search over $A$ on held-out data with the same validation budget. Beating it means our closed-form $A^\star$ lands within the search's noise band at zero search cost, and does better when validation data is small. For rotation we will use a random Hadamard rotation as in QuaRot, under identical bits and data.
For LLM context only, the strongest reported low-bit numbers are QuIP's 2-bit Llama 2 70B WikiText-2 perplexity of 6.33 against 123.91 for OPTQ (QuIP), and CherryQ's 2-bit LLaMA2-7B C4 perplexity of 9.55 (Cherry on Top). We do not claim to beat these. The LLM check below tests transfer of predicted knobs, not overall performance.
The ablation that isolates the key ingredient replaces the CGMT-predicted coordinate law with a Gaussian of matched variance. This is exactly what the additive-noise and "everything is Gaussian after rotation" intuitions assume. If that substitution predicts as well as the full law, the CGMT machinery is not doing real work.
Why not just…?
Why not just simulate? For linear models simulation is cheap, and a reviewer could argue that sweeps make theory unnecessary. That is a real objection for any single setting. Simulation cannot say why the cliff moves with $\delta$ or with sparsity. It also cannot cheaply cover the joint space of $(b, A, \lambda, \delta, \text{prior}, \text{rotation})$, and it gives no closed-form rule to carry into frozen-feature networks. We will report how many simulated configurations are needed to recover the theory's $A^\star$ and $\lambda^\star$ curves to equal accuracy. If that number is small, the practical case weakens to explanation alone, and we will say so.
Why not just use the additive uniform-noise model? It is what most engineers use, and at 4 or more bits we expect it to be nearly as accurate. Its known failure in the related precoding problem, where Bussgang linearization breaks once a box constraint distorts the input distribution (Ma et al.), suggests it will fail precisely at low bit-widths. The predictor comparison in the Baselines section settles this directly. If the additive model holds down to 2 bits, the central claim is wrong.
Why not just apply lattice or Babai bounds? They hold for any input, but they are worst-case and are stated layer by layer (GPTQ as Babai; Lattice equivalence proof). They cannot give a sharp error curve or a cliff location. We will plot the bound next to the measured error to show the gap.
Proposed Experiments
The synthetic validation uses Gaussian designs with $p$ from 200 to 4000, ridge, Lasso and logistic estimators, Gaussian, Bernoulli-Gaussian and Student-t true signals, and bit-widths 1–8 with and without Hadamard rotation. The decisive check is that measured quantized test error matches $\mathcal{E}$ within finite-size fluctuations shrinking roughly as $1/\sqrt{p}$, the usual CGMT concentration rate. The predicted cliff bit-width, $A^\star$ and $\lambda^\star$ must also match empirical optima. A second decisive check is the sign of the rotation benefit: rotation should help for sparse and heavy-tailed signals and give no gain for dense Gaussian ones.
The frozen-feature transfer follows the Thumb-on-the-Scale protocol: ResNet34 features on CelebA and CIFAR10, PCA effective dimension, and a quantized retrained head. Success means the theory, using only the effective $\delta$ and spectrum, ranks clipping levels and $\lambda$ values correctly and places the accuracy cliff within one bit.
A modest LLM probe applies the predicted $A^\star(b)$ rule to round-to-nearest quantization of LLaMA-2-7B linear layers, with and without Hadamard rotation, using the empirical weight distribution in place of the CGMT law. We measure WikiText-2 and C4 perplexity against min-max scaling. This tests whether the low-bit clipping picture carries over, not whether we can beat QuIP or CherryQ. A null result here would confine the contribution to the linear and last-layer regime, and that limit would be reported as such.
References
- Regularized Linear Regression: A Precise Analysis of the Estimation Error
- The Impact of Regularization on High-dimensional Logistic Regression
- QuIP: 2-Bit Quantization of Large Language Models With Guarantees
- QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
- SpinQuant: LLM Quantization with Learned Rotations
- Asymptotic Analysis of One-bit Quantized Box-Constrained Precoding in Large-Scale Multi-User Systems
- The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm
- Thumb on the Scale: Optimal Loss Weighting in Last Layer Retraining
- Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Models
- The Lattice Geometry of Neural Network Quantization: A Short Equivalence Proof of GPTQ and Babai's Algorithm