Compressing a language model's weights to 2 or 3 bits makes it cheap to run. The leading one-shot methods decide how to round each weight using a quadratic model of the loss: near the trained weights, the loss is treated as a bowl whose shape is set by second derivatives, the Hessian. At 4 bits this works almost perfectly. At 2 bits the same methods often collapse. Chemists and materials scientists have a long, well-understood version of this failure. The "harmonic approximation" treats atoms as sitting in parabolic wells, and it overestimates defect energies and bond stiffness once displacements get large. Their remedy is not a better parabola. It is to integrate the true forces along the actual displacement path, which splits the total energy change exactly into per-coordinate contributions.
We propose to carry that remedy over to quantization. The move from full-precision to quantized weights is treated as a displacement. The true loss change is computed by integrating the gradient along the straight path between the two weight settings, which takes a few gradient evaluations. The total is then split exactly across layers and weight groups. This accounting does two jobs. First, it tests a hypothesis nobody has checked: that the 2-bit "cliff" is a breakdown of the quadratic model itself, rather than only an outlier or clipping problem. Second, it gives an interaction-aware sensitivity measure for deciding which layers get more bits, under a fixed average bit budget.
Core Claim
The quadratic assumption enters current methods at two separate places, and the literature does not tell them apart. GPTQ and AdaRound minimize each layer's output reconstruction error. That objective is exactly quadratic in the layer's weights, so it is not itself "anharmonic" (non-quadratic). What is assumed is that this layer-local bowl stands in for the model's final loss.
The second-order Taylor expansion of the final loss is a further, separate assumption. The same applies to sensitivity scores built from it, such as CherryQ's Fisher diagonal. We claim the 2-bit failure can be decomposed into three measurable parts: the mismatch between the layer-local proxy and the global second-order model, the gap between that model and the true loss, and the interaction between layers quantized together. The exact tool for measuring all three is a path integral. Writing $w$ for the full-precision weights, $\Delta$ for the quantization displacement, and $g_\ell$ for the gradient block belonging to layer or group $\ell$:
$$\Delta L = L(w+\Delta)-L(w) = \sum_\ell \int_0^1 g_\ell(w+t\Delta)\cdot\Delta_\ell\,dt \equiv \sum_\ell \mathcal{W}_\ell .$$
This gives each group's "work" $\mathcal{W}_\ell$. The work terms sum to the true joint loss change up to quadrature error. They also include the effects of other groups moving at the same time, which isolated probing cannot capture.
The global harmonic prediction along the same direction is $\Delta L_{\text{quad}} = g^\top\Delta + \tfrac12\Delta^\top H\Delta$. It is computed with one Hessian-vector product (a way to get $H\Delta$ without forming $H$). We define the anharmonicity ratio $A_b = \Delta L/\Delta L_{\text{quad}}$ at bit width $b$. The prediction is falsifiable: $A_b \approx 1$ at 4 bits, and $A_b$ departs sharply from 1 at the bit width where perplexity breaks.
If instead $A_b$ stays near 1 at 2 bits, the cliff is a failure of the layer-local proxy, not of quadratic modeling. That would redirect the field toward better global quadratic objectives rather than higher-order ones. Either outcome is informative.
We are explicit that the per-group decomposition is mathematically the same as integrated-gradients attribution. What is new is applying it to the quantization displacement: to separate these three error sources, and to allocate bits using a sensitivity measure that respects conservation and includes interactions.
Why the Physics Analogy Earns Its Place
Two chemistry papers motivate the hypothesis. Cheng and Ceriotti compute defect free energies by thermodynamic integration, an exact integral along a switching path. They show that harmonic and 0 K estimates overestimate vacancy formation free energy in BCC iron by about 60% near melting, and stacking-fault energy in FCC nickel by nearly 300%. Quadratic models do not merely add noise at large amplitude; they are biased in a consistent direction.
SITH supplies our method. It replaces the harmonic spring decomposition used by JEDI with a numerical line integral of forces along a stretching path. That split total energy change across internal coordinates with a conservation error near $10^{-9}$ Hartree. It found harmonic stiffness overestimated at 0.35 versus a true 0.23 Ha/Ų near bond rupture. Our $\mathcal{W}_\ell$ is the direct counterpart of SITH's per-coordinate energy.
Bouchbinder shows that purely quadratic systems have non-interacting modes. Coupling between modes appears only through higher-order terms. This is the analogue of our interaction term: the joint work minus the sum of isolated-quantization losses.
We do not import the heavier machinery from that literature: third- and fourth-order force-constant tensors, or tensor-network dynamics as in Barandun et al.. Explicit cubic tensors over billions of weights are infeasible. The line integral captures all orders along the one direction that matters, at the cost of $K$ gradient evaluations.
Research Program
The first stage is diagnostic. The models are Llama-2 7B and 13B, plus a Qwen 7B model, since Jin et al. report GPTQ perplexity above 123,000 for Qwen at 2 bits. Each is quantized with GPTQ and with QuIP-style incoherence processing at 4, 3, and 2 bits. For every configuration we compute four quantities on a held-out calibration set of 128 sequences:
- the true $\Delta L$;
- the path decomposition $\mathcal{W}_\ell$, using Gauss–Legendre quadrature with $K \in \{4, 8, 16\}$ to check convergence;
- the global quadratic prediction and $A_b$;
- the sum of GPTQ's layer-local objectives, converted to the loss scale through a per-layer Gauss–Newton weighting.
Together these split the total loss into proxy mismatch, anharmonicity, and interaction for each layer and bit width.
We will also test the open question raised by QuIP's random rotations. Rotations leave quadratic forms unchanged but mix higher-order structure. Comparing $A_b$ with incoherence processing on and off tells us whether rotation reduces or amplifies the non-quadratic part.
The second stage is an allocator. Under a fixed average budget (2.5 and 3.0 bits, all side information counted), each layer or 128-column group is assigned a width in {2, 3, 4}. The procedure starts from a uniform 2-bit assignment. It then repeatedly moves bits to the groups whose path-integrated work falls most per added bit, re-quantizing with unchanged GPTQ or LDLQ rounding and recomputing $\mathcal{W}_\ell$ after each round. Because the work is measured on the jointly quantized model, the allocation accounts for interactions between layers.
The rounding algorithm is deliberately left alone. That way, any gain can be attributed to where the bits go, not to a new rounding algorithm.
Baselines
The naive reference is uniform-width GPTQ at the matched average bit count. It shows how much any allocation buys at all. At 3 bits on LLaMA2-7B, CherryQ reports GPTQ at C4 perplexity 8.28.
The strongest existing sensitivity-driven method is CherryQ. It ranks parameters by a Fisher-diagonal estimate of second-order impact and keeps about 0.39% of them in 16-bit. It reports WikiText-2 perplexity 9.55 at 2 bits and C4 perplexity 7.39 at 3 bits on LLaMA2-7B. It is the strongest competitor on our specific question: whether a better sensitivity measure improves where precision goes.
We run it two ways. The first is as published, with its 16-bit parameters counted in the bit budget. The second is as a Fisher-driven group allocator plugged into our procedure, which uses the same {2, 3, 4} formats, calibration data, and number of re-quantization rounds. This second run isolates the scoring rule.
Beating CherryQ means lower WikiText-2 and C4 perplexity at equal counted bits and equal calibration data. It also means that our GSM8K accuracy is not worse. No reported result was found on Lacuna for GSM8K under CherryQ.
The strongest one-shot 2-bit reference for Llama-2 7B in the file is QuIP#, which PV-Tuning reports at WikiText-2 perplexity 6.19. AQLM is reported at 6.64. We apply our allocator on top of incoherence processing to test whether it adds to QuIP-style gains. We treat 6.19 as a ceiling to approach, not one we expect to beat with scalar formats.
PV-Tuning itself reports 5.71. It fine-tunes on about a billion tokens and belongs to a different compute class, so we report it but do not claim to beat it.
The key ablation replaces path-integrated work with isolated endpoint probing. In that version, each group is quantized alone, with everything else at full precision, and the resulting loss increase is measured. The ablation uses the same compute. It isolates the one thing the path decomposition adds: accounting for interactions under joint quantization.
Why not just…?
Why not just measure each layer's loss increase by quantizing it alone? This is the most serious threat. If the interaction term is small, isolated probing gives the same ranking at similar cost, and our allocator reduces to it. The diagnostic stage measures the interaction fraction directly, and the ablation runs this exact alternative. If isolated probing matches us, we will report that the value lies in the diagnosis, not the allocator.
Why not just use the Fisher or Hessian sensitivity, as CherryQ does? CherryQ's score is a local, symmetric, second-order quantity. The chemistry evidence from Cheng and Ceriotti and SITH shows that such scores become biased at large displacement. 2-bit rounding is a large displacement: 4 grid levels, per the file. The quantity $A_b$ measures directly whether that bias exists here. If $A_b \approx 1$, Fisher allocation should tie with ours, and we will say so.
Why not just fine-tune after quantizing, like PV-Tuning? It works: 5.71 perplexity, as reported in PV-Tuning. But it needs 1B tokens, roughly 1.5× fine-tuning cost, and memory for gradients of the dequantized weights. Our method needs a few gradient passes over 128 sequences. The two are complementary, and our allocation can serve as PV-Tuning's starting point. We will test whether it shortens tuning.
Why not just increase GPTQ's damping or re-estimate Hessians sequentially? Both are cheap fixes to the layer-local proxy. They belong in the uniform baseline, tuned on the same calibration set. If a tuned-damping GPTQ closes the gap, the diagnosis points to proxy mismatch rather than anharmonicity, and the decomposition will show which it is.
Proposed Experiments and Decisive Checks
The first check is conservation. $\sum_\ell \mathcal{W}_\ell$ must match the measured $\Delta L$ to within 1% by $K=8$. Otherwise the attribution is not trustworthy and the allocator is dropped.
The second is the cliff test, on Llama-2 7B and Qwen 7B across 4, 3, and 2 bits. The hypothesis is supported if $A_b$ grows sharply exactly where WikiText-2 perplexity explodes, and if the anharmonic share of $\Delta L$ exceeds the proxy-mismatch share at 2 bits. It is refuted if the global quadratic prediction tracks true $\Delta L$ at 2 bits.
The third is the interaction test. It measures what fraction of joint $\Delta L$ is missing from the sum of isolated losses, per bit width.
The fourth is allocation. At 2.5 and 3.0 average bits, we compare perplexity on WikiText-2 and C4, plus GSM8K and MMLU accuracy, across uniform GPTQ, the CherryQ-style allocator, isolated probing, and our allocator. Each is reported with and without incoherence processing, with three calibration seeds.
As an external check that the diagnostic generalizes beyond language models, we will compute $A_b$ for ViT-Base at 3 bits on ImageNet. There Evol-Q reports a jagged loss landscape and a 10.30% top-1 gain over FQ-ViT, and a jagged landscape predicts large departures from quadratic behavior.
References
- OPTQ: Accurate Quantization for Generative Pre-trained Transformers
- Up or Down? Adaptive Rounding for Post-Training Quantization
- Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Models
- Computing the absolute Gibbs free energy in atomistic simulations: Applications to defects in solids
- SITH: A Quantum-Chemical Framework for Predicting Bond Destabilization in Stretched Molecules
- Spatial distribution of thermal energy in equilibrium
- n -Mode Quantized Anharmonic Vibronic Hamiltonians for Matrix Product State Dynamics
- A Comprehensive Evaluation of Quantization Strategies for Large Language Models
- QuIP: 2-Bit Quantization of Large Language Models With Guarantees
- PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression
- Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers