When a large language model is compressed to very low precision (storing each weight in 2 to 4 bits instead of 16), the parts of the network differ greatly in how much damage they can absorb. Mixed-precision methods therefore give more bits to sensitive layers and fewer to robust ones. Today that allocation is chosen by optimizing a stand-in for the true loss: a per-layer error estimate that is summed across layers, as if each layer's damage were independent of the others. Below 3 bits, that independence assumption is exactly what breaks. Errors compound or cancel across layers, and accuracy falls off a cliff, with reasoning abilities collapsing first.
We propose to drop the additive stand-in and search directly over the end-to-end loss. The search uses one structural fact: giving any layer more bits almost never makes the whole model worse. If the end-to-end loss is monotone in this sense, a single expensive evaluation tells us about whole regions of configurations. A configuration that misses the accuracy target rules out every configuration with fewer bits everywhere. A configuration that meets it certifies every configuration with more bits everywhere. This is the same constraint-pushing logic that lets exact search algorithms discard subtrees early. Here it makes searching over measured model quality affordable. As a by-product, the search returns the whole set of near-optimal allocations rather than one, and that set can then be filtered by measured hardware speed.
Core Claim
Mixed-precision allocation is a search for the minimal elements of a monotone feasibility function over a discrete lattice. Let $b \in \{2,3,4,8\}^n$ assign a bit-width to each of $n$ units (linear sublayers). Let $L(b)$ be calibration perplexity, $L_{16}$ the full-precision perplexity, and $\tau$ a tolerance. Define
$$ f(b) = \mathbf{1}\left[\, L(b) \le L_{16} + \tau \,\right]. $$
If $b \le b'$ coordinatewise implies $L(b) \ge L(b')$, then $f$ is a monotone Boolean function on the lattice. Each query of $f$ then resolves an entire cone of configurations. We claim three things. First, end-to-end loss on real LLMs is monotone in this sense up to a small, measurable violation rate. Second, the gap between true loss and the additive per-layer estimate is large exactly in the 2-to-3-bit regime. Third, a monotone-pruned search that uses the additive estimate only to order its queries finds allocations with lower held-out perplexity and better reasoning accuracy than allocations that optimize the estimate, at a fixed budget of a few hundred evaluations.
Relation to Prior Work
Three allocation methods motivate the gap, and each optimizes a stand-in rather than the model. Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming solves an exact integer program over per-layer bit-widths. Its objective, however, is a sum of independently measured layer costs, and its authors report that 2-bit quantization remained unsolved. BAQ derives a closed-form allocation from Hessian-based sensitivities under an equal-loss principle, which again assumes additive, independent noise. The data-free methods NSDS and AlphaQ rank layers or experts by static weight statistics, such as excess kurtosis, spectral structure, and heavy-tailed spectra, and never consult the loss at all.
All of these are exact or principled with respect to a proxy. We are exact with respect to monotonicity of the real loss, and we treat the proxy only as a guide for which configuration to evaluate next. The limitation that QuIP itself acknowledges, that layers are optimized in isolation without accounting for cross-layer error accumulation, is precisely what our end-to-end queries capture.
The search method comes from the monotone-pruning literature. A query-optimal algorithm for finding counterfactuals shows that for monotone functions, minimal "certificates" can be found with query counts that grow only logarithmically in dimension, provided the function's sensitivity is low. This suggests, without guaranteeing, that searching over hundreds of units need not be exponential. Falling Trees shows that a monotonicity constraint can speed up exact search, and that the same pruning can enumerate a Rashomon set, meaning all models within a tolerance of optimal. We borrow that second idea to return all near-optimal allocations. From Approximate to Optimal Solutions: A Case Study of Number Partitioning supplies the general pattern of seeding an exact search with a good heuristic answer and pruning provably dead branches.
One line of work constrains the proposal and could undercut it. Rotation methods (QuIP, UniSVQ) multiply weights by random orthogonal matrices to even out sensitivity. BAQ reports that its gains shrink substantially when combined with QuIP. Allocation of any kind may therefore matter less after rotation, and we test that directly rather than assume it away.
Research Program
We first make queries cheap. Each unit is quantized once at each candidate bit-width using GPTQ-style rounding, with inputs taken from the full-precision model. Any mixed configuration can then be assembled by swapping in precomputed tensors, so evaluating $f(b)$ costs one forward pass over a fixed calibration slice of C4. The cost is that GPTQ's sequential error correction across layers is lost. We measure how much this matters by comparing assembled configurations against sequentially requantized versions on a sample.
Before building any search on monotonicity, we audit it. For random configuration pairs $b \le b'$, we record how often and by how much $L(b') > L(b)$. We do this with and without rotation, and across calibration seeds. We also measure the interaction gap $L(b) - \hat L(b)$, where $\hat L$ is the additive per-unit estimate, as a function of average bit-width. If monotonicity violations are common, the project's exactness claims fail. In that case we report the audit and fall back to treating $f$ as noisily monotone, with repeated queries near the decision boundary.
The search keeps two sets of configurations: those known infeasible, together with everything below them, and those known feasible, together with everything above them. It starts from the BAQ or integer-programming allocation at the target memory budget. It then proposes moves that lower total bits, ordered by the additive estimate. Any proposal already covered by a known cone is resolved without evaluation. Every evaluation adds a new cone. The search ends at a fixed query budget, or when every configuration under the memory target is resolved.
The output is the set of feasible configurations that are minimal within the explored region, plus the set of all evaluated configurations within $\epsilon$ of the best. The second set is our quantized analogue of a Rashomon set. Because our units are whole sublayers, each configuration can run on standard per-tensor low-bit kernels. We measure real latency for the near-optimal set and select the fastest member. This sidesteps the custom-kernel problem that column-wise schemes such as BAQ and CherryQ face.
Baselines
The naive reference is uniform precision at the same average bit-width, with every unit at 2, 3, or 4 bits using the same GPTQ-style rounding. It sets the floor that any allocation must beat.
The strongest allocation competitor is BAQ. It is the most recent principled method in the file, it targets LLMs at 2 to 3 bits, and its closed form is the clearest embodiment of the additive assumption we challenge. We run BAQ's sensitivity estimates on the same calibration slice. Since our units are whole sublayers, we compare in two ways: BAQ's allocation is aggregated to sublayer granularity, and separately run at its native column granularity with its reported metadata cost charged to its bit budget. The only reported number we can anchor to is BAQ's C4 perplexity of 121.31 for OPT-1.3B at 2 bits, against 4028 for its baseline. We therefore include OPT-1.3B and first reproduce that figure. Beating BAQ means lower held-out WikiText-2 and C4 perplexity at matched average bits, and higher GSM8K and ARC-Challenge accuracy, across three seeds.
The exact proxy optimizer is the integer program of Hubara et al., re-solved on our per-unit costs. It isolates whether any gain comes from optimizing the true loss or merely from solving the proxy problem exactly.
For a data-free reference, we run NSDS, which allocates down to 2.6 bits on Llama-2, Llama-3.1, and Qwen2.5. The file reports only relative gains for NSDS, such as 3.07% on PIQA over its zero-shot baseline for Qwen2.5-7B at 3 bits, and no absolute number, so we rerun it ourselves.
For context outside allocation, we cite CherryQ's reported LLaMA2-7B C4 perplexities of 7.39 at 3 bits and 9.55 at 2 bits. CherryQ uses training, so it is a reference point rather than a matched competitor.
The key ablation keeps the same query budget and the same additive ordering but disables cone pruning, so that every proposal is evaluated. A second ablation uses random query ordering. Together they separate the contribution of monotonicity from the contribution of good heuristics.
Why not just…?
Why not just solve the additive problem exactly with an integer program or BAQ's closed form? If cross-layer interaction is small, this is the right answer and our search is wasted effort. The interaction audit decides the question. If $L(b) - \hat L(b)$ stays small and consistent in sign below 3 bits, we concede the point. The file's evidence points the other way: QuIP acknowledges ignoring cross-layer accumulation, and the 2-bit cliff in Evaluating Quantized Large Language Models is a nonlinear, model-level effect. That is suggestive, not proof.
Why not just run greedy or evolutionary search on the true loss? This is the real threat. Such searches also optimize end-to-end loss, and with enough queries they may match us. Our comparison holds the number of evaluations fixed at 100, 300, and 1000 for greedy bit-dropping, for a seeded evolutionary search, and for our method. The claim survives only if cone pruning reaches a better configuration, or a larger certified near-optimal set, at equal query budgets. If greedy matches us at every budget, the contribution reduces to the monotonicity audit and the Rashomon analysis.
Why not just rotate the weights and use uniform bits? QuIP reaches 6.33 WikiText-2 perplexity on Llama 2 70B at 2 bits without any mixed precision, and BAQ reports that rotation erases much of its own advantage. We therefore run every method on both rotated and unrotated models. If allocation adds nothing after rotation, we report that. We still compare the rotated-uniform model against the fastest allocation in our near-optimal set, since QuIP reports kernels about 1.5x slower than OPTQ, which leaves room for a rotation-free allocation that runs faster at similar perplexity.
Proposed Experiments
The models are OPT-1.3B (to anchor against BAQ), LLaMA2-7B and LLaMA2-13B (to anchor against CherryQ and CRVQ results), and Llama-3-8B. Average budgets are 2.5, 3.0, and 3.5 bits over the choices {2, 3, 4, 8}. Units are the seven linear sublayers per transformer block. Calibration uses C4. Evaluation uses held-out WikiText-2 and C4 perplexity, zero-shot ARC-Easy, ARC-Challenge, and PIQA, and GSM8K, because multi-step math collapses first and is where cross-layer interaction should matter most. We also measure tokens per second on an A100 for each configuration in the near-optimal set.
Four checks decide the outcome. The first is the monotonicity violation rate, which must be low enough to justify exact pruning. The second is the interaction gap as a function of bit-width, which must grow below 3 bits for the premise to hold. The third is perplexity and GSM8K accuracy at a fixed query budget against greedy search, BAQ, and the integer program. The fourth is whether the near-optimal set varies meaningfully in measured latency, which tells us whether returning many allocations instead of one has practical value. Each check can falsify part of the proposal independently, and the first two are informative even if the search itself does not win.
References
- Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming
- BAQ: Efficient Bit Allocation Quantization for Large Language Models
- Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity
- AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
- QuIP: 2-Bit Quantization of Large Language Models With Guarantees
- A query-optimal algorithm for finding counterfactuals
- Falling Trees: A Model Class for Interpretable Risk Prioritization
- From Approximate to Optimal Solutions: A Case Study of Number Partitioning
- UniSVQ: 2-bit Unified Scalar-Vector Quantization
- Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Models
- Evaluating Quantized Large Language Models