Post-training quantization means shrinking a trained language model's weights to 2–4 bits without retraining it. The best methods now treat this as optimization: they walk through the network one transformer block at a time and run gradient descent on a few learnable quantization parameters. The goal is to make each quantized block reproduce what the full-precision block would have output. Every block gets the same fixed number of epochs. That is costly: OmniQuant reports roughly 16 hours on one A100 to quantize LLaMA-2-70B. It is also blind. A block whose small leftover error barely reaches the model's final output gets as much effort as a block whose error is amplified by everything after it. And no block is ever revisited once later blocks have been quantized.
Classical numerical optimization has a well-developed answer to this kind of waste. When an outer algorithm calls an inner solver, solve the inner problem only as precisely as the outer algorithm's current progress warrants, and tighten that precision as the outer problem nears a solution. This rule is called a forcing sequence. We propose to import that discipline into blockwise quantization. The whole-network output mismatch becomes the outer residual. Each block's reconstruction problem becomes an inexactly solved inner subproblem. Its stopping tolerance is set by how strongly that block's error propagates to the output. The claim is simple: under a fixed calibration budget, this reallocation reaches lower perplexity than uniform schedules, and it matches their quality in a fraction of the time.
Core Claim
Blockwise quantization is an inexact block coordinate descent on the end-to-end discrepancy between the quantized and full-precision models. It should be run like one. Let $D_t$ be the output discrepancy after the $t$-th pass over the blocks, measured as the KL divergence between quantized and full-precision next-token distributions on the calibration set. Let $L_k$ be block $k$'s reconstruction loss, the quantity OmniQuant already minimizes. Let $a_k$ be an estimated amplification factor: how much a unit perturbation of block $k$'s output changes $D$, obtained cheaply from a few random-direction perturbations propagated forward. Inner gradient steps on block $k$ stop when their expected contribution to the outer objective falls below a forcing threshold:
$$a_k \, \Delta L_k \;<\; \eta_t \, \frac{D_t}{K},$$
Here $\Delta L_k$ is the recent per-step decrease in $L_k$, $K$ is the number of blocks, and $\eta_t \in (0,1)$ is a forcing sequence that shrinks across passes. Early passes are loose and cheap. Later passes revisit blocks, with tighter tolerances, whose inputs have changed because their neighbors were re-quantized.
What is new is not stopping early. It is that the stopping rule is coupled to the downstream effect of each block's error and to global progress, and that blocks are revisited.
Positioning Against Prior Work
The method comes from the inexact second-order literature. Holt's inexact Levenberg–Marquardt method stops the inner linear solve once its residual falls below a multiple of the current gradient magnitude. Proximal Newton-type methods and the Shamanskii-type extension show that forcing sequences preserve fast convergence while avoiding oversolving. All three assume convexity, which we do not have.
The closest theoretical shelter for our nonconvex setting is inexact block successive convex approximation. It proves that stopping block subproblems after finitely many inner iterations still yields subsequence convergence to stationary points. We borrow its block structure and accept its weaker guarantee rather than claim rates. MAID motivates the idea of adapting lower-level tolerances to upper-level progress with a verifiable line search. Its setting is bilevel hyperparameter learning on denoising and logistic regression. Ours is a sequential chain of nonconvex block problems with no hypergradient.
On the compression side, OmniQuant supplies the inner problem: learnable weight clipping and equivalent channel scaling trained on 128 calibration samples. It uses fixed epochs per block and a single forward sweep. AdaRound shows that layer-wise optimization beats heuristic rounding, from about 24% to 68.7% top-1 on 4-bit ResNet-18. It likewise uses a static schedule. ARMOR runs block coordinate descent with predetermined iteration counts for 2:4 sparsity. It is the natural second testbed.
Two works motivate why per-block effort should differ. LLMCBench finds that quantization degrades multi-step reasoning more than factual recall. That suggests the generic per-block error proxy misallocates effort. Work on bypassable attention layers shows blocks differ sharply in importance.
Research Program
We will build the scheduler as a wrapper around the public OmniQuant optimization loop, leaving its parameterization untouched. Any gain therefore comes from allocation, not from a new quantizer.
Three components need study. The first is how to estimate $a_k$ cheaply and stably. We will compare random-perturbation forward propagation, a Jacobian–vector-product estimate, and a trivial constant. We will also check how quickly the estimates go stale as blocks change.
The second is the forcing sequence $\eta_t$. We will compare geometric decay against a residual-driven rule in the spirit of the adaptive $\gamma=1$ rule of PSOPT.
The third is whether revisiting blocks helps or causes oscillation. Revisits are the part the single-sweep literature has never tested, and BSCA guarantees only stationarity of subsequences.
As a secondary transfer test, the same scheduler will control the continuous inner updates in ARMOR's block coordinate descent.
Baselines
The naive reference points are round-to-nearest and GPTQ, which involve no calibration-time optimization. They show how much of the gap any optimization closes. On LLaMA-13B at 2-bit weights with 16-bit activations (W2A16), the file lists GPTQ at WikiText-2 perplexity 3832 (OmniQuant).
The strongest existing method is OmniQuant with its default fixed schedule. It is reported at perplexity 13.21 for LLaMA-13B W2A16 in the file's WikiText-2 listing, and at +4.99% to +11.80% average zero-shot accuracy over SmoothQuant at 4-bit weights and 4-bit activations (W4A4) (OmniQuant). It is the strongest competitor because our method shares its exact parameterization. We will rerun it ourselves on the same calibration samples, seeds, and hardware. We will run it both at its default epoch count and at a compute-matched budget equal to our total inner steps. Beating it means one of two things: lower WikiText-2 and C4 perplexity at equal GPU-hours, or equal perplexity (within seed noise) at no more than half the GPU-hours.
The file reports no absolute ARMOR perplexities, only a "nearly 50%" gap reduction on Llama-2-13B. For the ARMOR transfer we therefore compare against our own rerun of ARMOR's fixed iteration counts.
Two ablations isolate the key ingredient. The first keeps adaptive stopping and revisits but sets $a_k$ to a constant. This tests whether the propagation weighting matters. The second keeps the full stopping rule but performs a single sweep with no revisits. This tests whether revisiting matters.
Why not just…?
Why not just stop each block when its loss plateaus? This is the most serious threat. Plateau-based early stopping also saves compute and needs no theory. Our answer is empirical. Plateau stopping asks whether the block's loss is still falling. Our rule asks whether further falling still matters downstream, given the error already present. These differ exactly when a block has a long slow tail of improvement but low amplification, or a fast plateau but high amplification. We will run plateau stopping, with a tuned patience, as a baseline at matched compute. If it matches us, the propagation coupling is not worth it, and we will report that.
Why not just allocate more epochs to sensitive blocks using a fixed heuristic, such as first and last blocks or a Hessian-trace score? A static allocation captures part of the idea. However, it cannot respond to how upstream re-quantization changes a block's inputs, and it cannot revisit. The constant-$a_k$ and no-revisit ablations bracket this alternative. We will also run an explicit sensitivity-weighted fixed allocation computed once from $a_k$ at initialization. If it does as well as the adaptive version, the contribution shrinks to "use propagation-based sensitivity," which is still useful but smaller.
Why not just fine-tune end-to-end, or heal afterwards with a compensation method? End-to-end tuning of quantization parameters is what blockwise methods avoid for memory reasons. EoRA and RestoreLCC note that gradient fine-tuning of large models is costly and that PEFT failed to converge at 60% sparsity. Compensation methods add parameters or kernels after the fact, and they are complementary: EoRA can be applied on top of either schedule. We will report an EoRA-on-top result for both OmniQuant and our method to confirm that the gains stack.
Proposed Experiments
We will quantize LLaMA-7B and LLaMA-13B at W2A16, 3-bit weights with 16-bit activations (W3A16), and W4A4, using 128 C4 calibration samples, matching OmniQuant's protocol. Evaluation covers perplexity on WikiText-2, C4, and PTB, and zero-shot accuracy on BoolQ, PIQA, HellaSwag, Winogrande, and ARC-Challenge. We add GSM8K because LLMCBench flags multi-step reasoning as what quantization damages most. A generic block loss may miss that damage, while an output-KL outer residual might catch it. The primary plot is perplexity against calibration GPU-hours, drawn as a full curve for each method rather than a single point.
Three checks decide the question. The first is whether our curve dominates both default and compute-matched OmniQuant at W2A16, the regime where remaining error is largest and allocation should matter most. The second is whether it beats plateau early stopping at matched compute. If it does not, the core claim fails. The third is whether revisits lower final perplexity or merely redistribute it.
We expect gains to shrink at W4A16, where every block is easy. Such shrinkage would be consistent with the forcing-sequence view, since loose tolerances suffice when the outer residual is already small. A time-to-quality speedup on LLaMA-2-70B, measured against the reported 16-hour budget, is the practical payoff. The ARMOR 2:4 transfer, scored on WikiText-2 perplexity and GPQA, tests whether the rule generalizes beyond quantization.
References
- OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
- An inexact Levenberg-Marquardt method for large sparse nonlinear least squres
- Proximal Newton-Type Methods for Minimizing Composite Functions
- New Proximal Newton-Type Methods for Convex Optimization
- Inexact Block Coordinate Descent Algorithms for Nonsmooth Nonconvex Optimization
- An adaptively inexact first-order method for bilevel optimization with application to hyperparameter learning
- Up or Down? Adaptive Rounding for Post-Training Quantization
- ARMOR: High-Performance Semi-Structured Pruning via Adaptive Matrix Factorization
- LLMCBench: Benchmarking Large Language Model Compression for Efficient Deployment
- What Matters in Transformers? Not All Attention is Needed
- EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation
- Restoring Pruned Large Language Models via Lost Component Compensation