Post-training quantization (PTQ) is the primary tool for squeezing massive neural networks into the limited memory of consumer hardware. Traditionally, this process follows a "divide and conquer" strategy: each layer is quantized in isolation by minimizing a local error metric, such as the Mean Squared Error (MSE) between the full-precision and quantized weights.
However, as we push toward ultra-low bit-widths (2-bit or 3-bit), this local approach hits a "performance cliff." The reason is simple but profound: minimizing error within a single layer does not guarantee the preservation of the final model output. Small, localized discrepancies can accumulate, amplify, or interact across the network's depth, leading to what Hao Yu 0027 and colleagues call a "butterfly effect" where minor early-stage noise results in catastrophic global performance drops.
The Failure of Local Metrics
The fundamental problem with layer-wise optimization is the lack of a theoretical link between local weight distortion and the global decision boundary. As Yiren Zhou et al. argued in Adaptive Quantization for Deep Neural Network, traditional metrics prioritize minimizing local noise rather than final accuracy.
In complex architectures like Transformers, this isolation is particularly damaging. For instance, Zhihang Yuan et al. observed in PTQ4ViT that standard calibration metrics like MSE often fail to predict how quantization in Vision Transformers (ViTs) will affect the end task. A layer might appear stable in isolation but still contribute to a significant cumulative error across the full inference chain.
From Layers to Blocks and Submodules
To bridge this gap, recent research has shifted the optimization objective from individual linear layers to larger functional units. By considering "sub-blocks"—which include residual connections and normalization layers—methods can account for how errors interact with the surrounding architecture.
- Cross-Block Coordination: Xin Ding et al. introduced CBQ, which models long-range dependencies between transformer blocks. This allows the quantizer to see how errors in early stages propagate and amplify, enabling more intelligent rounding decisions.
- Functional Submodules: Yuma Ichikawa and the team behind LPCD argue that we should optimize entire submodules (like the QK or MLP blocks) as single entities. This prevents the "sequential degradation" described by Wen-Pu Cai in LCQ, where errors simply stack up layer by layer.
Attention-Aware Optimization
The attention mechanism is the heart of modern LLMs and ViTs, yet standard PTQ treats its constituent projections (Query, Key, Value) as independent linear layers. Junhan Kim and colleagues developed aespa and BoA to address this. These methods optimize for the reconstruction of the entire attention module's output rather than individual weight matrices.
By deriving "attention-aware" Hessian matrices, these frameworks recognize that some weights are more critical to the attention score than others. Similarly, Ziyi Guan et al. used Hessian-trace sensitivity in APTQ to intelligently mix bit-widths, ensuring that the most influential components of the attention mechanism receive higher precision.
Global Error Compensation
If errors are inevitable at 2-bit precision, can we at least ensure they don't break the model? Geonho Lee et al. proposed RILQ, which uses a model-level discrepancy loss to allow layers to "cooperatively absorb" errors. Instead of trying to fix a high-rank quantization error within a single low-rank layer (which is mathematically difficult), RILQ distributes the compensation burden across the broader network architecture.
This global perspective is also vital for iterative processes like diffusion. Songwei Liu et al. pointed out in Error Propagation Mechanisms and Compensation Strategies for Quantized Diffusion Models that conventional strategies focus on a single execution step. However, in diffusion, errors propagate across a multi-step trajectory. True optimization must consider the dynamic interactions between steps that ultimately determine the quality of the final generated image.
The Path to End-to-End PTQ
The ultimate goal is to move toward true end-to-end optimization that remains computationally feasible. Jiawei Liu et al. demonstrated this with PD-Quant, which replaces local optimization with a "prediction difference" metric. By looking at the final output, the model can identify which quantization errors actually matter for the task at hand.
While full Quantization-Aware Training (QAT) is often too expensive for LLMs, new methods like YAQA by Albert Tseng and Christopher De Sa use efficient Hessian sketches to minimize end-to-end error. These approaches suggest that the future of quantization lies not in making each layer "perfect," but in making the whole model resilient to the noise of compression.