Mitigating Error Propagation in Deep Model Compression

599 concepts372 papers2 proposals

The standard recipe for compressing a massive neural network seems straightforward: break the network down into its constituent parts, compress each part in isolation to minimize its individual reconstruction error, and stitch them back together. Whether through quantization (reducing bit-precision), pruning (removing weights), or low-rank factorization, this "divide-and-conquer" strategy is computationally efficient and scales well to models with billions of parameters.

However, this local approach suffers from a severe, systemic blind spot: error propagation. When layers or blocks are compressed in isolation, the tiny approximation errors introduced at each step do not remain confined. Instead, they compound, amplify, and snowball as they pass through the deep architecture, frequently leading to a catastrophic collapse in the final model's performance.

Understanding why this compounding effect occurs—and how to prevent it—is the central challenge of modern post-training compression.


The Anatomy of Cascading Drift

To understand why local optimization fails, we must look at how data flows through a deep neural network during inference. A network is a chain of non-linear compositions:

$$Y = f_L( \dots f_2(f_1(X)) )$$

When we compress a model layer-by-layer, we typically calibrate each layer using the clean, uncorrupted activations generated by the original full-precision model. For example, when optimizing the weights of Layer 2, we assume its input is the pristine output of the original Layer 1 ($f_1(X)$).

In practice, however, Layer 1 has already been compressed into a noisy approximation, $\tilde{f}_1$. As a result, during actual inference, Layer 2 receives a corrupted input:

$$\tilde{X}_2 = f_1(X) + \epsilon_1$$

This mismatch triggers what is known as input distribution drift. As Shin et al. and Li et al. observe in the context of structural pruning, altering the parameters of an early layer fundamentally shifts the feature space for all subsequent layers. Because downstream layers are forced to process feature distributions they were never calibrated to handle, their own outputs deviate even further from the target distribution.

[Clean Input] ---> ( Layer 1 + Noise ε1 ) ---> [Shifted Distribution] ---> ( Layer 2 + Noise ε2 ) ---> [Severe Drift]

This sequential dependency creates a compounding effect. As shown by Arai et al. in Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization, these errors do not accumulate linearly; in low-bit regimes (such as 2-bit or 3-bit quantization), they grow exponentially. Early work by Wu in Quantized Convolutional Neural Networks for Mobile Devices highlighted that even minor estimation errors in early layer responses snowball across successive layers, garbling the signal long before it reaches the final task head.


The Coupling of Quantization Noise

Why don't these random errors simply cancel each other out over a deep architecture? The answer lies in how downstream layers interact with upstream noise.

In a shallow or highly linear network, errors might distribute evenly. But in deep, non-linear models like Transformers, layers are highly coupled. Nahshan et al. demonstrate in Loss aware post-training quantization that at ultra-low bitwidths, quantization noise across layers becomes deeply intertwined. Downstream layers often amplify the noise introduced upstream rather than mitigating it.

This amplification is particularly destructive in attention mechanisms. As Wang et al. show in AQ-DETR: Low-Bit Quantized Detection Transformer with Auxiliary Queries, when quantization errors propagate through a transformer encoder, the resulting feature distortion breaks the precise query-key matching required by the decoder.

Furthermore, traditional compression metrics often rely on weight-matching proxies (e.g., minimizing $\|\tilde{W} - W\|_2^2$). As Yvinec et al. argue, this local weight optimization fails to capture the complex interactions of the global activation flow. A weight modification that appears benign in isolation can cause an explosive distortion when multiplied across dozens of layers.


Moving Beyond Isolation: Global and Block-Wise Solutions

To prevent the performance collapse caused by error propagation, researchers have developed strategies that replace isolated layer-wise optimization with coordinated, multi-layer calibration.

1. Block-Wise and Modular Reconstruction

Instead of optimizing a single weight matrix at a time, modern frameworks expand their optimization horizon to functional units or "blocks" (such as an entire Transformer layer containing both attention and feed-forward networks).

By optimizing parameters across a broader block, the algorithm allows internal components to actively coordinate and compensate for each other's errors. Bai et al. in Towards Efficient Post-training Quantization of Pre-trained Language Models and Chen et al. in EfficientQAT: Efficient Quantization-Aware Training for Large Language Models demonstrate that block-by-block optimization effectively contains the drift, preventing errors from cascading freely across the entire transformer chain.

2. Exploiting Inter-Layer Dependency

Layers do not have to be passive victims of upstream noise. If the optimization framework has a global view of the network, a downstream layer can adjust its weights to counteract the errors introduced by an upstream layer.

[Layer 1 (Introduces Error)] ---> [Layer 2 (Adjusts weights to cancel Layer 1 Error)] ---> [Clean Output]

This concept of error cancellation is central to Network-Wise Quantization (NWQ), introduced by Wang et al. in Leveraging Inter-Layer Dependency for Post -Training Quantization. By optimizing the network globally, NWQ allows rounding errors in one part of the architecture to be systematically neutralized by adjustments in another, drastically reducing the cumulative noise floor.

3. Sequential Calibration Realignment

If a framework must compress the model sequentially due to memory constraints, it can eliminate input distribution drift by changing how it calibrates downstream layers.

Instead of feeding pristine activations to downstream layers during calibration, the framework feeds them the actual, noisy activations generated by the already-compressed preceding layers. In GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration, Li et al. employ asymmetric calibration to explicitly account for this accumulated noise. By optimizing each layer against the real, drifted distribution it will encounter during inference, the model prevents the sequential "butterfly effect" and maintains stable representations even at ultra-low bitwidths.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_quantization_error_propagation_66b08e95,
  title = {quantization error propagation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/quantization-error-propagation/rdr_66b08e95fb904c4aa1040a1dc22c143b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_quantization_error_propagation_66b08e95_v1,
  title = {quantization error propagation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/quantization-error-propagation/rdr_66b08e95fb904c4aa1040a1dc22c143b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake