Bias Amplification in Model Compression

311 concepts136 papers4 proposals

Deep learning models are routinely compressed via pruning, quantization, and distillation to make them deployable on hardware-constrained edge devices. While these techniques are traditionally framed as purely technical, hardware-centric optimizations, a growing body of research reveals that model compression is not a neutral process. Instead, it systematically introduces a "fairness tax," disproportionately degrading performance on minority subgroups and amplifying existing societal biases.

The Capacity Crunch: Why Compression Harms the Tail

To understand why compression amplifies bias, we must look at how neural networks prioritize information under resource constraints. When a model's parameter count or numerical precision is aggressively reduced, it faces a capacity crunch.

Standard compression pipelines guide this reduction using global optimization objectives, such as minimizing aggregate loss, perplexity, or average error. Because minority subgroups or rare data patterns contribute minimally to the global loss compared to the majority data, the features and weights critical for processing these "tail" distributions are discarded first.

As shown by Xiaofeng Lin et al. in FairGRAPE: Fairness-Aware GRAdient Pruning mEthod for Face Attribute Classification and Zhe Zhao et al. in Balancing Model Efficiency and Performance: Adaptive Pruner for Long-tailed Data, standard global importance metrics fail to account for category-specific parameter contributions. Consequently, the compressed model defaults to the most frequent training patterns, relying on high-frequency statistical shortcuts and demographic stereotypes to maintain high average performance, a phenomenon detailed by Alireza Mohammadshahi et al. in What Do Compressed Multilingual Machine Translation Models Forget?.

How Different Compression Paradigms Distort Fairness

The mechanism of bias introduction manifests differently depending on the specific compression lever being pulled: pruning, quantization, or distillation.

Pruning and Sparsification

Pruning removes connections deemed redundant by global metrics. However, in Balancing Act: Constraining Disparate Impact in Sparse Models, Meraj Hashemizadeh et al. demonstrate that this creates an "excess accuracy gap" where minority subgroups suffer far greater accuracy drops than the global average. Torsten Hoefler et al. confirm in Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks that pruning sacrifices a model's ability to generalize to minority groups even when aggregate accuracy remains stable.

In high-stakes domains like computational pathology (Structured Model Pruning for Efficient Inference in Computational Pathology by Mohammed Adnan et al.) or facial recognition (EFaR 2023: Efficient Face Recognition Competition by Jurek Elliesen et al.), this means edge cases or underrepresented demographics are systematically forgotten to preserve the majority's performance.

Quantization

Reducing bit-depth alters numerical precision, but its effects are far from linear. In How Many Bits Does it Take to Quantize Your Neural Network?, Mathias Lechner et al. demonstrate that the effects of quantization are non-monotonic; adding more bits does not linearly improve robustness or fairness. Instead, quantization introduces highly unpredictable and erratic shifts in demographic bias.

As Elisabeth Kirsten shows in The Impact of Inference Acceleration on Bias of LLMs, these fairness shifts vary wildly across different model architectures and optimization algorithms. In multimodal contexts like vision-language models, Eric Slyman et al. find in You Never Know: Quantization Induces Inconsistent Biases in Vision-Language Foundation Models that the impact of quantization is highly inconsistent, challenging simple linear narratives of compression-induced harm.

Knowledge and Dataset Distillation

When a compact student model is trained to mimic a larger teacher, the capacity bottleneck forces a structural reorganization of latent space. In Is a Prestigious Job the same as a Prestigious Country? A Case Study on Multilingual Sentence Embeddings and European Countries, Jindrich Libovický shows that distilled models often conflate distinct social dimensions (such as professional prestige and national origin) into a single axis of representation, baking intersectional discrimination directly into the embeddings. Consequently, students frequently inherit or amplify the biases of their teachers, as explored by Emma Strubell et al. in Bridging Fairness and Environmental Sustainability in Natural Language Processing and Andrew Silva et al. in Towards a Comprehensive Understanding and Accurate Evaluation of Societal Biases in Pre-Trained Transformers.

This bias amplification extends from model space to data space: in FairDD: Fair Dataset Distillation, Qihang Zhou et al. reveal that condensing massive datasets into tiny synthetic versions prioritizes majority features, leading to the near-total erasure of minority group characteristics in the condensed set.

The Illusion of Aggregate Metrics

A major reason this problem goes unnoticed in standard deployment pipelines is the reliance on macro-level evaluation metrics. In Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression, Zhichao Xu et al. point out that optimizing for aggregate metrics like perplexity or global accuracy masks severe, localized harms against specific protected groups. A compressed model can appear perfectly stable on a global leaderboard while its utility for marginalized populations completely collapses.

Key Foundational Work

Properties Of Winning Tickets On Skin Lesion Classification (2020) — contributed 8 concept elements to this direction.
Characterising Bias in Compressed Models (2020) — contributed 6 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 142 papers
  2. Version 1 · 2026-10-04 · 163 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_compression_induced_bias_10382b51,
  title = {compression-induced bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/compression-induced-bias/rdr_10382b5153714e0f865f7c64342588fd},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 2, frozen text):

@misc{lacuna_compression_induced_bias_10382b51_v2,
  title = {compression-induced bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/compression-induced-bias/rdr_10382b5153714e0f865f7c64342588fd/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake