Compressing Neural Networks Without Original Data

198 concepts146 papers1 proposal

Deep neural networks are notoriously heavy, making compression techniques like quantization and pruning essential for deploying models on resource-constrained edge devices. However, standard compression workflows suffer from a critical dependency: they require access to the original training or calibration datasets to minimize accuracy loss and fine-tune parameters.

This dependency creates a massive bottleneck in sensitive domains like healthcare, defense, and finance. Strict privacy regulations, security concerns, or proprietary restrictions frequently lock away the source data, rendering traditional data-driven compression unusable. To bridge this gap, researchers have developed data-free model compression (also known as zero-shot quantization or pruning), a paradigm that optimizes models using only their pre-trained weights and internal metadata.

Hallucinating Data via Deep Inversion

How can a compression algorithm calibrate a model without seeing a single real sample? The core intuition is that a pre-trained model's weights already contain a footprint of the training data distribution. By treating the model itself as a source of information, we can "hallucinate" or synthesize calibration data from random noise.

A primary mechanism for this is matching internal statistics, particularly from Batch Normalization (BN) layers. The running means and variances stored in BN layers act as architectural anchors. As demonstrated in frameworks like ZeroQ by Yaohui Cai et al. and Generative Zero-Shot Network Quantization (GZNQ) by Xiangyu He et al., an optimization loop or a generator network can synthesize images by forcing the synthetic data's internal activations to match these stored BN statistics. Similarly, Zi Wang models a teacher network's intermediate feature space as a multivariate normal distribution to synthesize high-quality pseudo-samples for data-free knowledge distillation.

For architectures without BN layers, such as Vision Transformers (ViTs), alternative structural properties must be exploited. For instance, PSAQ-ViT by Zhikai Li et al. leverages the unique way self-attention modules distinguish foreground from background to guide data synthesis from random noise. In specialized domains like image deraining, Weiqi Zou et al. show that deep inversion of a pre-trained model can "dream" up in-distribution rainy images to prune heavy networks without original samples.

Overcoming Mode Collapse and Easy Samples

A major challenge with data synthesis is "mode collapse," where a generator produces highly repetitive, low-diversity samples. If the synthetic data is too "easy," it fails to capture the full complexity of the real-world distribution, causing the compressed model to perform poorly on edge cases.

To solve this, researchers focus on expanding sample diversity and difficulty:

  • Relaxing Constraints: Haotong Qin et al. introduced Diverse Sample Generation (DSG) by intentionally relaxing constraints on synthetic data generation to better mimic real-world variety.
  • Normalization Flows: To combat mode collapse across both CNNs and Transformers, Kai Zhao developed EnhancingDFQ, which couples the internal features of the full-precision model with normalization flows to generate highly diverse synthetic data.
  • Adversarial Dynamics: Biao Qian framed data-free quantization as a zero-sum game (AdaSG), forcing a generator and a quantized network to compete so that the generated data actively targets the specific weaknesses of the compressed model. Similarly, the HAST framework by Huantong Li generates intentionally difficult synthetic samples using adversarial perturbations to bridge the performance gap.
  • Loss Landscape Optimization: Rather than just matching statistics, SADAG by Hoang Anh Dung et al. generates data designed to minimize the "sharpness" of the model's loss landscape, ensuring the quantized model settles into flatter, more robust regions.

Data-Free Structural Transformations

While data synthesis is highly effective, another branch of data-free compression avoids data generation entirely. Instead, these methods rely on purely mathematical transformations of the pre-trained weights or feature spaces, enabling plug-and-play deployment with zero calibration data.

One key insight is that while model weight matrices might appear complex and full-rank, their internal activations or features often exhibit low-rank properties. Hao Yu 0027 et al. discovered this paradox in Transformers, allowing them to build a few-shot unsupervised compression framework that mimics these low-rank features.

For Large Language Models (LLMs), DecoQuant by Peiyu Liu et al. uses matrix decomposition to isolate problematic outlier values into a tiny high-precision tensor, allowing the bulk of the memory-heavy Key-Value (KV) cache to be compressed to 4-bit without any calibration data. Similarly, Targeted Low-rank Refinement (TLR) by Li Shen et al. uses a tiny low-rank matrix addition to approximate and recover the information lost during LLM weight pruning.

Other mathematical approaches alter the network structure directly:

  • Network Decoupling: Jianbo Guo et al. mathematically proved that standard convolutions can be exactly represented as a sum of depthwise separable convolutions, offering a training-free path to convert heavy CNNs into lightweight structures.
  • Function Space Alignment: For CNNs, Alper KALLE et al. incorporate input covariance statistics into new Alternating Least Squares (ALS) algorithms to minimize error directly in the layer's output function space, bypassing the need for expensive fine-tuning phases.

Decentralized and Rigorous Privacy Frameworks

When data cannot be centralized due to legal or ethical boundaries, data-free concepts extend naturally into federated environments. Frameworks like FedPrLLM by Pengxin Guo et al. and FedSpaLLM by Guangji Bai et al. enable Large Language Model pruning across private, decentralized silos by utilizing layer-wide parameter comparisons or localized calibration without ever sharing raw data.

Finally, to prevent inadvertent data leakage during the compression process itself, Arturs Backurs introduced formal frameworks for differentially private model compression. This approach ensures that pruning or compressing a model preserves the rigorous mathematical privacy guarantees established during the model's initial training, preventing deployment-stage optimizations from compromising sensitive user data.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 167 papers
  2. Version 1 · 2026-10-04 · 169 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_data_privacy_constraints_privacy_preserv_2ce60198,
  title = {data privacy constraints (privacy-preserving model compression)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/data-privacy-constraints-privacy-preserving-model-compression/rdr_2ce60198c85c44689425604eb1d81db3},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 2, frozen text):

@misc{lacuna_data_privacy_constraints_privacy_preserv_2ce60198_v2,
  title = {data privacy constraints (privacy-preserving model compression)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/data-privacy-constraints-privacy-preserving-model-compression/rdr_2ce60198c85c44689425604eb1d81db3/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake