Is In-Domain Data Really Needed? A Pilot Study on Cross-Domain Calibration for Network Quantization

Haichao Yu, Linjie Yang, Humphrey Shi

Introduction

Post-training quantization (PTQ) is a critical technique for deploying deep learning models on resource-constrained edge devices. By converting high-precision 32-bit floating-point weights and activations into low-precision integers (e.g., 8-bit), PTQ significantly reduces memory footprints and accelerates inference. However, standard PTQ pipelines rely on a small set of "calibration data" to determine the optimal dynamic ranges for quantization. Conventionally, this data is sampled from the original training distribution.

The Challenge: Data Inaccessibility

In many practical applications, the original training data is inaccessible at the time of quantization. This is particularly common in fields like medical imaging, where strict privacy regulations prevent data sharing, or in federated learning scenarios where data remains decentralized. While "data-free" quantization methods attempt to synthesize calibration data, these approaches are often computationally intensive and struggle to replicate the complex spatial correlations found in real-world imagery.

Proposed Method: Cross-Domain Calibration

This study investigates whether out-of-domain (OOD) data can serve as an effective substitute for original training data during calibration. The researchers propose a pipeline that bridges the gap between the source domain (where the model was trained) and a disparate calibration domain.

Overview of the Cross-Domain Calibration pipeline.

The core of their approach involves two key steps:

  1. BatchNorm (BN) Adjustment: Since BatchNorm layers capture the statistics of the training domain, the authors reset these statistics and pass the OOD data through the model. This "re-centers" the model's internal activations to the new data distribution before quantization parameters are calculated.
  2. Gram Matrix Selection: To identify the most suitable OOD dataset, the authors use the Gram Matrix—a representation of image "style." By calculating the $L_2$ distance between the Gram matrices of the source domain and various candidate OOD domains, they can predict which calibration set will minimize quantization error.

Experimental Setup and Diversity

The researchers conducted a comprehensive pilot study across 10 distinct tasks and 13 image domains. This included standard natural image datasets (ImageNet, MSCOCO) as well as specialized domains like satellite imagery (Agriculture-Vision), autonomous driving (Cityscapes), and medical imaging.

An example of the diverse domains tested: a chest X-ray from the NIH dataset.

Key Findings

The study yielded several surprising insights:

  • Performance Stability: With BatchNorm adjustment, the performance gap between in-domain and OOD calibration is remarkably small. For instance, on ImageNet with ResNet-50, OOD calibration achieved 75.70% accuracy, nearly matching the 75.73% in-domain baseline.
  • Real Data Superiority: Real images from entirely different domains often outperformed synthetic data generated by state-of-the-art data-free methods like ZeroQ. This suggests that the inherent structural statistics of real images are more beneficial for calibration than perfectly matched but artificial statistics.
  • Texture over Semantics: The success of OOD calibration appears to depend more on low-level feature similarity (textures and edges) than high-level semantic content. This validates the use of Gram matrices as a selection criterion.

Conclusion

This research challenges the assumption that in-domain data is a prerequisite for effective network quantization. By leveraging publicly available OOD datasets and simple statistical adjustments, practitioners can achieve high-performance quantization even when original training data is unavailable. This opens new avenues for model compression in privacy-sensitive and data-constrained environments.

Summary updated

Abstract

Post-training quantization methods use a set of calibration data to compute quantization ranges for network parameters and activations. The calibration data usually comes from the training dataset which could be inaccessible due to sensitivity of the data. In this work, we want to study such a problem: can we use out-of-domain data to calibrate the trained networks without knowledge of the original dataset? Specifically, we go beyond the domain of natural images to include drastically different domains such as X-ray images, satellite images and ultrasound images. We find cross-domain calibration leads to surprisingly stable performance of quantized models on 10 tasks in different image domains with 13 different calibration datasets. We also find that the performance of quantized models is correlated with the similarity of the Gram matrices between the source and calibration domains, which can be used as a criterion to choose calibration set for better performance. We believe our research opens the door to borrow cross-domain knowledge for network quantization and compression.
BibTeX
@inproceedings{yu2021is,
  author = {Haichao Yu and Linjie Yang and Humphrey Shi},
  title = {{Is In-Domain Data Really Needed? A Pilot Study on Cross-Domain Calibration for Network Quantization}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  url = {https://openreview.net/forum?id=oo0ZJXoPqhPE},
}

Loading related content...

Figure