Is In-Domain Data Really Needed? A Pilot Study on Cross-Domain Calibration for Network Quantization

Haichao Yu, Linjie Yang, Humphrey Shi

Abstract

Post-training quantization methods use a set of calibration data to compute quantization ranges for network parameters and activations. The calibration data usually comes from the training dataset which could be inaccessible due to sensitivity of the data. In this work, we want to study such a problem: can we use out-of-domain data to calibrate the trained networks without knowledge of the original dataset? Specifically, we go beyond the domain of natural images to include drastically different domains such as X-ray images, satellite images and ultrasound images. We find cross-domain calibration leads to surprisingly stable performance of quantized models on 10 tasks in different image domains with 13 different calibration datasets. We also find that the performance of quantized models is correlated with the similarity of the Gram matrices between the source and calibration domains, which can be used as a criterion to choose calibration set for better performance. We believe our research opens the door to borrow cross-domain knowledge for network quantization and compression.

Introduction

To deploy deep neural networks on edge devices, quantization is used to convert high-precision 32-bit floating-point weights into low-precision integers (e.g., 8-bit). Post-Training Quantization (PQ) is a popular approach because it does not require retraining the entire model. However, PQ typically requires a small "calibration dataset" to determine the dynamic ranges of activations.

Conventionally, this data is sampled from the original training set. However, in many real-world scenarios, the original training data is inaccessible due to privacy (e.g., medical records), security, or proprietary restrictions. While "data-free" methods exist to synthesize calibration data from scratch, they are often computationally expensive and difficult to tune. This study investigates a simpler alternative: Cross-Domain Calibration, using unrelated real-world images to calibrate models.

The pipeline of cross-domain calibration where a model trained on Domain A is calibrated using data from Domain B.

Methodology

The proposed pipeline for cross-domain calibration involves three key components:

1. Naive Cross-Domain Calibration

This involves simply using images from a completely different domain (e.g., using natural images from Open Images) to calibrate a model trained on a specialized domain (e.g., X-rays).

2. BatchNorm (BN) Adjustment

The core technical contribution is the adjustment of BatchNorm layers. The authors argue that BN layers act as "domain anchors," storing statistics specific to the training data. To bridge the domain gap, they:

  1. Reset the running means and variances of all BN layers in the pre-trained float model.
  2. Pass the out-of-domain calibration data through the model to accumulate new BN statistics.
  3. Fold these adjusted BN layers into the weights before calculating quantization parameters.

This process effectively "re-centers" the model's activations for the new calibration data without altering the learned weights.

3. Domain Selection via Gram Matrices

To optimize performance, one must select the "best" available out-of-domain dataset. The authors propose using the similarity of Gram matrices (a technique borrowed from style transfer) as a proxy for domain discrepancy. By comparing the "fingerprint" (Gram matrix) of the original data with public datasets, users can identify the most suitable calibration set without ever accessing the sensitive raw images.

Key Results and Insights

The study evaluates 10 tasks across diverse domains, including natural images, medical imaging (X-ray, Ultrasound), and autonomous driving.

Example of a specialized domain image (Chest X-ray) where in-domain data is often restricted due to privacy, making cross-domain calibration essential.
  • Performance Parity: On ResNet-50 trained on ImageNet, out-of-domain calibration with BN adjustment achieved 75.70% accuracy, nearly matching the in-domain baseline of 75.73%.
  • Superiority over Synthetic Data: In specialized domains like medical imaging, real out-of-domain data often provides a more robust signal than synthetic data. For the NIH Chest X-ray task, the proposed method achieved an AUC of 0.761, significantly outperforming the synthetic-data-based ZeroQ (0.718 AUC).
  • Efficiency: Unlike synthesis-based methods that require hundreds of backpropagation iterations, BN adjustment only requires a few forward passes, making it much faster and easier to implement.

Conclusion

This pilot study challenges the assumption that in-domain data is necessary for post-training quantization. By leveraging real-world out-of-domain images and simple BatchNorm adjustments, practitioners can achieve high-performance quantization even when the original training data is strictly protected. The correlation between Gram matrix similarity and quantization performance further provides a mathematically grounded way to select optimal calibration datasets.

Summary updated

Abstract

Post-training quantization methods use a set of calibration data to compute quantization ranges for network parameters and activations. The calibration data usually comes from the training dataset which could be inaccessible due to sensitivity of the data. In this work, we want to study such a problem: can we use out-of-domain data to calibrate the trained networks without knowledge of the original dataset? Specifically, we go beyond the domain of natural images to include drastically different domains such as X-ray images, satellite images and ultrasound images. We find cross-domain calibration leads to surprisingly stable performance of quantized models on 10 tasks in different image domains with 13 different calibration datasets. We also find that the performance of quantized models is correlated with the similarity of the Gram matrices between the source and calibration domains, which can be used as a criterion to choose calibration set for better performance. We believe our research opens the door to borrow cross-domain knowledge for network quantization and compression.
BibTeX
@inproceedings{yu2021is,
  author = {Haichao Yu and Linjie Yang and Humphrey Shi},
  title = {{Is In-Domain Data Really Needed? A Pilot Study on Cross-Domain Calibration for Network Quantization}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  url = {https://openreview.net/forum?id=oo0ZJXoPqhPE},
}
Figure