Introduction
Post-training quantization (PTQ) is a critical technique for deploying deep learning models on resource-constrained edge devices. By converting high-precision 32-bit floating-point weights and activations into low-precision integers (e.g., 8-bit), PTQ significantly reduces memory footprints and accelerates inference. However, standard PTQ pipelines rely on a small set of "calibration data" to determine the optimal dynamic ranges for quantization. Conventionally, this data is sampled from the original training distribution.
The Challenge: Data Inaccessibility
In many practical applications, the original training data is inaccessible at the time of quantization. This is particularly common in fields like medical imaging, where strict privacy regulations prevent data sharing, or in federated learning scenarios where data remains decentralized. While "data-free" quantization methods attempt to synthesize calibration data, these approaches are often computationally intensive and struggle to replicate the complex spatial correlations found in real-world imagery.
Proposed Method: Cross-Domain Calibration
This study investigates whether out-of-domain (OOD) data can serve as an effective substitute for original training data during calibration. The researchers propose a pipeline that bridges the gap between the source domain (where the model was trained) and a disparate calibration domain.
The core of their approach involves two key steps:
- BatchNorm (BN) Adjustment: Since BatchNorm layers capture the statistics of the training domain, the authors reset these statistics and pass the OOD data through the model. This "re-centers" the model's internal activations to the new data distribution before quantization parameters are calculated.
- Gram Matrix Selection: To identify the most suitable OOD dataset, the authors use the Gram Matrix—a representation of image "style." By calculating the $L_2$ distance between the Gram matrices of the source domain and various candidate OOD domains, they can predict which calibration set will minimize quantization error.
Experimental Setup and Diversity
The researchers conducted a comprehensive pilot study across 10 distinct tasks and 13 image domains. This included standard natural image datasets (ImageNet, MSCOCO) as well as specialized domains like satellite imagery (Agriculture-Vision), autonomous driving (Cityscapes), and medical imaging.
Key Findings
The study yielded several surprising insights:
- Performance Stability: With BatchNorm adjustment, the performance gap between in-domain and OOD calibration is remarkably small. For instance, on ImageNet with ResNet-50, OOD calibration achieved 75.70% accuracy, nearly matching the 75.73% in-domain baseline.
- Real Data Superiority: Real images from entirely different domains often outperformed synthetic data generated by state-of-the-art data-free methods like ZeroQ. This suggests that the inherent structural statistics of real images are more beneficial for calibration than perfectly matched but artificial statistics.
- Texture over Semantics: The success of OOD calibration appears to depend more on low-level feature similarity (textures and edges) than high-level semantic content. This validates the use of Gram matrices as a selection criterion.
Conclusion
This research challenges the assumption that in-domain data is a prerequisite for effective network quantization. By leveraging publicly available OOD datasets and simple statistical adjustments, practitioners can achieve high-performance quantization even when original training data is unavailable. This opens new avenues for model compression in privacy-sensitive and data-constrained environments.
Summary updated