Introduction
Post-Training Quantization (PTQ) is a critical technique for deploying deep neural networks on resource-constrained edge devices. By converting 32-bit floating-point (FP) weights and activations into low-precision integers (e.g., 4-bit or 2-bit), PTQ significantly reduces memory footprint and computational latency. Unlike Quantization-Aware Training (QAT), PTQ is highly efficient as it requires no full retraining and only a small set of unlabeled calibration data.
However, as bit-widths drop to ultra-low levels, standard PTQ methods often suffer from severe accuracy degradation. Traditional calibration techniques typically rely on minimizing simple distance metrics, such as Mean Squared Error (MSE) or Cosine Similarity, between FP and quantized activations. The authors of this CVPR paper argue that these metrics are insufficient because they focus on numerical alignment rather than the preservation of underlying semantic information.
The CL-Calib Framework
To address these limitations, the researchers introduce CL-Calib (Contrastive Learning Calibration), a "plug-and-play" module designed to maximize information retention during the quantization process.
The core idea is to treat quantization calibration as a self-supervised contrastive learning task. Instead of just matching values, CL-Calib ensures that the quantized model's internal representations remain informative relative to the original FP model.
Technical Implementation
The framework operates by defining positive and negative pairs of activations:
- Positive Pairs: The quantized activation and the corresponding FP activation derived from the same input image. These are "pulled" together in the feature space.
- Negative Pairs: Activations derived from different images, which are "pushed" apart to maintain discriminative power.
A significant challenge in applying contrastive learning to PTQ is the lack of data; contrastive methods usually require large batches and extensive training. CL-Calib overcomes this by using the remaining layers of the pre-trained FP model as a "critic function."
As shown in the diagram above, the frozen FP network acts as a fixed encoder that projects activations into a space where their semantic similarity can be judged. This allows the model to optimize quantization parameters (scaling factors and zero points) based on how well they preserve the features necessary for the final prediction.
Key Insights
- Mutual Information Maximization: The authors provide a theoretical proof that minimizing the contrastive loss (InfoNCE) is equivalent to maximizing the lower bound of Mutual Information (MI) between the FP and quantized activations. This shifts the objective from "matching numbers" to "preserving signal."
- Stability and Regularization: By using the frozen FP network as a critic, the method avoids the common "collapse" issue in contrastive learning. Furthermore, CL-Calib acts as a regularizer, preventing the quantization parameters from overfitting to the small calibration dataset (typically only 128 images).
Experimental Results
The primary evaluation on ImageNet demonstrates that CL-Calib consistently enhances state-of-the-art PTQ methods like QDrop and PD-Quant. The improvements are most pronounced in challenging 2-bit (W2A2) configurations:
- MobileNetV2: Accuracy improved by +4.20%.
- MNasNet: Accuracy increased by +5.30%.
- ResNet-18: Observed a +3.21% gain.
The method also showed strong generalization across different architectures, including Vision Transformers (ViT) and tasks like object detection.
Conclusion
CL-Calib represents a significant step forward for ultra-low-bit quantization. By leveraging contrastive learning to preserve mutual information, it allows models to maintain high accuracy even when compressed to 2 bits. While it introduces some computational overhead during the calibration phase and requires tuning of loss-weighting hyper-parameters, its ability to recover accuracy without retraining makes it a powerful tool for edge AI deployment.
Summary updated