Post-training quantization compresses a trained network to low-precision integers using only a small "calibration set," a few hundred unlabeled examples used to pick value ranges and adjust weights so each layer's quantized output matches its full-precision output. Calibration sets are assumed to be representative, but in deployment they usually reflect whatever traffic is at hand. When that traffic is long-tailed, rare classes contribute only a handful of examples. This proposal asks whether the accuracy that rare classes lose at 2–4 bits is caused by these classes having too few distinct examples in the calibration objective, rather than too little weight. If so, the fix is to manufacture new, plausible rare-class inputs for each layer. We do this by interpolating, SMOTE-style, between the full-precision activations of same-class calibration examples. Upweighting the few real examples does not achieve the same thing.
SMOTE (Synthetic Minority Over-sampling Technique) creates new minority examples on line segments between a minority point and one of its nearest same-class neighbors. Our use of it is narrow and mechanical. At each layer being quantized, we build synthetic inputs by interpolating real activations of rare-class calibration examples, grouped by the full-precision model's own predicted class so no labels are needed. The reconstruction target is simply the full-precision layer applied to that synthetic input, so the synthetic points never need a label and are never "wrong" in the way pixel-space SMOTE images are. The two-sentence version: skewed calibration sets under-constrain the weight adjustment for rare-class activation directions, and per-class reweighting cannot add constraints that are not there. Layer-wise interpolation between rare-class activations adds them cheaply, and it should recover most of the gap to an oracle class-balanced calibration set.
Core Claim
The motivating evidence comes from ClimbQ. That work shows quantization is "zero-sum": majority classes, with wider activation spread, dominate clipping thresholds, so minority features get squeezed into one or two bins. It reports minority-class accuracy drops as large as 14% on 2-bit CIFAR-10-LT models even when overall accuracy looks acceptable. ClimbQ addresses this inside quantization-aware training, using variance rescaling, a CDF projection, and a Levene's-test-derived reweighting loss over the full long-tailed training set. It never generates samples and does not treat post-training quantization from a tiny calibration set. The method we build on is AdaQuant from Accurate Post Training Quantization With Small Calibration Sets. AdaQuant optimizes each layer's weight perturbation sequentially to minimize the squared error between full-precision and quantized layer outputs, and it reaches under 1% degradation for W4A4 ResNet50 on ImageNet. That paper assumes the calibration set is drawn uniformly from the data distribution and never tests skewed calibration. The closest work on the augmentation side is DeepSMOTE, which interpolates in an autoencoder's latent space and decodes new training images, and From SMOTE to Mixup for Deep Imbalanced Classification, which explains interpolation's benefit through uneven class margins. Both act at full precision on the training set. Neither touches calibration or quantized layers.
A structural fact sharpens the claim and limits it. For a squared-error calibration objective, duplicating a rare-class example is mathematically identical to upweighting it. Duplication and reweighting can therefore only change how hard the optimizer fits the few existing rare-class activation vectors. They cannot enlarge the subspace those vectors span. Interpolated points also lie inside the convex hull of the real ones, so they cannot widen a min–max range estimate. Any benefit must come through the weight-adjustment step and through MSE-based clipping, which is sensitive to density inside the range. This yields a falsifiable boundary condition. Interpolation should help most when the number of distinct rare-class calibration examples is small relative to the effective rank of the layer's input, and its advantage over reweighting should vanish as that number grows.
Research Program
The method, which we call interpolated minority calibration, runs inside sequential AdaQuant. For every calibration example, the full-precision model's predicted class serves as its group. Classes whose calibration count falls below the balanced share are marked rare. At layer $l$, we take the full-precision inputs $a_i$ of rare-class examples. For each, we draw a same-group nearest neighbor $a_j$ among $k$ candidates, with $k=5$ following the heuristic justified in Concentration and excess risk bounds for imbalanced classification with synthetic oversampling. We then form $\tilde a = a_i + \lambda(a_j - a_i)$ with $\lambda \sim U(0,1)$, and add the pair $(\tilde a, f_l(\tilde a))$ to that layer's objective. Here $f_l$ is the full-precision layer. Because AdaQuant is sequential, the quantized student sees its own propagated activations. We therefore interpolate the student-side inputs with the same $(i, j, \lambda)$ draw, so teacher and student stay paired. Synthetic points are generated until every class contributes equally to the objective.
The empirical program has three parts. First, we measure whether the problem exists in post-training quantization at all, as distinct from ClimbQ's training-time setting. This means quantifying per-class accuracy loss and per-class bin occupancy under skewed versus balanced calibration. Second, we compare interpolation against reweighting, against noise-based smoothing, and against an oracle. Third, we test the boundary condition by sweeping the number of distinct rare examples. Neighbor search happens in the layer's activation space, which is high-dimensional. The analysis in Concentration and excess risk bounds for imbalanced classification with synthetic oversampling warns that its bound degrades with dimension as $(k/n_1)^{1/d}$, where $n_1$ is the minority count and $d$ the feature dimension. So we also run a variant that searches neighbors in the penultimate-layer embedding and reuses those pairs at every layer.
Baselines
The naive reference point is AdaQuant run on the skewed calibration set exactly as given. This is what a practitioner gets by default.
The strongest cheap competitor is class-balanced reweighting of the AdaQuant objective using the same pseudo-labels, which is equivalent to duplicating rare examples. It isolates the central question of whether new points matter beyond new weight. It will use the same calibration examples, the same optimizer steps, and the same per-layer iteration budget as our method. Optimizer steps are counted per synthetic-inclusive batch so that our method gets no extra optimization.
The upper reference is an oracle balanced calibration set of the same total size drawn from real data. We count our method as succeeding if it closes at least half the tail-class accuracy gap between the naive baseline and this oracle, while reweighting closes significantly less.
We include a density-smoothing competitor in the spirit of KDEO from the same theory paper. It perturbs rare-class activations with Gaussian noise whose bandwidth follows Scott's rule, $h \propto n_1^{-1/(d+4)}$. This tests whether the benefit is merely jitter rather than direction-following interpolation.
A class-agnostic Mixup of calibration activations, mixing across all examples, tests whether class-aware neighbor selection matters.
Two ablations isolate the key ingredient. Input-space SMOTE, which interpolates calibration images as in raw-pixel SMOTE, tests whether working on activations matters. Cross-class interpolation tests whether same-class pairing matters.
As a compute-heavy external reference, we cite ClimbQ's own reported result rather than claim to beat it, since it uses full quantization-aware training on the whole training set. On CIFAR-100-LT at imbalance ratio 200 with 2-bit MobileNet-V2, ClimbQ+ reports 25.45% accuracy while LLSQ and BatchQuant stagnate near 10–13% (ClimbQ). Where compute allows, we will rerun ClimbQ on our trained models. I found no reported result on Lacuna for post-training quantization under a class-skewed calibration set, so the naive and oracle conditions are the first measurements of that gap.
Why not just…?
Why not just upweight or duplicate the rare-class calibration examples? This is the real threat, and the comparison is designed around it. Under a squared-error objective, upweighting adds no new constraint directions. Prior evidence that interpolation beats pure loss reweighting exists, but only at full-precision training time. In From SMOTE to Mixup for Deep Imbalanced Classification, MAMix-DRW reaches 0.79/0.82 per-class accuracy on minority classes C8/C9 of CIFAR-10 at ratio 100, versus 0.63/0.66 for LDAM-DRW. Whether that transfers to calibration is exactly what the sweep over distinct rare-example counts settles. If reweighting matches interpolation even with very few distinct rare examples, the claim is wrong and we will report that.
Why not just quantize without calibration data? Calibration-free methods such as AdpQ avoid skew by construction, but they discard the activation information that lets AdaQuant cross the 4-bit barrier. The AdaQuant paper reports that data-free approaches degrade substantially below 8 bits. We include a data-free round-to-nearest condition so readers can see where the skewed-calibration cliff falls relative to having no data at all.
Why not just add noise to rare-class activations? If isotropic noise with Scott's-rule bandwidth performs as well as neighbor interpolation, the mechanism is generic smoothing, not filling in rare-class directions, and the simpler method should be preferred. The KDEO baseline tests this directly.
Why not just retrain with ClimbQ? ClimbQ needs the full long-tailed training set and full quantization-aware training. Post-training quantization exists precisely because that is often unavailable or too expensive. The CRVQ paper reports 50 GPU hours of post-training quantization versus over 7,000 for training-based quantization on a 13B model (CRVQ). We position our method as the calibration-only alternative, not a replacement.
Proposed Experiments
The main testbed reuses ClimbQ's models and benchmarks: ResNet-56 and MobileNet-V2 on CIFAR-10-LT and CIFAR-100-LT, at W4A4, W3A3, and W2A4. We run two regimes. In the first, the full-precision model is trained balanced and only the calibration set is skewed, at ratios of 10, 50, and 200. In the second, both training data and calibration set are long-tailed. Calibration sizes are 128, 512, and 1024. Evaluation is on the balanced test sets. We report overall accuracy, mean tail-class accuracy (the rarest third of classes), worst-class accuracy, and the per-class drop from full precision. We also report per-class activation bin occupancy, which tests ClimbQ's squeezing mechanism in the post-training setting. To check scale, we run ResNet50 on ImageNet with a calibration set skewed toward a random subset of classes. This is the setting where AdaQuant's balanced-calibration result, under 1% W4A4 degradation, serves as the oracle anchor.
The decisive checks are three. The first is the oracle-gap criterion above. The second is the boundary-condition sweep: we fix total calibration size and vary distinct rare examples per class from 1 to 32. Interpolation's advantage over reweighting should be largest at small counts and shrink toward zero, and a flat or reversed curve would falsify the mechanism. The third is a guard against harm, motivated by the minority-collapse analysis and ClimbQ's warning that over-serving rare classes can cost robustness elsewhere: majority-class accuracy must not fall by more than one point relative to naive calibration. Because Jian Cheng et al. argue that statistical re-normalization cannot recover 1–2-bit models, we expect W2 gains to be smaller and report them separately rather than averaging them in. All conditions use three seeds for both calibration draws and interpolation draws, with confidence intervals on tail-class accuracy.
References
- ClimbQ: Class Imbalanced Quantization Enabling Robustness on Efficient Inferences
- Accurate Post Training Quantization With Small Calibration Sets
- DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data
- From SMOTE to Mixup for Deep Imbalanced Classification
- Concentration and excess risk bounds for imbalanced classification with synthetic oversampling
- AdpQ: A Zero-shot Calibration Free Adaptive Post Training Quantization Method for LLMs
- CRVQ: Channel-Relaxed Vector Quantization for Extreme Compression of LLMs
- Exploring deep neural networks via layer-peeled model: Minority collapse in imbalanced training
- Learning Compression from Limited Unlabeled Data