Pose estimators running on phones, robots, or embedded cameras have to be small. In practice that means quantization, which stores weights and activations at 4 bits or fewer instead of 32. These models are also deployed where the training data did not come from: a model trained on synthetic humans gets used on real people behind furniture, and a model trained on people gets used on animals. Each problem has its own literature. Adversarial domain adaptation trains a feature extractor to fool a classifier that tries to tell source images from target images. Compression research studies how much accuracy a smaller model loses. The two problems compound, though. Test-Time Model Adaptation for Quantized Neural Networks shows that the loss gap between quantized and full-precision classifiers grows sharply as precision drops, and that it widens further under distribution shift. No work in our file measures this for keypoint regression, and no work tries to close both gaps with one cheap mechanism that adds nothing to inference time.
Our proposal is to freeze a quantized pose network and learn only a small set of per-channel offsets on its internal features. These offsets fold into the existing bias terms, so inference cost is unchanged. The offsets are trained with an adversarial discriminator that receives an unusual pair of inputs: features of the full-precision model on source images, and features of the quantized, compensated model on target images. Fooling this discriminator forces the cheap model to look, internally, like the trusted expensive model on familiar data. Anatomical plausibility of the predicted skeleton serves as an additional label-free check. The claim is that one minimax objective on a few thousand foldable parameters recovers most of the accuracy that both quantization and domain shift remove. We also claim it does this more reliably than adapting first and quantizing afterwards, and more cheaply than adversarially retraining the quantized weights.
Core Claim
Let $f$ be the full-precision feature map of a pose network trained on source data. Let $\tilde f_\theta$ be the same network after 4-bit (or lower) quantization, with frozen weights plus learnable per-layer, per-channel scale-and-offset parameters $\theta$. Let $D$ be a small discriminator over intermediate features, and $\tilde g_\theta$ the quantized network's keypoint output. We solve
$$ \min_\theta \max_D \; \mathbb{E}_{x_s}\big[\log D(f(x_s))\big] + \mathbb{E}_{x_t}\big[\log\big(1 - D(\tilde f_\theta(x_t))\big)\big] + \lambda\, \mathcal{L}_{\text{kin}}\big(\tilde g_\theta(x_t)\big) + \mu\, \mathcal{L}_{\text{sup}}\big(\tilde g_\theta(x_s), y_s\big), $$
where $x_s, y_s$ are labelled source images and keypoints, $x_t$ are unlabelled target images, $\mathcal{L}_{\text{kin}}$ penalizes bone-length ratios and joint angles outside anatomical ranges, and $\mathcal{L}_{\text{sup}}$ keeps source accuracy. After training, the scale and offset fold into the preceding convolution's bias and quantization scale. The deployed network is bit-for-bit the same size and speed as the plain quantized one.
Two things are new relative to the file. The first is the asymmetric reference: the discriminator compares a quantized-target model against a full-precision-source model, rather than source against target within the same network. The second is restricting adaptation to foldable continuous parameters. This sidesteps the non-differentiable staircase of quantized weights that makes ordinary adversarial training unreliable there. The working hypothesis behind the restriction is that both quantization error and appearance shift in convolutional pose networks show up mostly as shifts in per-channel feature statistics and a few dominant directions. The experiments test this hypothesis directly rather than assume it.
Relation to Prior Work
Three pose papers motivate the problem and supply the adaptation recipes we compress.
- Cross-Domain Adaptation for Animal Pose Estimation uses a domain discriminator to transfer human keypoint models to animals (65.7 mAP on Animal-Pose).
- Global Adaptation meets Local Generalization (PoseDA) separates global scale and translation alignment from adversarial bone-level augmentation for 3D lifting (61.3 mm MPJPE on MPI-INF-3DHP from Human3.6M).
- Unsupervised Domain Adaptation for Occlusion Resilient Human Pose Estimation (OR-POSE) combines a mean teacher, an anatomical prior, and occlusion curricula for synthetic-to-real transfer (76.9% [email protected] on 3DOH50K).
All three adapt dense full-precision networks. None asks what survives quantization. We reuse their discriminator design and OR-POSE's anatomical-prior idea, but apply them to a frozen low-bit network.
Two compression papers supply the mechanism.
- Restoring Pruned Large Language Models via Lost Component Compensation shows that a constant vector folded into biases can restore much of a pruned LLM's accuracy. That vector is built from SVD directions of the dense-versus-pruned activation gap, with learned magnitudes. We borrow the bias-folding trick and, in one variant, the SVD directions. We replace its generic reconstruction objective with an adversarial plus kinematic one, so that the compensation also absorbs domain shift.
- Compress, Then Prompt recovers compressed LLMs with learned soft prompts. Prompts have no clean analogue in convolutional heatmap networks, which is why we act on biases instead.
Two adaptation papers constrain the design.
- ZOA (Test-Time Model Adaptation for Quantized Neural Networks) motivates the problem through its quantization-gap result. It also shows that backpropagation-based test-time adaptation needs thousands of megabytes, which rules out on-device adversarial training. We therefore adapt once, offline, on unlabelled target images collected at the deployment site.
- DARDA selects pre-trained sub-networks on Jetson-class devices to cope with corruptions. It handles only photometric corruptions for classification, and it stores a library of models. We store one model.
Model Compression for Domain Adaptation through Causal Effect Estimation shows that compression can help target-domain accuracy in text classification. That is an encouraging precedent we do not rely on.
For backbones, we use real-time pose architectures:
- the soft-gated HourGlass/U-Net of Toward fast and accurate human pose estimation via soft-gated skip connections, at 94.1% PCKh on MPII;
- the TNet backbone of Context-Guided Adaptive Network for Efficient Human Pose Estimation, at 68.1 AP and 25.4 FPS on COCO.
Neither paper tests domain shift, so we would be measuring that for the first time.
Research Program
The first stage is diagnostic. For each backbone, we take the source-trained full-precision model and quantize it post-training at W8A8, W4A4, and W3A3. We measure target accuracy before any adaptation, to establish whether the compounding seen by ZOA for classification also appears for heatmap regression. A key question is whether it is worse for regression: heatmap peaks require sub-pixel precision, so small activation errors may move the predicted joint. We also record per-layer and per-channel statistics of the activation gap. These show whether most of the gap is captured by per-channel mean and scale shifts, which per-channel offsets can fix, or by a few SVD directions, which need the RestoreLCC-style variant.
The second stage builds the method in two versions:
- per-channel scale-and-offset compensation on every block;
- low-rank compensation, using top and "minor" SVD directions of the gap between full-precision source features and quantized target features, with magnitudes learned adversarially.
The discriminator is attached at two or three feature resolutions. Activation rounding is passed through with a straight-through estimator: the forward pass rounds, and the backward pass treats rounding as the identity. Weights are never updated, so the estimator is only ever applied to activations. The kinematic loss uses skeleton bone-length ratios and joint-angle ranges in the spirit of OR-POSE's prior and COMETH's range-of-motion constraints (COMETH). For animals, it is restricted to the coarse limb symmetries available from Animal-Pose annotations.
The third stage measures deployment. We fold the parameters, export the model, and measure latency, memory, and energy on a Jetson-class device against the uncompensated quantized model. The expected latency difference is zero, and we verify it rather than assume it.
Proposed Experiments
There are three transfer settings, chosen because the file reports a strong full-precision adapted number for each:
- SURREAL to 3DOH50K, an occluded synthetic-to-real shift, with [email protected] as the metric and OR-POSE's reported 76.9% as the full-precision reference;
- COCO humans to Animal-Pose, with mAP and the 65.7 mAP reported for WS-CDA as reference;
- Human3.6M to MPI-INF-3DHP for the lifting stage, with MPJPE and PoseDA's 61.3 mm as reference.
In the lifting setting, the 2D front end is the quantized model and the lifting network stays full precision. We additionally report MPII PCKh and COCO AP on the source side, to check that compensation does not damage in-domain accuracy. Every method is run at W4A4 and W3A3 on the same backbone, with the same unlabelled target images, the same labelled source data, and a matched GPU-hour budget for tuning.
Baselines
The naive reference is the source-trained model quantized post-training with no adaptation. It shows how much accuracy is lost to the combination of shift and quantization. The full-precision model with no adaptation, and the full-precision adapted models above, bound the problem from the other side.
The most natural competitor is adapt-then-quantize. We train the full-precision model with the published adversarial method for each setting: OR-POSE's recipe on 3DOH50K, the WS-CDA discriminator on Animal-Pose, and PoseDA on 3DHP. We then apply the same post-training quantizer. It is strong because it reuses the best available adaptation unchanged and costs nothing extra at compression time. Beating it means higher target accuracy at the same bit width. We expect the margin to grow as bits fall from 8 to 3, since the adaptation was never told about rounding.
The strongest competitor is quantization-aware adversarial training. It updates all weights of the quantized network under the same discriminator and source loss, using straight-through gradients, with the same tuning budget in GPU-hours. It has far more freedom than our method and could plausibly match or beat it. Beating it means equal accuracy with a training cost and parameter count at least an order of magnitude lower, plus lower variance across seeds. If it is clearly more accurate at equal budget, we will report that our method only trades accuracy for cost.
Target batch-norm re-estimation is the cheapest foldable alternative. It re-computes normalization statistics on target images, then quantizes. It is the closest in spirit, since it also only changes per-channel affine terms, and it must be beaten clearly for the adversarial objective to be justified. We also run a forward-only test-time adapter in the style of ZOA, transplanted to heatmap outputs using the kinematic loss as its objective, as the on-device alternative.
Three ablations isolate the key ingredients:
- Reference model: replace the full-precision source reference with the quantized network's own source features. This is the standard symmetric discriminator, and it isolates the value of the asymmetric reference.
- Kinematic loss: set $\lambda = 0$.
- Discriminator: remove it while keeping the kinematic loss, compensation, and a feature-matching loss to the full-precision model on source images.
Why not just…?
Why not just adapt the full-precision model and then quantize it? This is the default, and it may suffice at 8 bits. The ZOA results suggest it breaks down at low precision, where the quantized model's loss gap grows sharply under shift. Those results are for classification, though, so the bit-width sweep in our first and main experiments is what settles it for pose. If adapt-then-quantize matches us at W4A4 and W3A3, the central claim fails.
Why not just re-estimate batch-norm statistics on target data before quantizing? This is the most serious threat, because it changes the same foldable quantities for free. Our counter-hypothesis is that the quantization gap is aligned with the full-precision source features rather than with target statistics alone, and that occlusion shifts are not captured by first- and second-order statistics. Our first-stage gap analysis and the direct comparison test this. If batch-norm re-estimation plus the kinematic loss reaches our accuracy, we will conclude the discriminator is unnecessary and report the simpler method.
Why not just retrain the quantized weights adversarially, or attach LoRA adapters? Full quantization-aware adversarial training is our strongest baseline and is run at matched budget. LoRA adapters add inference latency in convolutional networks unless merged, and merging them into low-bit weights requires re-quantization. RestoreLCC also reports that unconstrained adapters underperform targeted compensation at high compression. That evidence comes from pruned LLMs, so our comparison is what confirms or refutes it for pose.
Why not just use the quantized network's own source features as the discriminator's reference? That is the symmetric ablation. If it matches the full-precision reference, the method reduces to ordinary adversarial adaptation restricted to biases. The contribution would then be the parameter restriction alone, which is still useful but much smaller.
Decisive Checks
The idea holds if three conditions are met at W4A4 and W3A3 across all three settings:
- Adversarial bias compensation beats both adapt-then-quantize and batch-norm re-estimation on target accuracy.
- The full-precision-source reference beats the symmetric discriminator.
- Measured device latency equals that of the plain quantized model.
The idea is falsified if the gap analysis shows most of the error lies outside per-channel and low-rank directions, which would leave nothing for foldable parameters to fix. It is also falsified if batch-norm re-estimation recovers most of the accuracy our method recovers. On 3DOH50K, a useful success threshold is coming within a few points of OR-POSE's full-precision 76.9% [email protected] at 4 bits. That would show the low-bit model need not give up adaptation quality to meet real-time budgets.
References
- Test-Time Model Adaptation for Quantized Neural Networks
- Cross-Domain Adaptation for Animal Pose Estimation
- Global Adaptation meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimation
- Unsupervised Domain Adaptation for Occlusion Resilient Human Pose Estimation
- Restoring Pruned Large Language Models via Lost Component Compensation
- Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt
- DARDA: Domain-Aware Real-Time Dynamic Neural Network Adaptation
- Model Compression for Domain Adaptation through Causal Effect Estimation
- Toward fast and accurate human pose estimation via soft-gated skip connections
- Context-Guided Adaptive Network for Efficient Human Pose Estimation
- COMETH: Convex optimization for multiview estimation and tracking of humans