When a vision model trained on simulator or game-engine images has to run on a small device, someone must shrink it. That means removing channels (pruning) or storing weights in fewer bits (quantization). Every compression method uses data to decide what to keep. In a simulator-to-real workflow the only labeled data on hand is synthetic, so compression keeps whatever matters most on synthetic images. Synthetic-trained networks are known to lean on simulator-specific cues such as clean edges, noise-free surfaces, and canonical lighting. Our hypothesis is that compression scored on synthetic data actively protects those cues while discarding the more generic visual features that actually carry over to real images. If that is right, the usual compression–accuracy curve measured on real images is mostly a choice about what data scores the units, not a fixed property of the model.
This note proposes to test that hypothesis directly and to replace synthetic-only scoring with a label-free, target-free score. A unit (a convolutional channel or attention head) is kept when it matters for the task and also behaves the same way on synthetic images and on ordinary real photographs. The real photographs come from ImageNet, which the backbone was pretrained on anyway, so no image from the deployment domain is ever needed. The concrete prediction goes beyond "less degradation": light compression with this score should raise real-domain accuracy above the dense model. That would be a simulator-to-real analogue of the "super tickets" phenomenon. If the prediction holds, compression becomes a tool for removing simulator shortcuts rather than a cost that must be paid.
Core Claim
We make two claims, each testable on its own. First, at a fixed compression ratio, a model's real-domain accuracy depends strongly on the domain of the data used to score channels and calibrate quantization ranges. Synthetic scoring makes the gap between synthetic and real accuracy widen as compression increases. Second, adding a domain-specificity term fixes this. That term measures how differently a unit's activations are distributed on synthetic images versus generic real images. Weighting synthetic task importance by it preferentially removes simulator-specific units. The result is a non-monotonic real-accuracy curve with a peak above the dense model at light compression, and a slower decline at heavy compression.
Write $I_S(u)$ for the first-order Taylor importance of unit $u$ on the synthetic task loss, the same estimator Transfer channel pruning uses. Write $d(u)$ for the symmetric KL divergence between Gaussian fits of $u$'s activations on a synthetic batch and on a batch of generic ImageNet images. The pruning score is
$$s(u) = I_S(u)\, \exp\big(-\alpha\, d(u)\big),$$
where $\alpha \ge 0$ controls how strongly domain-specific units are penalized. Units with the lowest $s(u)$ are pruned first, and $\alpha = 0$ recovers ordinary synthetic-scored pruning. The scale $\alpha$ is chosen on a held-out synthetic split plus ImageNet images only. Using target data to set it would leak the test domain.
Where This Sits in Prior Work
Three papers motivate the hypothesis. Automated Synthetic-to-Real Generalization (ASG) shows that fine-tuning ImageNet models on synthetic data overwrites real-world priors. It counters this with a KL penalty toward a frozen ImageNet model, lifting VisDA-17 accuracy from 28.2% to 61.1%. ASG never compresses its models. Adversarially Masking Synthetic to Mimic Real shows that clean synthetic LiDAR scans give networks an easy shortcut, and removes it by masking input points. We go after the same shortcut, but inside the network's units instead of at the input. Super Tickets finds that lightly pruned BERT subnetworks beat the dense model on GLUE, which suggests pruning can act as a regularizer. That evidence is in-distribution NLP only. Whether it transfers to a simulator-to-real gap is exactly what we test.
Four papers constrain the design, and from three of them we reuse methods. TCP prunes domain-adaptation CNNs using Taylor importance on both source and target. It shows that source-only "two-stage" pruning causes negative transfer, and that higher layers are more domain-specific. TCP was run only on real-to-real Office-31 and ImageCLEF-DA and needs target images. We need no target images and test on a large synthetic gap. Joint Optimization for DNN Model Compression and Corruption Robustness (HCRC) shows that compressing after robustifying strips away robustness, and that training-time quantization beats post-training quantization. Its target is common corruptions rather than a simulator gap. Merge-Friendly Post-Training Quantization (HDRQ) is the only work in the file that quantizes on GTA5→Cityscapes. It addresses merging several target-adapted models and assumes adaptation is already done. Our setting is a single model with no target data, where the question is what compression itself keeps. RTF-Q builds a bank of quantized subnets for domain adaptation on Office benchmarks and requires target pseudo-labels.
Research Program
The work has three stages, each depending on the previous one.
The first stage is a measurement study of the calibration gap. We take dense models trained on synthetic data with plain fine-tuning and with ASG. We then compress each one three ways, scoring channels and setting quantization clipping ranges on synthetic images, on generic ImageNet images, or on unlabeled target images. The third option is an oracle reference that a deployed system would not have. Comparing real-domain accuracy across these three, at matched compression, measures how much of the compression loss on real images comes from scoring on the wrong domain.
The second stage introduces the score $s(u)$ for structured channel and head pruning. We sweep FLOPs reduction from 10% to 70%. A mechanistic check tests whether the pruned units really are shortcut detectors. We ask whether their removal hurts synthetic accuracy more than accuracy on synthetic images that have been noised and blurred to destroy rendering artifacts. We also ask whether $d(u)$ is concentrated in higher layers, as TCP's layer finding would predict.
The third stage extends the same idea to post-training quantization at W8, W4, and W3. Here the clipping range of each layer is set from ImageNet activation statistics, weighted by $\exp(-\alpha d(u))$, instead of from synthetic statistics alone. This is the secondary track. Pruning is where the claim is sharpest.
Baselines
The naive reference is the dense synthetic-trained model, which sets the zero-compression real accuracy. Next to it sits the same model compressed by magnitude pruning and by standard Taylor pruning, both scored on synthetic data, followed by post-training quantization calibrated on synthetic images. This is what a practitioner does today. Beating it at every compression ratio is the minimum bar.
The strongest target-free competitor is ASG plus compression-aware training. The model is fine-tuned while pruning or with simulated quantization, with ASG's KL-to-ImageNet loss kept active throughout. This is the most serious threat, because the KL loss already pulls the network toward real-image behavior while it shrinks. Its reported dense numbers are 61.1% accuracy on VisDA-17 and 29.64 mIoU on GTA5→Cityscapes with FCN-Res50 (ASG). We found no result on Lacuna for any compressed single model on VisDA-17 or GTA5→Cityscapes without target data, so we will produce that baseline ourselves. Our method beats it if, at matched FLOPs, it has higher real accuracy with the same training epochs, the same ASG initialization, and the same hyperparameter search budget. We will also run our score inside that training loop. If the combination wins, the score is complementary rather than a replacement.
For quantization, the closest reference is HDRQ's Hessian- and distance-regularized quantization, run in single-model mode. HDRQ reports +4.21 mIoU over standard post-training quantization at W4A4 on GTA→Cityscapes/IDD in the merging setting. Its harmonic-mean accuracy at W3A3 on Office-Home is 64.70% versus 62.99%. As an upper reference that uses target data, we run TCP with unlabeled Cityscapes or COCO images. We do not expect to beat it. We report how much of its advantage we recover without target data.
Two ablations isolate the key ingredient. Setting $\alpha = 0$ but scoring on ImageNet images tests whether real data alone explains any gain. Random pruning at matched layer-wise ratios tests whether gains come from where pruning happens rather than which units are removed. All methods share backbones, data, epoch counts, and a fixed number of tuning trials. Every tuning decision uses only synthetic validation and ImageNet images.
Why not just…?
Why not just calibrate and score on ImageNet images? This is a real threat and may account for much of the gain. It is the $\alpha = 0$, ImageNet-scored ablation. The contrastive term earns its place only if it beats this ablation by a margin larger than seed variance. If it does not, the result becomes a measurement paper: "score compression on generic real images, not synthetic ones." That would still be a useful finding, and stage one is designed to report it cleanly either way.
Why not just adapt with standard domain adaptation, then compress? That route needs target images, which edge deployments often lack. Even when they are available, TCP's two-stage baseline and HCRC's sequential pipelines both lost to joint methods, because compression applied afterward strips the adapted representations. We still include the target-data version as an upper reference, so the cost of going target-free is reported, not hidden.
Why not just rely on the generic super-ticket effect from magnitude pruning? If light magnitude pruning alone raises real accuracy, our claim reduces to "pruning regularizes," and the domain-specificity score adds nothing. The magnitude and random baselines across the full sweep settle this directly. A real-accuracy peak that appears only with $\alpha > 0$ is the evidence we need.
Why not just re-estimate normalization statistics on real images after compression? Recomputing batch-norm statistics on ImageNet images is cheap and could mask shortcut units without removing them. We run it on top of every baseline. Our gain must survive with it applied to all methods, including ViTs, which have no batch-norm layers to recompute.
Why not just scale up synthetic data? Driving in the Matrix shows that more synthetic diversity helps, but ASM shows that clean synthetic data keeps the shortcut in place. Scaling also leaves the question of what compression should keep unanswered. We train on the full available synthetic sets, so this alternative is already at its ceiling in our setting.
Proposed Experiments
The main benchmarks are VisDA-2017 classification and GTA5→Cityscapes segmentation. For classification we use ResNet-50 and ViT-B, with synthetic CAD renders as source and COCO as target, measured by accuracy. For segmentation we use FCN-Res50, to match ASG's 22.17/29.64 mIoU, and DeepLabv3+, measured by mIoU. SynLiDAR→SemKITTI serves as a cross-modality check, where source-only reaches 15.2% mIoU and ASM 22.8%, against a 55.5% oracle. For every method we report full curves of real accuracy versus FLOPs and versus bit-width, plus synthetic accuracy, across three seeds.
Three checks decide the outcome. The first is whether the synthetic-minus-real gap widens with compression under synthetic scoring. This confirms or refutes the calibration-gap claim. The second is whether some light pruning ratio with $\alpha > 0$ gives higher real accuracy than the dense ASG model while beating the ImageNet-scored $\alpha = 0$ ablation. This is the simulator-to-real super-ticket claim. The third is whether removing the pruned units hurts clean synthetic accuracy more than artifact-destroyed synthetic accuracy. This tests whether they encode simulator cues. A negative answer to the second check, with a positive first check, still yields a usable result: compression for simulator-trained models should be scored on generic real images, and the file contains no prior measurement of how large that effect is.
References
- Transfer channel pruning for compressing deep domain adaptation models
- Automated Synthetic-to-Real Generalization
- Adversarially Masking Synthetic to Mimic Real: Adaptive Noise Injection for Point Cloud Segmentation Adaptation
- Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
- Joint Optimization for DNN Model Compression and Corruption Robustness
- Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
- RTF-Q: Unsupervised domain adaptation based retraining-free quantization network
- Driving in the Matrix: Can virtual worlds replace human-generated annotations for real world tasks?