Image-processing programs such as JPEG encoders, edge detectors and denoisers are compiled under a rule that every output bit must match the original program. Their outputs, though, are judged by people, who cannot see most small numerical changes. This note proposes letting a code optimizer trade exactness for speed under a perceptual budget: a transformed program is accepted only if an image-quality model, one that predicts human judgments, says its outputs look as good as the exact program's. The less obvious part is a warning that drives the whole design. A search procedure that optimizes code against a quality model behaves like an adversarial attacker on that model. Learned image-quality models are known to be fooled by small perturbations, so the optimizer will likely find "fast" programs that pass the check while looking worse to people.
The core claim has two parts, and both are testable. First, at equal human-judged quality, a perceptual budget lets an optimizer reach larger speedups than the numerical-error budgets used in prior approximate-computing systems. Second, this gain survives only if the acceptance check is built to resist the optimizer, which we do by combining an adversarially trained metric with a convex pixel-domain metric and auditing accepted programs on held-out judges. If the second part fails, the first part is an artifact of gaming the metric.
Where the two directions meet
Recent work on the gap between program intent and what compilers can exploit has made code optimizers stronger, but all of it keeps strict equivalence. Learning Performance-Improving Code Edits fine-tunes language models to rewrite C++ for speed on the PIE dataset. It also documents that when unit tests are weak, the optimizer finds "cheats" that change program behavior. ECCO pairs a language-model strategist with a genetic algorithm to order LLVM passes, and Compiler-R1 trains an agent with reinforcement learning to do the same. Compiler-R1 also notes that its instruction-count reward is an imperfect proxy for runtime. These papers supply our search machinery. They also motivate the main risk: optimizers exploit whatever oracle checks them, whether that oracle is a test suite or an instruction counter. We differ from all three by relaxing exactness, which none of them do.
The perceptual side supplies the oracle and the reasons to distrust it. Perceptual Fidelity Aware MSE (PAMSE) blurs both images with a Gaussian before taking squared error. The blur stands in for the eye's insensitivity to fine detail. PAMSE stays convex and cheap, and it raises rank correlation with human scores on TID2008 from 0.7718 (plain MSE) to 0.9162. BiRQA is a small learned full-reference metric hardened by adversarial training. It reaches 0.988 rank correlation on LIVE, and its white-box adversarial correlation on KADID-10k rises from 0.30–0.57 for undefended models to 0.60–0.84. That improvement is the constraining fact for this proposal: undefended learned metrics are easy to fool, and even defended ones are imperfect. Bayesian Tensor Inference for facial hallucination and the Nano Banana Pro evaluation show the opposite failure. Outputs that people prefer can score worse on pixel error, so a pixel-error budget may reject good approximations.
The closest existing system is Rumba. Rumba runs approximate hardware accelerators on kernels including jpeg and sobel. It predicts large numerical output errors and re-executes the offending loop iterations exactly, cutting average error from 20.6% to 10% while keeping 2.2x energy savings. Rumba measures quality as scalar numerical error, works at the accelerator boundary, and does not search over program transformations. We move the budget into the compiler's search loop and replace numerical error with a guarded perceptual check.
Research Program
A program $p$ is a candidate transformation of the exact program $p_0$. The optimizer solves
$$\max_{p} \; \mathrm{speedup}(p) \quad \text{subject to} \quad Q\big(p(x), p_0(x)\big) \ge \tau \ \text{for all } x \in C,$$
where $C$ is a calibration set of input images, $Q$ is a full-reference quality score comparing the transformed and exact outputs, and $\tau$ is the budget.
The transformation space has three tiers. The first is floating-point relaxation, such as fast-math reassociation and narrowing of precision. The second is loop perforation, which skips a fraction of iterations in marked pure loops; Rumba's purity condition makes this safe to apply. The third is source edits proposed by a language model in the PIE style, for example replacing an exact filter with a separable or lower-order approximation. The search procedure is an ECCO-style loop, with a language model that proposes transformation strategies and a genetic algorithm that tunes them, so that only the acceptance criterion varies across conditions.
The guard is the main design ingredient. $Q$ is the minimum of two calibrated scores. One is PAMSE, which is convex and so offers no isolated blind spots where an adversarial optimizer can push the score up without changing the image. The other is adversarially trained BiRQA, which captures structural damage that blurred MSE misses. Acceptance also requires that the program pass on a held-out input set that the search never sees. As an audit, we measure a gaming rate: the fraction of accepted programs that a held-out metric family or a human two-alternative forced-choice test rejects. The held-out metrics are no-reference NIQE and MUSIQ plus LPIPS, all available through LowLevelEval.
Workloads are Rumba's image kernels (jpeg, sobel) and the opencv suite in CompilerGym. Inputs come from LowLevelEval datasets such as DIV2K and SIDD, which give natural images with varied texture. Speed is measured in cycles under a fixed simulator configuration, with bare-metal runs repeated to control the noise that the PIE work reports.
Baselines
The naive reference point is the exact program compiled with clang -O3. Every speedup is reported against it, and its outputs define the quality reference.
The strongest exact competitor is ECCO-style pass-order search under strict equivalence, run with the same search budget and on the same opencv programs. ECCO reports a 24.44% average cycle reduction over opt -O3 across seven CompilerGym suites. That figure is not specific to our kernels, so we rerun ECCO's method rather than quote it. To show that relaxation buys something exact tuning cannot, the perceptual-budget optimizer must beat this rerun at a quality level people cannot distinguish from exact output.
The strongest approximate competitor is a Rumba-style numerical-error budget. It uses the same transformation space and search, and accepts a program if its mean relative output error stays under a threshold. Rumba reports error falling from 20.6% to 10% with 2.2x energy savings on its suite, but no reported result was found on Lacuna for Rumba under any perceptual metric or human study. A PSNR-threshold budget is added as a second numerical baseline. For fairness, each numerical threshold is chosen so that its accepted outputs match the perceptual condition's human-judged quality on a calibration split. The comparison is then speedup at matched human quality, not speedup at an arbitrary threshold.
Two ablations isolate the guard. One replaces the guard with undefended BiRQA alone. The other replaces it with PAMSE alone. If the single-metric optimizers report larger speedups but higher gaming rates, the guard is doing real work. All conditions get the same number of compiled and evaluated candidates and the same calibration and held-out images.
Why not just…?
Why not just use a PSNR or relative-error threshold, as Rumba does? This is the serious threat. On smooth kernels such as sobel, numerical error and perceived quality may track closely enough that a matched PSNR budget reaches the same speedup. The matched-human-quality comparison settles this directly. We expect a perceptual advantage mainly on transforms that move or smooth fine texture, where the hallucination results above show pixel error and preference diverging. If no advantage appears on any workload, we report that perceptual budgets add nothing to approximate compilation.
Why not just plug in one good learned metric? BiRQA's own adversarial numbers show that undefended learned metrics lose much of their agreement with humans under white-box pressure. A search over programs applies the same kind of pressure through a less direct channel. The single-metric ablation plus the gaming-rate audit test whether this happens in practice. That is an open empirical question, and we report it either way.
Why not just lower the input resolution or iteration count globally? Uniform knobs are a fair baseline and are included as a one-parameter sweep. Their weakness is that they spend the error budget evenly, while a guided search can concentrate approximation where the eye is least sensitive. If the sweep matches the searched programs at matched quality, the program-level search is unnecessary.
Decisive checks
The proposal succeeds if three things hold on at least the jpeg and opencv workloads. The guarded perceptual budget beats exact ECCO-style tuning by a clear margin in speedup at human indistinguishability, meaning forced-choice accuracy near chance. It beats the matched numerical budgets. Its gaming rate stays low, while the single-metric ablations show measurably higher gaming. If speedups appear only together with high gaming rates, the conclusion is a negative but useful one: perceptual metrics are not yet safe objectives for program search. This parallels the weak-test cheating already observed in PIE.
References
- Learning Performance-Improving Code Edits
- ECCO: Evidence-Driven Causal Reasoning for Compiler Optimization
- Compiler-R1: Towards Agentic Compiler Auto-tuning with Reinforcement Learning
- Perceptual Fidelity Aware Mean Squared Error
- BiRQA: Bidirectional Robust Quality Assessment for Images
- Bayesian Tensor Inference for Sketch-Based Facial Photo Hallucination
- Is Nano Banana Pro a Low-Level Vision All-Rounder? A Comprehensive Evaluation on 14 Tasks and 40 Datasets
- Rumba: an online quality management system for approximate computing