A photographer's edit can now be copied from a single before-and-after pair. Small networks trained on that one pair apply the same retouching to new photos in tens of milliseconds. These methods run "open loop": they apply the learned mapping once, everywhere, with no check on whether it fits. When a new photo contains content the exemplar never showed, such as sky when the exemplar was a portrait or harsh side light when it was flat studio light, the mapping produces artifacts in those regions and nothing flags them. The usual fix for open-loop errors elsewhere in generative modeling is an iterative critique-and-revise loop. A vision-language model inspects the output and an editor repairs it, repeated several times. That costs seconds per image, which defeats the point of a 70 ms retouching model.
This note argues that one-shot retouching needs neither choice, because the single exemplar already contains enough information to act as its own critic. Hold out regions of the exemplar pair and train on the rest, and you can measure how badly the method fails on content it has not seen. From that you can fit a tiny predictor of per-patch error that needs no external model. The proposal is to use this predictor to flag the small fraction of target patches that are likely wrong. Only those patches are repaired, in the space of local color transforms rather than pixels, by spreading transforms from trustworthy neighbors. The claim to test is that this recovers most of the quality lost to content mismatch for a small fraction of single-pass latency. A second claim is that the useful number of refinement passes is about one, not the several used by agent loops.
Core Claim
Per-patch error of a one-shot transform on an unseen photo can be predicted from the exemplar pair alone, by calibrating on held-out regions of the exemplar. Selective refinement of high-predicted-error patches can then close a large part of the gap to an oracle that knows where errors are. This should cost milliseconds and require no vision-language critic. Plain fallbacks applied to the same flagged patches, such as reverting them to the original or to a single global color transform, should do measurably worse on edits that vary across space.
The closest existing work comes from two directions, and neither does this.
The open-loop one-shot methods supply the base model. INRetouch trains an 11.5K-parameter coordinate network, augmented with depth-wise convolutions so it can see local context, on one before-after pair. It runs a 4K image in 70 ms. Gokbudak et al. split images into Laplacian frequency bands, meaning layers from coarse to fine detail. They then train a small multi-layer perceptron to blend per-patch affine color transforms, also from one pair. Both papers report degradation when the target's content or lighting diverges from the exemplar, and neither has any verification step. Their failure is the motivation for this work.
The iterative-refinement literature supplies the closed-loop idea and also constrains the design:
- Fernando et al. use a vision-language critic that chooses among continue, backtrack, restart and stop while an image-to-image editor applies fixes. It beats compute-matched parallel sampling on compositional benchmarks, but every step is a full diffusion pass.
- Idea2Img refines prompts with GPT-4V across rounds.
- DocIntent wraps restoration tools in an assess-restore-verify loop with rollback, and shows that unchecked restoration can damage the content that matters. This is direct evidence that refinement needs a rollback path.
- Volcano quantifies the cost of such loops: its critique-revise step raises latency from 2.7 s to 5.8 s per response.
The most conceptually similar work is Meta-Reflection. It compiles multi-turn textual self-correction into a learned codebook inside a language model's intermediate layer, cutting first-token latency from 951 ms to 246 ms. It works on language hidden states, depends on a large offline corpus of teacher reflections, and needs access to model internals. This proposal differs in three ways. It works on continuous spatial transform fields. Its "teacher" is the held-out exemplar itself rather than an offline corpus. And it does not train a single-pass replacement for the loop; it makes the loop cheap enough to keep and selective enough that most pixels never enter it.
Research Program
The system has three parts, each small enough to build on top of the existing open-loop models.
Base model. The base model is either Gokbudak et al.'s patch-transform blender, which directly yields a per-patch affine color transform $A_p$ for each patch $p$, or INRetouch. For INRetouch we fit local affine transforms to its output in a sliding window, the same way a guided filter does, so that both models expose a field of transforms rather than raw pixels. Working in transform space matters for three reasons. Transforms vary smoothly where pixels do not. Spreading a transform cannot invent texture the way pixel inpainting can. And the result stays non-destructive: it is an edit applied to the original image, not a regenerated image. Non-destructive editing is the property JarvisArt obtains by predicting Lightroom parameters, but JarvisArt needs 50K supervised reasoning samples and step-by-step token generation to get it.
Exemplar-calibrated verifier. The exemplar pair is split into $K$ spatial folds. We train $K$ base models, each on $K-1$ folds, and record the true per-patch error on the held-out fold. On a laptop-class GPU this training takes seconds to minutes; One-Shot Image Restoration reports about 30 s for a comparable single-pair training. For each held-out patch we compute cheap features:
- the distance from the patch to its nearest neighbor among the exemplar's training "before" patches, in a fixed Laplacian-band statistics space;
- the disagreement among the $K$ fold models' predicted transforms;
- local gradient energy.
A tiny regressor $g$ maps these features to predicted error. At test time the $K$ fold models also serve as an ensemble, so disagreement is nearly free. The scientific question here is whether held-out regions of one image are a faithful proxy for the shift to a different image. That is not guaranteed, and the experiments below test it directly.
Selective refinement. Patches whose predicted error exceeds a threshold receive confidence $c_p$ near zero. All other patches keep confidence near one. We then solve for a refined transform field
$$ A^\star = \arg\min_{A} \sum_p c_p \lVert A_p - \hat A_p \rVert^2 + \lambda \sum_{(p,q)} w_{pq} \lVert A_p - A_q \rVert^2 , $$
where:
- $\hat A_p$ is the base model's transform for patch $p$;
- $c_p = \exp(-g(\text{features}_p)/\tau)$ is the verifier confidence with temperature $\tau$;
- $w_{pq}$ is an edge-aware similarity between neighboring patches $p$ and $q$ in the target image;
- $\lambda$ sets how strongly transforms spread.
This is one sparse linear system. Solving it with a fixed number of Jacobi sweeps on a coarse patch grid makes "number of refinement iterations" an explicit knob trading quality for latency.
After solving, a rollback rule keeps the original $\hat A_p$ wherever the refined transform raises fold-model disagreement. This mirrors DocIntent's comparison-based rollback at patch granularity and costs microseconds instead of a multimodal model call.
Baselines
The naive reference points are the identity mapping, which leaves the photo unedited, and a single global affine color transform fitted to the exemplar pair. Together they show how much of the edit is spatially varying at all. If selective refinement cannot beat the global transform on mismatched targets, the edit was effectively global and the question is moot for that preset.
The strongest one-shot competitors are INRetouch and Gokbudak et al., each run open loop exactly as published. Each is trained on the same exemplar pair, with the same patch and window settings, under a per-preset tuning budget that is identical across all methods. No reported result was found on Lacuna for INRetouch's PSNR on the Photo Retouching Dataset (PRD); the file says only that it beats StyleGAN, Deep Preset, Neural Preset and Image Analogies, with numbers in a table not reproduced. Its 70 ms per 4K image is the latency figure to beat. No reported number was found on Lacuna for Gokbudak et al. on MIT-Adobe FiveK filter reproduction either. Both baselines will therefore be re-measured under our protocol rather than quoted.
A matched-latency ensemble is the parallel-sampling counterpart. It averages the $K$ fold models' outputs with no verifier and no spreading. It costs the same as our verifier's inference and tests whether ensembling alone explains any gain. Fernando et al. found that sequential refinement beats compute-matched parallel sampling in compositional generation. We do not assume that transfers to retouching.
The supervised upper reference is ABPN, Adaptive Blend Pyramid Network. It reports 37.35 dB PSNR on CRHD-3K, trained on its 3,022 pairs, at about 14 ms for 4K on a Tesla P100. We report how much of the gap between one-shot open-loop and this supervised figure our method closes. We do not expect to beat it.
The expensive closed-loop reference is a scaled-down DocIntent-style loop. A vision-language model compares exemplar and output region by region, then accepts or rolls back each region's transform, for up to three rounds. It is run on a subset because of cost. It provides a quality and latency point for "a strong external critic," and tells us whether the exemplar-calibrated verifier gives up much accuracy.
Three ablations isolate the key ingredients. In the random-flag ablation, the same number of patches is chosen at random rather than by the verifier. The oracle-flag ablation uses true error to choose patches and bounds what better verification could achieve. In the refine-everything ablation, $c_p$ is the same constant for all patches, which tests whether selection matters or plain smoothing explains the gain.
Beating the open-loop baselines means two things. First, a statistically reliable PSNR and LPIPS gain in the high-mismatch strata described below. Second, no loss in the low-mismatch strata, with total latency under twice the open-loop figure. Beating the fallback alternatives means the same high-mismatch gain over the conservative fallbacks discussed next, which use exactly the same flagged patches.
Why not just…?
Why not just revert flagged patches to the identity or to the global transform, with no refinement at all? This is the real threat. If the verifier already says where the model is wrong, the cheapest response is to fall back to something safe. The experiments include exactly this: same verifier, same flags, with spreading replaced by identity or by the global affine transform. Our expectation is that conservative fallback wins for global presets and loses for spatially varying edits such as local exposure balancing or vignetting. In those cases a neighbor's transform is far closer to correct than either fallback. If fallback matches propagation across PRD and CRHD-3K, we will report that the contribution is the verifier alone and drop the spreading step.
Why not just train the one-shot model longer, larger, or with heavier augmentation? Both INRetouch and Gokbudak et al. describe mismatch failures as a matter of what the exemplar contains, not of capacity. More training on the same pair cannot show the model sky if the pair contains no sky. We test this by giving the open-loop baseline the full compute our method spends on fold training, either as a larger model or as longer training with color and geometric augmentation, and comparing in the high-mismatch strata.
Why not just use a vision-language critic loop like Fernando et al. or DocIntent? Those loops work, but each round costs a multimodal model call. Volcano's numbers show that even one critique-revise step roughly doubles latency into seconds. Our closed-loop reference measures how much quality the exemplar-only verifier gives up relative to such a critic. If that gap is large, the right conclusion is a hybrid that calls the vision-language model only on images the cheap verifier rates as globally risky. We would report that rather than claim parity.
Why not just collect paired data and train a supervised model like ABPN? For bespoke personal styles the pairs do not exist, which is the premise of Gokbudak et al. ABPN's 37.35 dB on CRHD-3K is reported as an upper reference to show how far one-shot methods remain from supervised ones, not as a competitor in the one-shot regime.
Proposed Experiments
The main testbed is the PRD used by INRetouch. It applies more than 170 Lightroom presets to 570 MIT-Adobe FiveK raw images, so every target has exact ground truth. For each preset we pick one image as the exemplar pair and evaluate on the rest. To make mismatch measurable, we stratify targets into bins by the distance between their "before"-patch statistics and the exemplar's, and report PSNR, SSIM and LPIPS per bin alongside wall-clock latency at 4K. CRHD-3K provides a local, non-preset retouching test where the supervised ABPN number anchors the scale: one pair is the exemplar and the remaining pairs are the test set. Gokbudak et al.'s algorithmic-filter setup on MIT-Adobe FiveK (Gaussian, unsharp mask, bilateral) adds edits with known structure.
Four checks decide the outcome.
First, verifier calibration. We measure the Spearman correlation between predicted and true per-patch error on actual target images. We compare it against the same correlation on held-out exemplar folds, to measure how much the self-calibration proxy degrades under real distribution shift. If correlation on targets falls near zero, the core claim fails regardless of downstream PSNR.
Second, gap closure. In the highest-mismatch bin, we measure what fraction of the open-loop-to-oracle-flag PSNR gap selective refinement recovers, compared with random flags and with the two fallbacks.
Third, the iteration curve. We plot quality against Jacobi sweeps and against a second verify-and-refine round. If quality saturates after the first pass, that indicates that exemplar-conditioned editing, unlike compositional generation, needs essentially one corrective pass.
Fourth, safety. We check that low-mismatch bins lose no quality and that rollback triggers rarely on them. This is the patch-level analogue of DocIntent's check that clean documents are left untouched.
References
- INRetouch: Context Aware Implicit Neural Representation for Photography Retouching
- One-shot Detail Retouching with Patch Space Neural Transformation Blending
- Iterative Refinement Improves Compositional Image Generation
- Idea2Img: Iterative Self-refinement with GPT-4V for Automatic Image Design and Generation
- DocIntent: Answerability-Guided Agentic Restoration for Real-World Document Visual Question Answering
- Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision
- Meta-Reflection: A Feedback-Free Reflection Learning Framework
- JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent
- One-Shot Image Restoration
- ABPN: Adaptive Blend Pyramid Network for Real-Time Local Retouching of Ultra High-Resolution Photo