Amplification-Weighted Interface Discriminators for Composable Restoration Cascades

When a photo is damaged by several things at once, such as blur, noise and low resolution, a common fix is to chain specialist models, one per problem. These chains fail in a predictable way. Each stage leaves small artifacts, and the next stage cannot tell them apart from real image detail, so it sharpens and enlarges them. The usual responses are to abandon the chain for one joint network, or to retrain the whole chain end to end. Both give up what made chains attractive: a toolbox of small models that can be recombined for whatever mix of damage an image has.

This note proposes a different fix. Each upstream model is trained so that its outputs look, at every spatial frequency, like the inputs the downstream model was trained on. The judge is an adversarial discriminator, a network trained to tell two sets of images apart, which here compares the upstream outputs with the downstream model's training inputs. It works on several scales of an image pyramid. Its per-scale penalties are weighted by how strongly the downstream model is measured to amplify errors in each frequency band. The bet is that error compounding in cascades is mostly a frequency-selective mismatch at the handoff between stages. If so, a scale-resolved adversarial loss aimed at that handoff should keep chains accurate without joint retraining, including for tool orderings never seen together in training.

Core Claim

The failure mechanism is well documented. Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning builds RL-Restore, a learned agent that picks a sequence of small restoration networks from a toolbox. It names the "middle state" problem: sharpening a blurred image magnifies residual noise and produces an intermediate image outside the next tool's training distribution. Deep Residual Network for Joint Demosaicing and Super-Resolution shows that downstream models treat upstream interpolation errors as true high-frequency content. On Error Propagation of Diffusion Models proves that, for iterative denoising, the factor linking one step's error to the final error is at least one. Taken together, these papers motivate the proposal: they establish that compounding happens and that it depends on how each stage responds to perturbations.

What they do not do is resolve the amplification by frequency. That band-level view is what points to a targeted fix. Let $D$ be a downstream tool, $x$ an input from its training distribution, and $P_k$ a projection onto band $k$ of a Laplacian pyramid (the image split into octave-wide frequency bands). We define the band amplification of $D$ as

$$a_k = \mathbb{E}_{x,\delta}\,\frac{\lVert D(x + P_k\delta) - D(x)\rVert}{\lVert P_k \delta\rVert},$$

where $\delta$ is a small random perturbation. This is the diffusion paper's amplification factor, measured per band for a restoration module instead of per sampling step.

An upstream tool $U$ is then trained with its usual fidelity loss plus an interface loss. The interface loss is a sum over scales of adversarial losses from discriminator heads $d_k$, each weighted by $w_k \propto a_k$. Each head compares band $k$ of $U$'s outputs with band $k$ of samples from $D$'s training-input distribution. No paired intermediate ground truth is needed; only $D$'s training inputs are required.

The claim is that amplification-weighted, scale-resolved interface matching does three things. It reduces how fast error grows with chain length. It makes separately trained tools composable. It beats both uniform multi-scale and single-scale discrimination, because it spends adversarial pressure on the bands where the next stage actually magnifies error.

Research Program

The discriminator design follows the explicit image-pyramid line of work. Line Art Colorization With Concatenated Spatial Attention and GigaGAN in Scaling up GANs for Text-to-Image Synthesis run separate discriminators at full, half and quarter resolution, so coarse heads judge layout while fine heads judge edges. We reuse that structure but change what the heads judge. They measure distance to a downstream module's input distribution rather than distance to natural images, and they are weighted by measured amplification rather than fixed equally.

We also test a feature-pyramid variant based on Projected GANs Converge Faster, which attaches discriminators to frozen pretrained features with random channel and scale mixing and converges up to 40 times faster than StyleGAN2. That paper also acts as a constraint. It reports that classification-pretrained features favour foreground objects over background, producing sharp faces on blurred backgrounds. Grain and ringing live largely in backgrounds and flat regions, so we expect the Laplacian-pyramid version to fit this task better. The comparison is part of the study.

The work has three parts. First, we measure the band amplification profiles $a_k$ for a toolbox of lightweight networks covering deblurring, denoising, JPEG removal and super-resolution, built in the RL-Restore style. We check that these profiles are stable across inputs and that they predict which bands of upstream error dominate the final error. If the profiles do not predict final error, the weighting idea fails and we say so.

Second, we train each tool once with a small set of interface discriminators, one per plausible successor. For a successor-agnostic variant, we weight each band by the maximum amplification over all successors. Each tool keeps its own fidelity loss, and for the final tool in a chain that is a full-reference loss against the clean image.

Third, we compose tools into chains of length one to four, including orderings and tool pairs never trained together, and measure how the error grows.

The evaluation must report full-reference metrics, not just perceptual ones. Is Nano Banana Pro a Low-Level Vision All-Rounder? is a direct warning here. It finds that generative foundation models win on no-reference scores such as NIQE and NIMA while trailing specialist models by over 4 dB PSNR on DIV2K super-resolution and 5–8 dB on denoising. An adversarial interface loss could fall into the same trap by inventing texture that matches the downstream distribution but not the scene. We therefore report PSNR, SSIM and LPIPS together and treat any PSNR loss as a cost, not something to hide.

Baselines

The naive reference is a plain cascade of independently trained tools, each fit with L1 loss on its own task and chained without adaptation. This is the setting where the middle-state problem appears. It anchors every error-growth curve.

The strongest existing method on the fixed deblur-then-super-resolve question is the Gated Fusion Network (GFN) from Gated Fusion Network for Joint Image Deblurring and Super-Resolution. GFN avoids cascading altogether by running deblurring and super-resolution as parallel branches in low-resolution feature space, merged by a learned gate. It reports 27.74 dB PSNR on LR-GOPRO, against 26.44 dB for the joint ED-DSRN baseline, and runs about 116 times faster than stacking EDSR with DeepDeblur. We retrain GFN with its published recipe on the same LR-GOPRO split. Our deblur-to-super-resolution chain gets a matched parameter count and the same number of training images and iterations. Our tools are trained separately; GFN is trained jointly on the pair. On this fixed pair, matching GFN within about 0.3 dB counts as success, because the whole point of our tools is that they were never trained for this specific combination. Clearly exceeding 27.74 dB is not expected and is not required.

For low-light enhancement followed by zooming, the reference is BrZoNet from Unveiling Details in the Dark. It separates illumination from reflectance and reports 22.79 dB PSNR at ×2 on RELLISUR, a 7.1% PSNR gain and 36.8% LPIPS gain over 13 methods. We treat it the same way as GFN.

For variable-length mixed-distortion chains on DIV2K, the strongest competitor is RL-Restore with joint toolchain training. The file reports only relative figures for it: 82.2% fewer computations than traditional models, about one-third of VDSR's parameters, and a 0.2–0.3 dB PSNR gain over baselines on severe unseen distortions. No absolute PSNR on Lacuna is available for its DIV2K mixed-distortion setting, so we rerun it on our distortion protocol with its toolbox size and agent. We compare two ways of filling that toolbox: tools trained with our interface loss, and tools trained with its joint procedure. Everything else, including the agent, is held fixed. Beating it means higher PSNR on severe and held-out distortion mixes at equal tool budget. It also means doing so without joint retraining when a tool is added or swapped.

The all-in-one AMIRNet from All-in-one Multi-degradation Image Restoration Network via Hierarchical Degradation Representation is included as the monolithic reference at matched parameters.

The ablations isolate the key ingredient. We compare amplification-weighted heads against three alternatives: uniform-weighted multi-scale heads, a single full-resolution PatchGAN-style discriminator like the one used in Modular Degradation Simulation and Restoration for Under-Display Camera, and a projected feature-pyramid discriminator. All share the same adversarial loss weight and tuning budget. As a further ablation, the discriminator's real samples switch from the downstream training distribution to natural clean images. This tests whether aiming at the handoff matters, as opposed to realism in general.

Why not just…?

Why not just train the whole chain end to end, as RL-Restore does? For a fixed chain this is a real threat and may win, so we run it as an upper bound on every fixed-pair benchmark. The case for our method rests on composability. With a toolbox of a dozen tools and chains up to length four, the number of orderings makes joint training of each chain impractical. Adding a new tool would force retraining of its neighbours. The decisive experiment is on held-out orderings and swapped-in tools. If jointly trained tools transfer as well as interface-trained ones to chains they were never trained in, the proposal loses its reason to exist.

Why not just train each downstream tool on its predecessor's outputs? This degradation-aware augmentation is the simplest alternative and addresses the same distribution mismatch from the other side. Its costs are that it ties each tool to specific predecessors, just as joint training does, and that it teaches the downstream tool to tolerate artifacts rather than asking the upstream tool not to produce them. We run it with the same pairs and compute. If it matches our method on held-out orderings, we will report that the adversarial component is unnecessary.

Why not just add LPIPS or L1 at the intermediate image? Both need a paired clean intermediate. Neither knows which bands the next stage amplifies. Pixel losses also average over plausible outputs and blur detail. We include an LPIPS-at-interface baseline, where the reference is a synthesized intermediate with only the remaining degradations. If it closes the error-growth gap, the frequency-weighted adversarial loss is not buying anything.

Why not just use a joint network like GFN or BrZoNet and skip cascades? For the specific pairs they were designed for, they are strong, and we do not expect to beat them there. They cover one fixed combination each, however. RL-Restore's results suggest dynamic chains help on severe, unseen distortion mixes, which is the regime where we compete.

Why not a single-scale discriminator? A single full-resolution patch judge cannot separate low-frequency colour and brightness drift from high-frequency ringing. The amplification profiles let us predict when that matters: when the downstream tool amplifies coarse and fine bands differently. The single-scale ablation tests this prediction directly.

Proposed Experiments

The first decisive check is the slope of error against chain length on DIV2K mixed blur, noise and JPEG, extended to four stages with super-resolution. We plot final-output PSNR and the per-band residual energy of the output error for the plain cascade, joint training, and our tools. The hypothesis predicts two things: a flatter slope for our tools, and the largest residual reductions in exactly the bands where $a_k$ is largest. A flatter slope without that band pattern would mean the mechanism is not what we claimed.

The second check is composability. Tools are trained on a subset of pairwise handoffs and tested on unseen orderings and on a newly added tool. Here we compare against joint training and against degradation-aware augmentation.

The third check covers the fixed-pair benchmarks, LR-GOPRO against GFN's 27.74 dB and RELLISUR ×2 against BrZoNet's 22.79 dB. We report PSNR, SSIM and LPIPS so that a perceptual gain cannot hide a loss in fidelity, following the warning from the Nano Banana Pro evaluation.

The fourth check is the head ablation described above: weighted, uniform, single-scale, projected-feature, and natural-image targets.

Two outcomes would falsify the approach. The first is that amplification profiles fail to predict which bands dominate the final error. The second is that augmentation-trained downstream tools match our held-out-ordering results.