Some deblurring networks undo blur inside a learned feature space instead of on raw pixels. They extract features from the blurry photo, apply a classical closed-form inverse filter to those features, and decode a sharp image. A Wiener filter is the standard example: it divides out the blur in the frequency domain while backing off wherever noise would dominate. These networks beat pixel-space methods on standard benchmarks, but they are trained with a single loss on the final image. The multi-scale refinement stages inside them receive no direct guidance about what they should be producing. Deep supervision, meaning auxiliary losses attached to intermediate layers, is the usual fix for unguided internal stages. Applied naively, though, it asks early stages for outputs they cannot yet produce. Deep Trajectory Supervision: Deep Supervision Strikes Back calls this an "impedance mismatch."
In deblurring we do not have to guess what an intermediate stage can produce, because the physics says so. For a known blur kernel and noise level, the Wiener gain at each spatial frequency measures how much of the true image at that frequency is recoverable from the blurry observation. We propose to supervise each intermediate stage of a feature-space deconvolution network only on the frequencies that are reliably recoverable at a threshold that loosens with depth. Early stages reproduce the high-confidence bands. Later stages take on progressively harder bands near the kernel's zeros. The final output is judged on everything. The schedule is computed per training sample from the kernel and noise. It adds no hyperparameters beyond one threshold sequence, and it costs nothing at inference because the auxiliary heads are discarded.
Core Claim
A resolution pyramid of downsampled ground truth is the default target for multi-scale supervision. It is the wrong target for deblurring, because what is recoverable depends on the kernel, not on resolution. A horizontal motion kernel destroys a stripe of frequencies that reaches far into the low-frequency range. A downsampled target still demands those frequencies from the coarsest stage, which forces it to hallucinate or ring. We claim that replacing resolution-defined targets with Wiener-reliability-defined targets gives two benefits over both terminal-only training and pyramid supervision. It gives sharper images with less ringing, and the gains should concentrate where kernels have near-zeros and noise is high. The supervision also does not create the procedural brittleness under shifted test conditions that When Intermediate Supervision Doesn't Help: Evidence from Recurrent CNNs documents for step-by-step targets.
Concretely, let $X(\omega)$ be the Fourier transform of the sharp training image. Let $K(\omega)$ be that of the kernel, $\sigma^2$ the noise variance, and $S(\omega)$ a fixed natural-image power spectrum. The target for stage $i$ of $N$ is
$$T_i = \mathcal{F}^{-1}\!\left[ M_i(\omega)\, X(\omega) \right], \qquad M_i(\omega) = \frac{|K(\omega)|^2}{|K(\omega)|^2 + \lambda_i \,\sigma^2 / S(\omega)},$$
Here $\lambda_1 > \lambda_2 > \dots > \lambda_N = 0$, so $M_N = 1$ and the last stage sees the full sharp image. With large $\lambda_i$, the mask $M_i$ keeps only frequencies whose signal-to-noise ratio after deblurring is high. Every intermediate target is the true image projected onto a band that the observation actually contains. The stage is therefore never asked for information that has not reached it. For Gaussian kernels, $|K|$ falls off monotonically with radius, so $M_i$ is nearly a radial low-pass and our method should reduce to pyramid supervision. This gives a built-in falsification test.
Research Program
The base network follows Deep Wiener Deconvolution: Wiener Meets Deep Learning for Image Deblurring and its extension DWDN: Deep Wiener Deconvolution Network for Non-Blind Image Deblurring. Both extract features, apply a closed-form feature-space Wiener filter with a learned noise-to-signal estimate, and refine through a multi-scale cascaded encoder-decoder trained only on the final image. These papers supply the method. We keep their operator and change only what the internal stages are trained to do. We attach a light image head to each refinement stage. Each head's output is compared to $T_i$ under an L1 loss plus a frequency-domain L1 term restricted to the support of $M_i$.
To separate "what band" from "what resolution," we build two variants. In the first, a same-resolution cascade, all stages run at full resolution and differ only in their mask. In the second, a pyramid cascade, stage $i$ also runs at a coarser resolution and $T_i$ is anti-aliased down to it. We then extend the feature-space Wiener step itself to act at every scale, using the kernel resampled to that scale. The point of this extension is to test whether per-scale inversion plus reliability supervision addresses the "semantic drift" problem, where the inverse filter amplifies representation errors in unsupervised features.
We then examine the regimes where the linear model breaks. INFWIDE: Image and Feature Space Wiener Deconvolution Network for Non-blind Image Deblurring in Low-Light Conditions handles saturated low-light images with a parallel pixel branch and a terminal reblurring loss. We add reliability-masked supervision to its feature branch only. We also test whether masking out clipped pixels when computing $T_i$ helps. Since saturation violates $y = x * k + n$, the masks are least trustworthy there, and we expect this to be the hardest case for the method.
The mechanism also bears on two other lines of work. T-MLP: Tailed Multi-Layer Perceptron for Level-of-Detail Signal Representation shows that coarse-to-fine auxiliary outputs act as natural denoisers, but it fits clean signals, so its frequency split is generic rather than tied to a degradation. Our masks are sample-specific and anisotropic. Deep Trajectory Supervision softens intermediate targets on a curvature schedule tuned as a hyperparameter for classification. We derive the softening from the measurement physics instead.
Baselines
The naive reference point is the base feature-space Wiener network trained with the terminal loss only, as in DWDN. We retrain it ourselves on the same synthetic pairs, built from the Berkeley Segmentation Dataset and the Waterloo Exploration Database with motion kernels and 0–5% Gaussian noise. It gets the same epochs, optimizer, augmentation, and learning-rate search as every other variant. The file gives only relative figures for DWDN: gains "up to 2.09 dB over competing methods" on Levin, and a 2.57 dB margin over IRCNN at 1% noise for the NeurIPS version. We found no absolute Levin PSNR for DWDN on Lacuna. Our retrained number is therefore the anchor, and we will report it next to the published relative margins as a sanity check.
The strongest competitor, and the one that most directly threatens the claim, is the same network with standard multi-scale deep supervision. Each stage gets an L1 loss against downsampled ground truth, as in Stacked Deconvolutional Network for Semantic Segmentation, which reached 86.6% mean IoU on PASCAL VOC 2012 with supervision at every upsampling stage. We tune its per-stage weights with the same search budget we give our $\lambda_i$ schedule. We will also run a variant that applies Deep Trajectory Supervision's exponential weight schedule to the pyramid targets, because that is the best available generic remedy for the impedance mismatch.
For external reference points, we compare with CiDeR from Leveraging Classic Deconvolution and Feature Extraction in Zero-Shot Image Restoration. It reports 30.63 dB PSNR and 0.8956 SSIM on Levin without paired training. That is a different regime, and we use it for context only. On NightShot, the bar is INFWIDE's reported 26.51 dB PSNR, against 25.74 dB for the next competitor. The key ablation isolates the ingredient: the same-resolution cascade with radial low-pass masks of matched bandwidth versus Wiener-reliability masks. Only the mask shape differs between the two.
To count as beating the strongest baseline, our method must exceed pyramid supervision on Levin and Sun et al. at 3–5% noise, averaged over three seeds with non-overlapping spread. The gain must also be larger on motion kernels than on Gaussian kernels. A uniform gain would suggest generic regularization rather than our mechanism. Finally, the method must reduce error measured specifically in low-reliability frequency bands.
Why not just…?
Why not just add ordinary pyramid deep supervision to DWDN? This is the real threat. A downsampled target is already a crude low-to-high frequency schedule, and it may capture most of the benefit. The same-resolution cascade isolates mask shape from resolution, and the Gaussian-versus-motion-kernel split tests the prediction that the two methods coincide when the kernel is radially monotone. If pyramid supervision matches us on motion kernels at high noise, the kernel-aware part of the idea is unnecessary. We would report that and fall back to the simpler method.
Why not just weight the terminal loss by the Wiener mask instead of adding heads? A frequency-weighted output loss changes what the final image optimizes for, but it gives the internal stages no guidance. Leaving internal stages unguided is exactly the gap that DWDN and INFWIDE leave open. We run this variant directly, with the same masks applied to the output only. If it matches our method, then intermediate supervision contributes nothing, and the effect lies in the loss weighting alone.
Why not just train terminal-only for longer or with a wider network? Deep Trajectory Supervision reports over 2x faster time-to-target on DeiT-III-S from scheduled intermediate losses, which suggests some gains are about optimization rather than capacity. We report PSNR against wall-clock time and include a terminal-only model with a matched parameter count and twice the training budget. This separates a convergence speedup from a better final solution.
Why not do deconvolution in pixels and then denoise with a supervised U-Net? The file's evidence says pixel-space inversion entangles amplified noise with edges. DCLS-SR's closed-form feature-space least squares reaches 28.99 dB on DIV2KRK ×4 against DANv2's 28.74 dB, and CiDeR outperforms pixel-domain Richardson-Lucy. We include one such two-stage pipeline at 1% and 5% noise as a floor. We do not expect it to compete.
Proposed Experiments
Training uses BSD and Waterloo synthetic pairs. Evaluation covers the Levin et al. benchmark (32 images, 8 kernels) and Sun et al. at noise levels of 1, 3, and 5%, plus NightShot for low-light saturation. We report PSNR and SSIM, and we add two diagnostics that target the mechanism. The first is error decomposed into Wiener-reliability bands of the ground-truth kernel. The second is a ringing measure: energy of the restored image's residual in frequencies where $|K|$ is near zero.
The first decisive check is the interaction test. The advantage over pyramid supervision should grow with noise and with the anisotropy of the kernel, and it should be near zero for Gaussian blur. The second decisive check addresses the brittleness warning in When Intermediate Supervision Doesn't Help: Evidence from Recurrent CNNs. At test time we feed the network perturbed kernels (additive kernel noise, small rotations and scale changes) and spatially varying blur approximated patchwise. If reliability-supervised models degrade faster than terminal-only models under this mismatch, intermediate targets are teaching a procedure tied to exact kernels. That would be a publishable negative result about deep supervision in physics-based restoration.
A secondary experiment carries the masks over to DCLS-SR on DIV2KRK ×4, replacing the Wiener gain with the constrained-least-squares gain from its estimated kernel. The bar is its reported 28.99 dB PSNR, and this experiment tests whether the idea survives when the kernel is estimated rather than known.
References
- Deep Trajectory Supervision: Deep Supervision Strikes Back
- When Intermediate Supervision Doesn’t Help: Evidence from Recurrent CNNs
- Deep Wiener Deconvolution: Wiener Meets Deep Learning for Image Deblurring
- DWDN: Deep Wiener Deconvolution Network for Non-Blind Image Deblurring
- INFWIDE: Image and Feature Space Wiener Deconvolution Network for Non-Blind Image Deblurring in Low-Light Conditions
- T-MLP: Tailed Multi-Layer Perceptron for Level-of-Detail Signal Representation
- Stacked Deconvolutional Network for Semantic Segmentation
- Leveraging Classic Deconvolution and Feature Extraction in Zero-Shot Image Restoration