Restoring Images with Spatially Varying Blur

23 concepts24 papers1 proposal

Standard image restoration and super-resolution algorithms often rely on an elegant mathematical simplification: the blur across an entire image is assumed to be uniform. Under this assumption, degradation is modeled as a global convolution ($I_{\text{blur}} = I_{\text{sharp}} * K$), where a single blur kernel $K$ applies identically to every pixel.

In the real world, this assumption falls apart. As noted by Pei Wang et al. et al. in Real-world image deblurring: challenges and prospects, real-world blur is inherently non-uniform and spatially varying. This occurs because objects move at different speeds and trajectories relative to the camera, lenses exhibit space-variant optical aberrations like coma or astigmatism where quality degrades from the center to the edges (Patrick Müller et al. et al. in Classification robustness to common optical aberrations), and varying scene depths create localized out-of-focus bokeh effects (Longguang Wang et al. et al. in Unsupervised Degradation Representation Learning for Blind Super-Resolution).

When a global restoration method is applied to these images, it fails catastrophically—either under-deblurring moving objects or over-sharpening naturally soft backgrounds.

The Mathematical Bottleneck

Why is spatially varying blur so difficult to correct? When the blur kernel changes based on spatial coordinates ($K(x, y)$), the operation is no longer a simple convolution but a coordinate-dependent matrix multiplication. Inverting this operation directly requires solving massive, computationally prohibitive systems of equations. To make restoration practical, researchers have developed mechanisms to efficiently estimate, simplify, and invert these localized degradation fields.

Pixel-Specific and Localized Kernel Prediction

One intuitive way to handle non-uniform blur is to predict a distinct kernel for every pixel or local region. Instead of estimating a single global matrix, deep networks can output a dense "kernel map."

For instance, in the context of single-image super-resolution, Jianrui Cai et al. et al. introduced the Laplacian Pyramid based Kernel Prediction Network (LP-KPN) in Toward Real-World Single Image Super-Resolution: A New Benchmark and a New Model. LP-KPN handles non-uniform degradations by predicting per-pixel kernels efficiently across different scales of an image pyramid, allowing the network to apply localized corrections without blowing up computational costs.

Similarly, Soo Ye Kim et al. et al. developed KOALAnet in KOALAnet: Blind Super-Resolution Using Kernel-Oriented Adaptive Local Adjustment. By explicitly estimating spatially-variant kernels alongside local feature adjustments, the network learns to distinguish between unwanted motion degradation and intentional aesthetic blur (like bokeh), preventing the over-sharpening artifacts that plague global methods.

[Blurred Input Image] 
       │
       ▼
[Deep Neural Network] ──► [Dense Spatial Kernel Map K(x,y)]
       │                               │
       └───────────► [Per-Pixel] ◄─────┘
                  [Adaptation Layer]
                       │
                       ▼
             [Sharp Restored Image]

Discretizing the Blur Field via Segmentation

If estimating a unique kernel for every single pixel is too dense or noisy, another approach is to treat the continuous blur field as a collection of discrete layers.

In Learning to Estimate and Remove Non-uniform Image Blur, Florent Couzinie-Devy et al. et al. framed non-uniform blur estimation as a multi-label segmentation problem. By partitioning the image into distinct regions or layers based on blur strength, the authors could apply localized optimization techniques to separate a moving object from a static background, making the subsequent deblurring step highly targeted and computationally manageable.

Transform-Domain and Latent Space Modeling

Instead of operating directly on raw pixel space, some frameworks map the image and the degradation into alternative domains where spatially varying operators become easier to untangle.

In classical optimization, Yi Zhang 0020 et al. et al. introduced the Double Discrete Wavelet Transform (DDWT) framework in Blur Processing Using Double Discrete Wavelet Transform. The core insight is that the DDWT can simultaneously sparsify both the blurred image and its spatially varying blur kernel. Moving the problem into the wavelet domain allows for efficient, non-iterative estimation and deblurring of complex object motion.

In modern generative modeling, this concept translates to latent spaces. Lingshun Kong et al. et al. proposed DeblurDiff in DeblurDiff: Real-Word Image Deblurring with Generative Diffusion Models, which integrates a Latent Kernel Prediction Network (LKPN) with generative diffusion models. Rather than predicting complex spatial kernels in high-resolution pixel space, the network iteratively estimates spatially variant kernels within the lower-dimensional latent space of a diffusion model, providing structural guidance that keeps the generative process anchored to the true underlying geometry of the scene.

Bypassing Inversion via Linearization and Homogenization

Even if you successfully estimate the spatially varying kernels, inverting them mathematically remains an algorithmic challenge. Modern research focuses on bypassing or simplifying this inversion step entirely.

To solve the optimization problem without expensive matrix inversions, Andres Almansa et al. et al. developed a linearized Alternating Direction Method of Multipliers (ADMM) framework within a Plug-and-Play (PnP) pipeline in Provably Convergent Plug & Play Linearized ADMM, applied to Deblurring Spatially Varying Kernels. By linearizing the forward model, the authors eliminated the need for costly matrix inversions, making PnP denoisers computationally practical for spatially varying kernels.

Alternatively, instead of trying to invert the complex blur directly, one can transform it into a simpler, uniform problem. In Learning Correction Filter via Degradation-Adaptive Regression for Blind Single Image Super-Resolution, Hongyang Zhou et al. et al. proposed learning an unsupervised "Correction Filter." This filter adaptively homogenizes the unknown, spatially-varying blur across the frame, converting it into a known, uniform downsampling format (like standard bicubic downscaling). Once the non-uniform degradation is standardized, off-the-shelf super-resolution models can restore the image perfectly without needing any retraining or specialized architecture.

Key Foundational Work

Blur Processing Using Double Discrete Wavelet Transform (2013) — contributed 1 concept elements to this direction.
Nonparametric Blind Super-resolution (2013) — contributed 1 concept elements to this direction.
Learning to Estimate and Remove Non-uniform Image Blur (2013) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_spatially_varying_blur_e683b08b,
  title = {spatially varying blur},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spatially-varying-blur/rdr_e683b08b0abe47349bc8dd960cbb02f2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_spatially_varying_blur_e683b08b_v1,
  title = {spatially varying blur},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spatially-varying-blur/rdr_e683b08b0abe47349bc8dd960cbb02f2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake