Blind Motion Deblurring With Pixel-Wise Kernel Estimation via Kernel Prediction Networks

Guillermo Carbajal, Patricia Vitoria, José Lezama, Pablo Musé

Introduction

Motion deblurring is a fundamental challenge in computer vision, particularly when it is "blind"—meaning both the sharp latent image and the blur kernel are unknown. While modern deep learning has shifted toward end-to-end "black-box" models that map blurry pixels directly to sharp ones, these approaches often struggle with generalization. Because they do not explicitly model the physical process of image degradation, they frequently fail when encountering real-world blur that differs from their training data.

This paper introduces a hybrid framework that combines the interpretability of classical physics-based models with the representational power of deep learning. By utilizing a Kernel Prediction Network (KPN) to estimate spatially-varying kernels and an unrolled optimization network for deconvolution, the authors achieve state-of-the-art results that remain grounded in the physical reality of motion blur.

The Challenge of Non-Uniform Blur

In real-world photography, blur is rarely uniform. Camera shake, moving objects, and depth variations result in different pixels being blurred in different directions and magnitudes. Estimating a unique blur kernel for every pixel is computationally prohibitive, as a dense kernel field represents a massive parameter space.

To address this, the authors propose a two-stage pipeline designed to handle non-uniform blur efficiently while maintaining a model-driven approach.

Proposed Architecture: A Hybrid Approach

1. Kernel Prediction Network (KPN)

Instead of predicting a full $K \times K$ kernel for every pixel, the KPN employs a "low-rank" representation. It predicts a small set of image-adaptive basis kernels and a corresponding set of per-pixel mixing coefficients. The final kernel at any given pixel is calculated as a weighted sum of these bases.

This basis decomposition serves two purposes: it drastically reduces the number of parameters the network must estimate, and it helps resolve the "aperture problem." In low-texture or flat regions (like a clear sky) where local blur is hard to discern, the network can still infer an accurate kernel by relying on the global basis kernels derived from sharper, high-contrast areas of the image.

2. Unrolled Optimization for Deconvolution

The estimated kernel field is passed to a second stage that performs non-blind deconvolution. This stage "unrolls" a linearized version of the ADMM (Alternating Direction Method of Multipliers) optimization algorithm. By treating the deconvolution as a deep plug-and-play restorer, the model uses a learned denoiser as a prior to reconstruct the sharp image based on the predicted kernels.

Crucially, the KPN and the deconvolution network are jointly fine-tuned. This end-to-end training allows the restoration stage to become robust to the inevitable small errors or artifacts produced during the kernel estimation phase.

Experimental Setup and Datasets

To train and evaluate the model, the authors developed the Segmentation-Based Dataset for Deblurring (SBDD). Using 118,000 images from the COCO dataset, they applied random camera-shake kernels to segmented objects, creating a synthetic environment that mimics complex, non-uniform real-world motion.

The model was further validated on rigorous real-world benchmarks, including the Köhler, Lai, and RealBlur datasets. Unlike synthetic tests, these benchmarks consist of actual photographs, providing a true measure of how well a model handles the nuances of physical sensors and natural light.

Results and Discussion

Generalization and Physical Grounding

By forcing the network to estimate a kernel before performing deconvolution, the authors impose a physical constraint that acts as a regularizer. This prevents the model from "hallucinating" details that do not align with the geometry of the blur. Consequently, when trained on the synthetic SBDD dataset, the model (J-MKPD) generalized to real-world photos significantly better than pure end-to-end models like MIMO-UNet or MPRNet, which often require dataset-specific tuning.

Rethinking Evaluation Metrics

The authors also address a critical issue in deblurring research: the reliance on PSNR. Reference-based metrics often penalize sharp restorations if they are slightly misaligned with a "ground truth" that may itself contain minor artifacts. The study suggests that reference-free perceptual metrics, such as Blur Strength and Sharpness Index, often provide a more accurate reflection of true restoration quality and visual clarity.

Limitations and Future Directions

Despite its strengths, the framework faces specific constraints:

  • Kernel Size: The model uses a fixed support size (e.g., $33 \times 33$ pixels). Extremely large blurs from high-speed motion may exceed this window.
  • Saturation: Very bright light streaks can confuse the kernel estimator, sometimes leading it to predict a "delta" (no-blur) kernel in saturated areas. While data augmentation (SBDD-LS) helps, this remains a challenge for night-time photography.

Future work may involve extending this pixel-wise kernel estimation to joint tasks like super-resolution or using the estimated kernels as diagnostic tools to interpret restoration failures in complex scenes.

Summary updated

BibTeX
@article{carbajal2023blind,
  author = {Guillermo Carbajal and Patricia Vitoria and José Lezama and Pablo Musé},
  title = {{Blind Motion Deblurring With Pixel-Wise Kernel Estimation via Kernel Prediction Networks}},
  journal = {IEEE Trans. Computational Imaging},
  year = {2023},
  doi = {10.1109/tci.2023.3322012},
}
Figure