Deep Image Deblurring: A Survey

Kaihao Zhang, Wenqi Ren, Wenhan Luo, Wei-Sheng Lai, Björn Stenger, Ming-Hsuan Yang, Hongdong Li

Introduction

Image deblurring is a fundamental yet "ill-posed" problem in computer vision. It involves recovering a sharp latent image from a blurry observation caused by factors such as camera shake, object motion, or defocus. Traditionally, this was modeled as a deconvolution problem where a sharp image is convolved with a blur kernel. However, real-world blur is often non-uniform and spatially varying, making traditional mathematical kernels insufficient for complex scenes.

Examples of real-world blur types: (c) Moving object blur and (d) Mixed blur.

Taxonomy of Deep Deblurring Methods

The field has shifted from traditional inverse filtering to deep learning-based end-to-end mapping. This survey categorizes modern methods into several architectural families:

  • Multi-scale and Scale-recurrent Networks: These architectures adopt a coarse-to-fine strategy, processing the image at multiple resolutions to handle large-scale blur before refining high-frequency details.
  • Generative Adversarial Networks (GANs): By employing a discriminator, GAN-based models like DeblurGAN-v2 prioritize "photo-realism" over simple pixel-wise accuracy, helping to recover textures that appear sharper to human observers.
  • Video Deblurring: Unlike single-image methods, video-based approaches leverage temporal redundancy. They often use Recurrent Neural Networks (RNNs) or 3D convolutions to align neighboring frames and aggregate information across time to "borrow" sharp details from adjacent frames.

Datasets and Benchmarking

The transition to deep learning has been fueled by the availability of large-scale datasets. The survey reviews several pivotal benchmarks:

  • Synthetic Data: The GoPro and HIDE datasets remain the gold standards for training. They are generated by filming high-speed video (240fps+) and averaging frames to simulate realistic motion blur.
  • Real-world Data: To address the "domain gap," datasets like RealBlur use specialized hardware (beam-splitters) to capture perfectly aligned blurry and sharp image pairs in the real world, accounting for true sensor noise and optical artifacts.

Key Insights: The Perception-Distortion Tradeoff

A critical finding in recent research is the fundamental tension between mathematical fidelity (measured by Peak Signal-to-Noise Ratio, or PSNR) and perceptual quality. Models optimized for PSNR often produce results that are mathematically accurate but appear "over-smoothed" to the human eye. In contrast, models using perceptual or adversarial losses achieve higher visual sharpness but may introduce minor "hallucinated" artifacts that weren't in the original scene.

Performance and Future Directions

While state-of-the-art (SOTA) methods have improved PSNR on the GoPro dataset from early results of ~24dB to over 32dB, significant challenges remain:

  1. The Generalization Gap: Models perform excellently on synthetic datasets but often fail when faced with real-world blur due to differences in noise patterns and sensor behavior.
  2. Computational Efficiency: Processing high-resolution (4K/8K) images remains a bottleneck. Even on high-end GPUs, generating a 4K deblurred image can take up to 30 seconds, limiting mobile and real-time applications.
  3. Unpaired Learning: A promising future direction involves training models on unpaired blurry and sharp images (using CycleGAN-style approaches) to reduce the reliance on expensive, perfectly aligned ground-truth datasets.

Summary updated

Abstract

Image deblurring is a classic problem in low-level computer vision with the aim to recover a sharp image from a blurred input image. Advances in deep learning have led to significant progress in solving this problem, and a large number of deblurring networks have been proposed. This paper presents a comprehensive and timely survey of recently published deep-learning based image deblurring approaches, aiming to serve the community as a useful literature review. We start by discussing common causes of image blur, introduce benchmark datasets and performance metrics, and summarize different problem formulations. Next, we present a taxonomy of methods using convolutional neural networks (CNN) based on architecture, loss function, and application, offering a detailed review and comparison. In addition, we discuss some domain-specific deblurring applications including face images, text, and stereo image pairs. We conclude by discussing key challenges and future research directions.
BibTeX
@article{zhang2022deep,
  author = {Kaihao Zhang and Wenqi Ren and Wenhan Luo and Wei-Sheng Lai and Björn Stenger and Ming-Hsuan Yang and Hongdong Li},
  title = {{Deep Image Deblurring: A Survey}},
  journal = {International Journal of Computer Vision},
  year = {2022},
  url = {https://openreview.net/forum?id=nOXFQ8ustG},
}
Figure