TL;DR
This paper introduces a "Split-and-Merge" algorithm to improve motion deblurring results from the DeblurGAN model. By dividing blurry images into smaller regions, deblurring them individually based on their specific blur levels, and then reassembling them, the authors achieve sharper images than with standard DeblurGAN. This approach effectively addresses the challenge of spatially varying blur within a single image.
What Problem Does This Paper Address?
Motion blur is a pervasive issue in photography, especially with mobile devices, caused by the relative movement between the camera and the subject during image capture. This blur is often not uniform across the entire image; some areas might be heavily blurred, while others are relatively sharp. Traditional deblurring methods, including advanced deep learning models like DeblurGAN, often struggle with this spatial variability. Applying a single deblurring process to the entire image can lead to over-deblurring in already sharp areas or insufficient deblurring in heavily blurred regions. This paper aims to tackle this challenge by proposing a method that accounts for and adapts to the varying degrees of blur within an image.
Methods
The core innovation is the "Split-and-Merge" algorithm. It builds upon the DeblurGAN framework, which uses a Generative Adversarial Network (GAN) for deblurring. The process involves several key steps:
- Resizing and Initial Blur Measurement: The input blurry image is resized to a standard dimension (e.g., 512x512). The blur degree of this initial image is measured using the total variance of the Laplacian, a metric referred to as the "BDL_value." A threshold (e.g., 2.5) is set to determine if an image or its parts are considered sufficiently sharp.
- Splitting: The image is then divided into smaller, equal-sized parts (e.g., four 256x256 segments).
- Part-wise Blur Measurement and Deblurring: The BDL_value is calculated for each individual part. If a part's BDL_value is below the sharpness threshold, it is deblurred using the trained DeblurGAN generator. This deblurring process is applied iteratively to each part until its BDL_value reaches the acceptable threshold. Crucially, the number of deblurring iterations can vary for each part, allowing for adaptive deblurring. If a part remains below the threshold after a certain number of iterations (e.g., 5), it is deblurred a fixed number of times, and that result is used.
- Merging: Once all parts have been deblurred to an acceptable level, they are reassembled to form the final, sharper image.
This method leverages the DeblurGAN's ability to generate perceptually convincing deblurred images while adding a layer of adaptive processing tailored to local blur characteristics.
Datasets
The paper uses the GOPRO dataset for training and evaluation. This dataset is described as having a "light version" (9GB) and a "complete version" (35GB), containing approximately 2103 pairs of artificially blurred and real sharp images in 720p resolution. The authors specifically mention using the light version for training and testing DeblurGAN.
Key Insights and Ideas
- Spatially Varying Blur is a Major Hurdle: The paper highlights that uniform deblurring across an entire image is suboptimal when blur levels differ significantly within that image.
- Adaptive Deblurring is Key: By splitting the image and applying deblurring iteratively based on local blur measurements, the algorithm can achieve better results than a one-size-fits-all approach.
- BDL_value as a Practical Metric: The use of the total variance of the Laplacian (BDL_value) provides a computationally efficient and effective way to quantify blur and guide the adaptive deblurring process.
- GANs for Local Detail Enhancement: DeblurGAN's strength in generating realistic textures is effectively utilized at a local level within the split parts.
Key Results
The paper presents both quantitative and qualitative results.
- Quantitative: Table 1 shows comparisons of BDL values for 20 images. For all 20 images, the "Split-and-Merge" algorithm resulted in a larger BDL_value (indicating less blur) compared to direct deblurring of the whole image. For example, for one image, the direct deblurring BDL value was 2.2689, while the Split-and-Merge result achieved 10.2795.
- Qualitative: Figure 5 provides a visual comparison. The right column (Split-and-Merge) shows noticeably sharper details (e.g., patterns on clothing, decorations, leaves) compared to the middle column (direct deblurring). The authors describe specific improvements in texture clarity and resolution.
Limitations and Future Work
- Limitations:
- The paper acknowledges that even with the Split-and-Merge approach, some parts might still be below the threshold after a fixed number of deblurring iterations. In such cases, the result after the maximum allowed iterations is used.
- The paper does not explicitly discuss the computational overhead introduced by the splitting and merging process compared to direct deblurring.
- Future Work: Not explicitly discussed in the paper. However, potential future directions could include exploring more sophisticated splitting strategies, investigating alternative blur measurement metrics, or optimizing the merging process for seamless integration.
Summary updated