Deep Joint Unrolling for Deblurring and Low-Light Image Enhancement (JUDE)

Tu Vo, Chan Y. ParkWACV 2025 · OpenReview

TL;DR

This paper introduces JUDE, a novel deep learning architecture that tackles the challenging problem of simultaneously enhancing low-light images and deblurring them. By unrolling a model-based optimization framework inspired by physical image properties, JUDE achieves state-of-the-art results on both synthetic and real-world datasets, offering a more interpretable and robust solution than purely data-driven approaches.

What Problem Does This Paper Address?

Capturing clear, well-lit images at night is a persistent challenge. Long exposure times, often used to gather enough light, introduce motion blur from camera movement or scene dynamics. This results in images that are both dark and blurry, obscuring details and negatively impacting downstream computer vision tasks like object detection. Previous methods have largely treated low-light enhancement and deblurring as separate problems, leading to suboptimal results when applied to images suffering from both degradations. End-to-end deep learning models, while powerful, often act as "black boxes," making their behavior difficult to analyze and limiting their generalization capabilities, especially when dealing with complex, combined degradations.

Methods

JUDE leverages the power of "algorithm unrolling," a technique that bridges model-based and learning-based approaches. Instead of training a monolithic deep network, JUDE unrolls the iterative steps of a physics-inspired optimization algorithm into a series of network blocks. This approach allows the model to explicitly incorporate physical models of image formation (Retinex theory for illumination/reflectance decomposition and a blurring model) into its learning process.

The core of JUDE involves formulating a joint optimization problem to deblur the input image and decompose it into reflectance and illuminance components. This problem is then solved iteratively using the Augmented Lagrangian Method (ALM). Each iteration of the ALM is translated into a "block" within the JUDE network. Within these blocks, variables like reflectance and illuminance are updated using closed-form solutions derived from the optimization, while other variables and learned priors are handled by Convolutional Neural Networks (CNNs), specifically a ResUNet architecture.

Key modules include:

  • Kernel Estimation Module: A CNN that predicts the blur kernel and a weight map to handle spatially varying blur.
  • Optimization Module: Composed of multiple unrolled blocks, each mirroring an iteration of the ALM solver. This module iteratively refines the image, reflectance, and illuminance.
  • Illuminance Enhancement Module: A separate network that brightens the estimated illuminance map.
  • Reflectance Denoiser Module: A network that removes noise from the estimated reflectance map.

Finally, the enhanced illuminance and denoised reflectance are recombined to produce the final, sharp, and well-lit image.

Datasets

The paper evaluates JUDE on two primary datasets:

  • LOL-Blur [58]: A synthetic dataset containing 10,200 image pairs for training and 1,800 for testing, specifically designed for low-light and blur challenges.
  • Real-LOL-Blur [31]: A dataset of 1,354 real-world nighttime blurry images, offering a more challenging evaluation of generalization capabilities.

Key Insights and Ideas

  • Unrolling for Interpretability and Generalization: By unrolling a model-based optimization, JUDE gains interpretability. The explicit incorporation of physical models allows it to generalize better to unseen scenarios compared to purely data-driven black-box models.
  • Joint Optimization is Crucial: Treating low-light enhancement and deblurring as a single, coupled problem, rather than separate tasks, leads to significantly better results, especially in challenging real-world conditions.
  • Leveraging Physics with Deep Learning: The paper demonstrates a successful synergy between traditional image processing principles (Retinex theory, blurring models) and modern deep learning techniques, creating a powerful hybrid approach.
  • Data-Driven Regularizers: The use of CNNs (ResUNet) to learn regularization terms within the optimization framework allows the model to adapt to complex data priors, further improving performance.

Key Results

JUDE achieves state-of-the-art performance on both the LOL-Blur and Real-LOL-Blur datasets.

  • Quantitative: On the LOL-Blur dataset, JUDE significantly outperforms existing methods across PSNR, SSIM, and LPIPS metrics, achieving the highest PSNR (26.884) and SSIM (0.932). On the Real-LOL-Blur dataset, JUDE leads in subjective quality metrics like ARNIQA, CONTRIQUE, and LIQE, indicating superior perceptual quality.
  • Qualitative: Visual comparisons show that JUDE produces sharper, brighter, and more detailed images compared to other methods, effectively recovering details in challenging areas that other approaches struggle with. The results are visually pleasing and demonstrate robust recovery of intricate details and accurate illumination.

Limitations and Future Work

The authors acknowledge that JUDE's parameter count is a limitation. They suggest future work could focus on:

  • Exploring alternative kernel prediction models.
  • Substituting the CNNs within the data module with more efficient architectures.
  • Developing methods to further reduce the model's parameter count without compromising performance.
  • Enhancing the model's efficiency and adaptability to varying degrees of blur and illumination.

Summary updated

Abstract

Low-light and blurring issues are prevalent when capturing photos at night, often due to the use of long exposure to address dim environments. Addressing these joint problems can be challenging and error-prone if an end-to-end model is trained without incorporating an appropriate physical model. In this paper, we introduce JUDE, a Deep Joint Unrolling for Deblurring and Low-Light Image Enhancement, inspired by the image physical model. Based on Retinex theory and the blurring model, the low-light blurry input is iteratively deblurred and decomposed, producing sharp low-light reflectance and illuminance through an unrolling mechanism. Additionally, we incorporate various modules to estimate the initial blur kernel, enhance brightness, and eliminate noise in the final image. Comprehensive experiments on LOL-Blur and Real-LOL-Blur demonstrate that our method outperforms existing techniques both quantitatively and qualitatively.
BibTeX
@inproceedings{vo2025deep,
  author = {Tu Vo and Chan Y. Park},
  title = {{Deep Joint Unrolling for Deblurring and Low-Light Image Enhancement (JUDE)}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  url = {https://openreview.net/forum?id=XPRUm48mSY},
}
Figure