Abstract
Night photography often struggles with challenges like low light and blurring, stemming from dark environments and prolonged exposures. Current methods either disregard priors and directly fit end-to-end networks, leading to inconsistent illumination, or rely on unreliable handcrafted priors to constrain the network, thereby bringing greater error to the final result. We propose Clearer Night Image Restoration with Vector-Quantized Codebook (VQCNIR) to achieve remarkable and consistent restoration outcomes on real-world and synthetic benchmarks. To ensure the faithful restoration of details and illumination, we incorporate two essential modules: the Adaptive Illumination Enhancement Module (AIEM) and the Deformable Bi-directional Cross-Attention (DBCA) module. Extensive experiments validate the remarkable benefits of VQCNIR in enhancing image quality under low-light conditions, showcasing its state-of-the-art performance on both synthetic and real-world datasets.
Introduction
Capturing clear images at night presents a significant technical challenge. To gather sufficient light in dark environments, cameras require long exposure times, which inherently increases the risk of motion blur from camera shake or moving subjects. Restoring these images requires solving two distinct problems simultaneously: low-light enhancement and deblurring.
Existing solutions typically fall into two categories: end-to-end networks that often struggle with inconsistent lighting, or methods relying on handcrafted mathematical priors that frequently fail in complex real-world scenes. VQCNIR addresses these limitations by leveraging data-driven high-quality priors from a pre-trained Vector-Quantized (VQ) codebook, effectively "hallucinating" sharp textures while maintaining structural integrity.
Methodology
The core of VQCNIR is the use of a Vector-Quantized (VQ) Codebook, which acts as a high-resolution dictionary of image patches. The network encodes a blurry, dark image and identifies the closest matching "clean" patches in this dictionary to guide the reconstruction process.
To bridge the gap between degraded input features and high-quality codebook features, the framework introduces two novel components:
- Adaptive Illumination Enhancement Module (AIEM): This module adjusts the illumination of input features before they are matched against the codebook. By using inter-channel correlations to dynamically estimate enhancement curves, it ensures the codebook query is performed within the correct brightness range.
- Deformable Bi-directional Cross-Attention (DBCA): To address spatial misalignment caused by blur, DBCA utilizes a parallel decoder structure. One decoder processes high-quality codebook features, while the other focuses on the original input's structure. DBCA employs deformable convolutions to align these features, fusing sharp textures from the codebook into the structural framework of the original image.
Results and Discussion
VQCNIR was primarily evaluated on the LOL-Blur dataset, which contains over 10,000 image pairs of low-light, blurry images. The model sets a new benchmark for night image restoration across several metrics:
- Quantitative Performance: On the synthetic LOL-Blur benchmark, VQCNIR achieved a PSNR of 27.79 dB, outperforming the previous state-of-the-art (LEDNet) by a significant 2.05 dB margin.
- Perceptual Quality: In real-world testing, the model excelled in non-reference metrics such as MUSIQ and NIQE, which align more closely with human perception of sharpness and naturalness than standard pixel-wise comparisons.
- Visual Fidelity: Qualitatively, the model avoids common artifacts like "ghosting" or over-smoothing, producing images that appear as though they were captured with professional lighting and stabilization.
Key insights from the study suggest that learned data-driven priors are superior to rigid handcrafted ones. Furthermore, aligning illumination is a critical prerequisite for effective codebook matching, and deformable convolutions are essential for handling the spatial smearing inherent in blurry images.
Conclusion
VQCNIR demonstrates that high-quality, data-driven priors can significantly improve the restoration of night images. While the model is highly effective, performance can degrade under extreme conditions of noise or total darkness, where codebook matching may lead to hallucinated artifacts. Future research will focus on increasing robustness to extreme noise and optimizing the architecture for real-time performance on mobile devices.
Summary updated