Abstract
Night photography is frequently compromised by the simultaneous presence of low-light conditions and motion blur. VQCNIR (Vector-Quantized Codebook for Night Image Restoration) addresses these challenges by utilizing a pre-trained VQ codebook as a high-quality image prior. Unlike traditional end-to-end models that learn from scratch, VQCNIR treats restoration as a matching and fusion task. It incorporates an Adaptive Illumination Enhancement Module (AIEM) to align input lighting with the codebook and Deformable Bi-directional Cross-Attention (DBCA) to correct geometric distortions. Evaluations on the LOL-Blur dataset demonstrate that VQCNIR significantly outperforms current state-of-the-art methods in both quantitative accuracy and perceptual quality.
Introduction
Capturing clear images in low-light environments remains a significant hurdle for modern imaging systems. To compensate for insufficient light, long exposure times are often employed, which introduces motion blur from camera shake or moving subjects. Existing deep learning approaches often struggle with this "double trouble," either failing to generalize to complex night scenes or producing unnatural artifacts due to a lack of robust image priors. VQCNIR proposes a data-driven solution that leverages a "memory bank" of high-quality features to guide the restoration process, ensuring results are both sharp and naturally lit.
Methodology
The VQCNIR framework is built around a Vector-Quantized (VQ) codebook derived from a VQGAN, serving as a library of sharp texture patches. The architecture consists of several specialized components designed to bridge the gap between degraded inputs and high-quality priors:
- Adaptive Illumination Enhancement Module (AIEM): This module uses Illumination Mutual Attention Convolution (IMAConv) to adjust the lighting of input features, ensuring they are compatible with the high-quality entries in the codebook environment.
- Deformable Bi-directional Cross-Attention (DBCA): To handle spatial distortions from motion blur, DBCA employs deformable convolutions to precisely align codebook features with the input image's distorted structures.
- Parallel Decoders: The system utilizes a fixed pre-trained decoder (Decoder G) for high-quality anchor features and a primary restorer (Decoder D) to fuse input data with the learned priors.
Results and Discussion
VQCNIR was rigorously tested on the LOL-Blur dataset, which includes 10,200 synthetic training pairs and a Real-LOL-Blur collection for testing generalization. Quantitatively, it achieved a PSNR of 27.79 dB, a 2.05 dB improvement over the previous state-of-the-art, LEDNet. Perceptually, its LPIPS score of 0.096 represents a nearly 2.3x improvement in visual fidelity compared to LEDNet (0.224).
On real-world data, VQCNIR consistently produced the highest scores in non-reference metrics such as MUSIQ (51.04) and NRQM (7.064), confirming its ability to generate visually pleasing and sharp results in actual night scenes. The model avoids the common "hallucination" problem of AI by using bi-directional cross-attention to ensure added details remain faithful to the original scene.
Conclusion
By combining learned image priors with adaptive alignment modules, VQCNIR sets a new standard for night image restoration. While the model shows immense promise, future work will address potential structural hallucinations in cases of extreme blur and optimize the computational efficiency of the cross-attention mechanisms for real-time mobile deployment.
Summary updated