Zero-Reference Joint Low-Light Enhancement and Deblurring via Visual Autoregressive Modeling with VLM-Derived Modulation

Wei Dong, Han Zhou, Junwei Lin, Jun ChenarXiv 2025 · DOI · arXiv

Abstract

Capturing high-quality images in low-light environments is a persistent challenge in computer vision, typically requiring a trade-off between exposure time and motion blur. Traditional methods often address enhancement and deblurring as decoupled tasks, frequently relying on scarce paired datasets. We present VAR-LIDE, a unified, zero-reference framework that leverages pre-trained Visual Autoregressive (VAR) models and Vision-Language Models (VLMs) to simultaneously enhance visibility and remove blur. By utilizing VLM-derived perceptual priors for adaptive modulation, VAR-LIDE achieves state-of-the-art performance on benchmark datasets without requiring ground-truth supervision.

Introduction

Real-world low-light photography is plagued by a "triple threat": insufficient visibility, high sensor noise, and motion blur. Existing solutions often struggle with the "pipeline problem," where sequential enhancement and deblurring steps amplify noise or destroy structural cues. Furthermore, the reliance on paired training data—identical scenes captured in both low-light/blurry and high-light/sharp conditions—limits the scalability and real-world applicability of supervised models.

Example of a challenging low-light input image exhibiting low visibility and potential blur.

Methodology

VAR-LIDE employs a pre-trained Visual Autoregressive (VAR) model as its generative backbone. Unlike standard transformers that predict tokens in a 1D sequence, VAR predicts images scale-by-scale (from low to high resolution), naturally preserving structural fidelity. The framework integrates three key components:

  • VLM-Informed Conditioning Module (VICM): Uses a VLM to score input visibility, guiding an iterative curve-estimation process to brighten the image optimally for the VAR model without overexposure.
  • Spatial-Frequency RoPE (SF-RoPE): A content-aware positional encoding that fuses spatial coordinates with frequency-domain phase information (from FFT) to better resolve structures smeared by blur.
  • VLM-Guided Recursive Phase Modulation (VGPM): Refines the image's Fourier phase based on VLM-derived blur scores to eliminate "ghosting" artifacts and sharpen edges.

The system is trained in a zero-reference manner, optimizing internal losses such as exposure control and structural entropy rather than comparing against ground-truth images.

Results and Discussion

The model was evaluated on the LOLBlur and Real-LOLBlur datasets. Quantitatively, VAR-LIDE outperformed traditional cascaded pipelines (e.g., EnlightenGAN + Blur2Blur) by over 5 dB in PSNR on the LOLBlur benchmark. Perceptually, it achieved superior scores in no-reference metrics like NIQE and CLIPIQA, indicating more natural restoration results.

Restoration result showing the effect of iterative adjustment guided by VLM priors.

Despite being unsupervised, VAR-LIDE demonstrates performance competitive with fully supervised state-of-the-art models like LEDNet. The inherent ability of pre-trained VAR models to suppress noise, combined with VLM-guided modulation, provides a robust solution for "in the wild" low-light restoration.

Conclusion

VAR-LIDE represents a significant advancement in unsupervised image restoration. By framing joint enhancement and deblurring as a guided generative task, it bypasses the need for paired data while maintaining higher efficiency than diffusion-based methods. Future work will focus on optimizing the architecture for mobile deployment and improving performance in highly cluttered scenes.

Summary updated

Abstract

Real-world dark images commonly exhibit not only low visibility and contrast but also complex noise and blur, posing significant restoration challenges. Existing methods often rely on paired data or fail to model dynamic illumination and blur characteristics, leading to poor generalization. To tackle this, we propose a generative framework based on visual autoregressive (VAR) modeling, guided by perceptual priors from the vision-language model (VLM). Specifically, to supply informative conditioning cues for VAR models, we deploy an adaptive curve estimation scheme to modulate the diverse illumination based on VLM-derived visibility scores. In addition, we integrate dynamic and spatial-frequency-aware Rotary Positional Encodings (SF-RoPE) into VAR to enhance its ability to model structures degraded by blur. Furthermore, we propose a recursive phase-domain modulation strategy that mitigates blur-induced artifacts in the phase domain via bounded iterative refinement guided by VLM-assessed blur scores. Our framework is fully unsupervised and achieves state-of-the-art performance on benchmark datasets.
BibTeX
@misc{dong2025zero,
  author = {Wei Dong and Han Zhou and Junwei Lin and Jun Chen},
  title = {{Zero-Reference Joint Low-Light Enhancement and Deblurring via Visual Autoregressive Modeling with VLM-Derived Modulation}},
  year = {2025},
  eprint = {2511.18591},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.18591},
}
Figure